Author: Mike Sullivan, Senior Software Engineer, Telcordia Technologies
Overview
JBoss Application Server's Transaction service can recover 2-phase commit transactions from a system crash if a recovery module is configured for each resource. If one is using 2-phase commit with Websphere MQ unrecovered transactions in Websphere MQ will occupy space if not recovered and can cause performance problems. This article shows how to configure a JBoss XARecovery module for Websphere MQ v7 and how to verify XA transactions are recovered from WebSphere MQ’s store.
System Specifications
These instructions were based upon the following configuration:
- JBoss 5.1.0 EAP
- Websphere MQ v7 (7.1.0.5)
- IBM Websphere MQ JCA [see 1] and [see 2]
Additional Components Needed
In order to setup a XARecovery module for Websphere MQ within the JBoss 5.1.0 application server one needs to acquire the following artifacts:
- IBM Support Pac – ME01 [see 5]
- Byteman – Java testing tool. This is needed to cause an XA failure [see 6]
The IBM “Websphere MQ Using Java” manual should be referred to understand the JMSAdmin tool [see 3].
Create the WebSphere MQ JMS Connection Factory Objects
Before creating the Websphere MQ JMS objects, install the ME01 Support Pac and place the jar in a JBoss class path directory.
The WebSphere MQ JMS administration tool (JMSAdmin) enables administrators to define JNDI names in Websphere MQ. The XARecovery module for Websphere MQ will need to use a Websphere MQ JNDI to get a XA connection factory in order to initiate the XARecovery.
The first task is to create a customized JMSAdmin.config (see /opt/mqm/java/bin/JMSAdmin.config). Let’s call it WMQ_jms.cfg and add the following lines to it:
INITIAL_CONTEXT_FACTORY=com.ibm.mq.jms.context.WMQInitialContextFactory PROVIDER_URL=192.4.210.68:1414/SYSTEM.DEF.SVRCONN
The JDNI input for JMSAdmin is (add this to file xaqcf_def.scp):
def xaqcf(WNPMQMXACF) qmgr(WNPMQM) tran(client) chan(SYSTEM.DEF.SVRCONN) + host(192.4.210.68) port(1414)
with the following key words explained:
- xaqcf – key word to say this is an XA queue connection factory object
- WNPMQMXA – the name of the queue factory, administrators choice
- tran(client) – the transport should be client
- chan(SYSTEM.DEF.SVRCONN) – Websphere MQ server connection channel
- host(192.4.210.68) port(1414) – the IP and PORT where the Websphere MQ server is running
The following is a sample of running the JMSAdmin tool followed by a display of the definition the xaqcf_def.scp file as re-directed input. The NameNotFoundException is from the delete operation if this was run the first time.
$ cat JMSAdmin.ksh CLASSPATH=$CLASSPATH:/opt/mqm/java/lib/jms.jar CLASSPATH=$CLASSPATH:/opt/mqm/java/lib/com.ibm.mq.jar CLASSPATH=$CLASSPATH:/opt/mqm/java/lib/com.ibm.mqjms.jar CLASSPATH=$CLASSPATH:/opt/mqm/java/lib/jta.jar CLASSPATH=$CLASSPATH:/opt/mqm/java/lib/connector.jar CLASSPATH=$CLASSPATH:/opt/mqm/java/lib/jndi.jar CLASSPATH=$CLASSPATH:/opt/mqm/java/lib/providerutil.jar CLASSPATH=$CLASSPATH:/opt/mqm/java/lib/fscontext.jar CLASSPATH=$CLASSPATH:/opt/mqm/java/lib/com.ibm.mqjms.jar CLASSPATH=$CLASSPATH:/opt/mqm/java/lib/mqcontext.jar /opt/mqm/java/bin/JMSAdmin -v -cfg $PWD/WMQ_jms.cfg < xaqcf_def.scp $ ./JMSAdmin.ksh Licensed Materials - Property of IBM 5724-H72, 5655-R36, 5724-L26, 5655-L82 (c) Copyright IBM Corp. 2008 All Rights Reserved. US Government Users Restricted Rights - Use, duplication or disclosure restricted by GSA ADP Schedule Contract with IBM Corp. Starting Websphere MQ classes for Java(tm) Message Service Administration Initializing JNDI Context... INITIAL_CONTEXT_FACTORY: com.ibm.mq.jms.context.WMQInitialContextFactory PROVIDER_URL: 192.4.210.68:1414/SYSTEM.DEF.SVRCONN Done. Welcome to the WebSphere MQ JMS administration tool command-line interface InitCtx> Binding non-administerable or not found javax.naming.NameNotFoundException: Object WNPMQMXACF not found on queue manager WNPMQM InitCtx> InitCtx> Stopping Websphere MQ classes for Java(tm) Message Service Administration $
Here’s the query of the created connection factory JNDI object:
$ cat ./JMSAdminQuery.ksh CLASSPATH=$CLASSPATH:/opt/mqm/java/lib/jms.jar CLASSPATH=$CLASSPATH:/opt/mqm/java/lib/com.ibm.mq.jar CLASSPATH=$CLASSPATH:/opt/mqm/java/lib/com.ibm.mqjms.jar CLASSPATH=$CLASSPATH:/opt/mqm/java/lib/jta.jar CLASSPATH=$CLASSPATH:/opt/mqm/java/lib/connector.jar CLASSPATH=$CLASSPATH:/opt/mqm/java/lib/jndi.jar CLASSPATH=$CLASSPATH:/opt/mqm/java/lib/providerutil.jar CLASSPATH=$CLASSPATH:/opt/mqm/java/lib/fscontext.jar CLASSPATH=$CLASSPATH:/opt/mqm/java/lib/com.ibm.mqjms.jar CLASSPATH=$CLASSPATH:/opt/mqm/java/lib/mqcontext.jar /opt/mqm/java/bin/JMSAdmin -v -cfg $PWD/WMQ_jms.cfg $ ./JMSAdminQuery.ksh Licensed Materials - Property of IBM 5724-H72, 5655-R36, 5724-L26, 5655-L82 (c) Copyright IBM Corp. 2008 All Rights Reserved. US Government Users Restricted Rights - Use, duplication or disclosure restricted by GSA ADP Schedule Contract with IBM Corp. Starting Websphere MQ classes for Java(tm) Message Service Administration Initializing JNDI Context... INITIAL_CONTEXT_FACTORY: com.ibm.mq.jms.context.WMQInitialContextFactory PROVIDER_URL: 192.4.210.68:1414/SYSTEM.DEF.SVRCONN Done. Welcome to the WebSphere MQ JMS administration tool command-line interface InitCtx> display xaqcf(WNPMQMXACF) ASYNCEXCEPTION(ALL) CCSID(819) CHANNEL(SYSTEM.DEF.SVRCONN) CLIENTRECONNECTOPTIONS(ASDEF) CLIENTRECONNECTTIMEOUT(1800) COMPHDR(NONE ) COMPMSG(NONE ) CONNECTIONNAMELIST(192.4.210.68(1414)) CONNOPT(STANDARD) FAILIFQUIESCE(YES) HOSTNAME(192.4.210.68) LOCALADDRESS() MAPNAMESTYLE(STANDARD) MSGBATCHSZ(10) MSGRETENTION(YES) POLLINGINT(5000) PORT(1414) PROVIDERVERSION(UNSPECIFIED) QMANAGER(WNPMQM) RESCANINT(5000) SENDCHECKCOUNT(0) SHARECONVALLOWED(YES) SSLFIPSREQUIRED(NO) SSLRESETCOUNT(0) SYNCPOINTALLGETS(NO) TARGCLIENTMATCHING(YES) TEMPMODEL(SYSTEM.DEFAULT.MODEL.QUEUE) TEMPQPREFIX() TRANSPORT(CLIENT) USECONNPOOLING(YES) VERSION(7) WILDCARDFORMAT(TOPIC_ONLY) InitCtx> end Stopping Websphere MQ classes for Java(tm) Message Service Administration
Define the JBoss Deployment for the XARecovery Module
With the Websphere MQ JMS JNDI defined, JBoss deployment information can be defined to setup the XARecovery module.
Two JBoss configuration files are needed to define the XARecovery module:
- New JMSProviderLoader ds.xml file to point to the Websphere MQ JNDI
- This file should be placed in one of JBoss’ deployment folders
- Update server/<profile>/conf/jbossts-properties.xml to include this new XARecovery module
The following is a sample ds.xml file, wsmq-jmsprovider-ds.xml:
<server> <mbean code="org.jboss.jms.jndi.JMSProviderLoader" name="jboss.jms:service=JMSProviderLoader,name=WSMQJmsWNPMQMProvider"> <attribute name="ProviderName">WSMQJmsWNPMQMProvider</attribute> <attribute name="ProviderAdapterClass"> org.jboss.jms.jndi.JNDIProviderAdapter </attribute> <attribute name="FactoryRef">WNPMQMXACF</attribute> <attribute name="QueueFactoryRef">MQQueueConnectionFactory</attribute> <attribute name="TopicFactoryRef">MQTopicConnectionFactory</attribute> <attribute name="Properties"> java.naming.factory.initial=com.ibm.mq.jms.context.WMQInitialContextFactory java.naming.factory.url.pkgs=com.ibm.mq.jms.naming java.naming.provider.url=192.4.210.68:1414/SYSTEM.DEF.SVRCONN java.naming.security.authentication=none </attribute> </mbean> </server>
The properties information should match the Websphere MQ JNDI entry created and the FactoryRef should match xaqcf name from the JMSAdmin input file.
Add to the server/<profile>/conf/jbossts-properties.xml in the name="jta" properties section the JMSProviderLoader reference:
<property name="com.arjuna.ats.jta.recovery.XAResourceRecovery.WSMQWNPMQM" value="org.jboss.jms.server.recovery.MessagingXAResourceRecovery;java:/WSMQJmsWNPMQMProvider"/> </properties>
Notice the WSMQJmsWNPMQMProvider should match the name in the JMSProviderLoader definition in the *-ds.xml file.
Restart JBoss and Verify the XARecovery Module is Defined Correctly
After a successful JBoss restart with these definitions in place, and no deployment errors, if the JMSProviderLoader is unable to connect to Websphere MQ then the following WARN statement will appear periodically (i.e. it may appear before the startup is complete):
2011-06-10 10:44:08,707 WARN [loggerI18N] [com.arjuna.ats.internal.jta.recovery.xarecovery1] Local XARecoveryModule.xaRecovery got XA exception javax.transaction.xa.XAException: Error trying to connect to provider java:/WSMQJmsWNPMQMProvider, XAException.XAER_RMERR
Testing the XARecovery Module
Testing an XA failure is difficult to do unless one uses something like Byteman to inject behavior change into the XA process.
This XA failure test uses byteman to inject a failure based upon a simple countdown capability in byteman.
Byteman Script
RULE trace XA prepare
CLASS com.ibm.mq.connector.xa.XARWrapper
METHOD prepare
AT EXIT
IF TRUE
DO traceln("*** at exit XAResource " + $0 + " prepare(" + $1 + ")"), createCountDown("xacounter",10)
ENDRULE
RULE trace XA commit
CLASS com.ibm.mq.connector.xa.XARWrapper
METHOD commit
AT ENTRY
IF countDown("xacounter")
DO traceln("killing JVM"), killJVM()
ENDRULE
This script uses a counter “xacounter” to count down from 10 the number of prepare/commit calls in Websphere MQ. Once the corresponding commit has reached 0 then the MQ commit will kill the JVM at entry, thereby causing a lingering prepare, at least one, to be left in the system.
Byteman and this script should be loaded with the JBoss JVM start up using these options:
-javaagent:/bytemandir/byteman.jar=script:/vendor/jboss/jboss-as/bin/xa_byteman.txt
Testing XA Recovery
With byteman and all the proper configuration and definitions in place, now we can create an XA failure.
- Have JBoss shutdown and Websphere MQ idle
- Checking Websphere MQ that no prepared transactions are lingering in MQ
Use the following command to ensure there are no prepared transaction:$ /opt/mqm/bin/dspmqtrn -m WNPMQM There are no matching prepared transactions.
- JBoss should be started with the above byteman script and leave the system idle, i.e. not doing any MQ work
- After JBoss is up inject XA MQ transactions for up to 10 or more transactions to trip the byteman countdown to go from 10 to 0.
- The server .log should have the following statements and the JVM should be shown as killed:
- 2011-06-10 13:55:58,342 INFO [STDOUT] killing JVM
- Check that Websphere MQ now has prepared transactions lingering in MQ:
$ /opt/mqm/bin/dspmqtrn -m WNPMQM AMQ7056: Transaction number 0,23113369 is in-doubt. XID: formatID 131075, gtrid_length 29, bqual_length 27 gtrid [312D2D33666662326463663A383234343A34646632353934323A343839] bqual [2D33666662326463663A383234343A34646632353934323A346663]
- Now undo the byteman countdown logic by removing the countdown and kill statements. They can be replaced by just trace statements.
- Restart JBoss
- Once JBoss has started, check the MQ prepared transactions again and one should see there are none
$ /opt/mqm/bin/dspmqtrn -m WNPMQM There are no matching prepared transactions
References
- Integration using the IBM Websphere MQ JMS Resource Adapter (RECOMMENDED WAY!!)
- Installing and configuring WebSphere MQ resource adapter on JBoss Application Server
- Websphere MQ Using Java – reference for JMSAdmin tool
- IBM Tutorial on Websphere MQ JMS with sample code if you want to verify the MQ JNDI is working:
- IBM Support Pack: ME01 from:
- http://www-01.ibm.com/support/docview.wss?uid=swg24004684&wv=1
- Note: ME01 mentions that M0SB is needed, however, on a system with MQ V7 installed, which includes PCF Java support, you should no longer need MS0B. The PCF support in MQ V7 provides the same PCF Java API as MS0B
- Byteman – Java testing tool. This is needed to cause an XA failure
Comments