Sporadic JBoss Startup failure using WebSphere MQ resource adapter

[sunryze]

We are currently facing sporadic JNDI lookup exceptions during JBoss startup that are related to a WebSphere MQ resource adapter. The connection factory published via JNDI by the adapter is not found during the webservice startup of the single EAR deployed in the standalone JBoss. This happens only occasionally, we cannot reliably reproduce the error. Some developers do not face the problem locally at all.

We have connected our standalone JBoss AS 7.2 to a WebSphere MQ 7.0.1.3 queue manager using the resource adapter wmq.jmsra.rar provided by the

WebSphere MQ installation. We publish the resource adapter's ConnectionFactory via JNDI and do a Spring JNDI lookup of the factory in a WSDL endpoint definition. From time to time we see the following exception trace:

2013-08-05 09:57:55,952 ERROR [org.springframework.web.context.ContextLoader] BK-'' Context initialization failed: org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'serviceEndpointJms': Invocation of init method failed; nested exception is javax.xml.ws.WebServiceException: org.springframework.jndi.JndiLookupFailureException: JndiObjectTargetSource failed to obtain new target object; nested exception is javax.naming.NameNotFoundException: jms/MQCF -- service jboss.naming.context.java.jboss.jms.MQCF...
Caused by: javax.xml.ws.WebServiceException: org.springframework.jndi.JndiLookupFailureException: JndiObjectTargetSource failed to obtain new target object; nested exception is javax.naming.NameNotFoundException: jms/MQCF -- service jboss.naming.context.java.jboss.jms.MQCF
...
Caused by: org.springframework.jndi.JndiLookupFailureException: JndiObjectTargetSource failed to obtain new target object; nested exception is javax.naming.NameNotFoundException: jms/MQCF -- service jboss.naming.context.java.jboss.jms.MQCF
...
Caused by: javax.naming.NameNotFoundException: jms/MQCF -- service jboss.naming.context.java.jboss.jms.MQCF
    at org.jboss.as.naming.ServiceBasedNamingStore.lookup(ServiceBasedNamingStore.java:103)
    at org.jboss.as.naming.NamingContext.lookup(NamingContext.java:197)

Here is the relevant excerpt from standalone.xml:

 <subsystem xmlns="urn:jboss:domain:resource-adapters:1.1">
  <resource-adapters>
    <resource-adapter>
      <archive>wmq.jmsra.rar</archive>
      <transaction-support>NoTransaction</transaction-support>
      <connection-definitions>
        <connection-definition class-name="com.ibm.mq.connector.outbound.ManagedConnectionFactoryImpl" jndi-name="java:jboss/jms/MQCF" enabled="true" pool-name="MQCF">
          <config-property name="port">1415</config-property>
          <config-property name="hostName">1.2.3.4</config-property> <!-- IP address, modified for security-->
          <config-property name="channel">CLIENT_SOMETHING</config-property> <!-- modified for security-->
          <config-property name="transportType">CLIENT</config-property>
          <config-property name="queueManager">QMGR_NAME</config-property> <!-- modified for security-->
        </connection-definition>
      </connection-definitions>
    </resource-adapter>
  </resource-adapters>
</subsystem>

And here is a snippet of the applicationContext.xml defining the webservice endpoint:

 <jaxws:endpoint id="serviceEndpointJms" xmlns:tns="http://namespace.example.com/" implementor="#webService.Service" wsdlLocation="wsdl/Service.wsdl"
        serviceName="tns:Service" endpointName="tns:ServiceSoap"
        transportId="http://cxf.apache.org/transports/jms" address="jms://">
  <jaxws:features>
    <bean class="org.apache.cxf.transport.jms.JMSConfigFeature">
      <property name="jmsConfig">
        <bean class="org.apache.cxf.transport.jms.JMSConfiguration">
           <property name="connectionFactory">
             <jee:jndi-lookup jndi-name="java:jboss/jms/MQCF" lookup-on-startup="false"
                proxy-interface="javax.jms.ConnectionFactory" resource-ref="false" />
           </property>
           <property name="targetDestination" value="${REQUEST_QUEUE_NAME}"/>
           <property name="replyDestination" value="${REPLY_QUEUE_NAME}"/>
           <property name="wrapInSingleConnectionFactory" value="false" />
        </bean>
      </property>
    </bean>
  </jaxws:features>

We suspect that this issue is caused by a wrong startup ordering, maybe due to concurrent deployment in JBoss. Is there a way to enforce that the resource adapter is completely started up (and all its JNDI objects are published) before any EAR gets deployed? We already tried declaring a dependency to wmq.jmsra.rar in our EAR's META-INF\jboss-all.xml and META-INF\jboss-deployment-structure.xml, but to no avail.

Or is there any other way to fix this problem?

Many thanks!

[sfcoy]

Try this:

1. Add a resource-ref element to your web.xml:

   {code:xml}<resource-ref>
           <description>WebSphere MQ Connection Factory</description>
           <res-ref-name>jms/MQCF</res-ref-name>
           <res-type>javax.jms.QueueConnectionFactory</res-type>
           <res-auth>Container</res-auth>
           <lookup-name>java:jboss/jms/MQCF</lookup-name>
   </resource-ref> {code}

2. Change your Spring configuration:

   {code:xml}<jee:jndi-lookup jndi-name="jms/MQCF" lookup-on-startup="false"
                   proxy-interface="javax.jms.ConnectionFactory" resource-ref="true" />
    {code}

JBoss will see the resource-ref and figure out that it must wait for the dependent resource to become available before proceeding.

Note that this configures Spring to perform a JNDI lookup on java:comp/env/jms/MQCF.

[sunryze]

Thanks for the quick reply! I'm a bit confused by your last comment: do you mean that Spring will not find the ConnectionFactory java:jboss/jms/MQCF, as it looks for it in another namespace, java:comp/env?