I've been working with JBoss 3.2.4 HA JMS and there have been some issues that I've run into that I can't figure out.

I've read the FAQ and done extensive searches pertaining to the exceptions that I'm seeing and what could cause them. I've taken a step-by-step approach to trying to figure out what the failure point is, but it appears that there is a timing issue.

My test deployment consists of two message driven beans that are deployed to server/all/deploy-hasingleton (in a jar), a destinations service xml file (also deployed to deploy-hasingleton) and a sar containing an MBean that allows me to send to my queue.

On startup I see a message from the EjbDeployer, indicating that it is deploying my EJB jar (failovertest-server.jar):

2004-06-23 09:44:30,799 DEBUG [org.jboss.deployment.MainDeployer] Done with create step of deploying uil2-service.xml
2004-06-23 09:44:30,799 DEBUG [org.jboss.deployment.MainDeployer] create step for deployment file:/C:/tools/jboss-3.2.4/server/all/deploy-hasingleton/jms/failovertest-server.jar
2004-06-23 09:44:30,799 DEBUG [org.jboss.ejb.EJBDeployer] create, failovertest-server.jar
2004-06-23 09:44:30,989 DEBUG [org.jboss.ejb.EJBDeployer] Verifying file:/C:/tools/jboss-3.2.4/server/all/deploy-hasingleton/jms/failovertest-server.jar
2004-06-23 09:44:31,039 DEBUG [org.jboss.ejb.EJBDeployer.verifier] Bean checked: FailoverTestMDB: Verified.
2004-06-23 09:44:31,049 DEBUG [org.jboss.ejb.EJBDeployer.verifier] Bean checked: FailoverTestStatusMDB: Verified.
2004-06-23 09:44:31,049 DEBUG [org.jboss.mx.modelmbean.ModelMBeanInvoker] No persistence-manager descriptor found, null persistence will be used
2004-06-23 09:44:31,049 DEBUG [org.jboss.jmx.adaptor.snmp.agent.SnmpAgentService] It's for me: javax.management.MBeanServerNotification: notificationType=JMX.mbean.registered source=JMImplementation:type=MBeanServerDelegate seq-no=187 time=1088009071049 message=null objectName=jboss.j2ee:module=failovertest-server.jar,service=EjbModule userData=null, handback:2147483647
2004-06-23 09:44:31,049 DEBUG [org.jboss.ejb.EJBDeployer] Deploying: file:/C:/tools/jboss-3.2.4/server/all/deploy-hasingleton/jms/failovertest-server.jar
2004-06-23 09:44:31,049 DEBUG [org.jboss.system.ServiceController] Creating service jboss.j2ee:module=failovertest-server.jar,service=EjbModule
2004-06-23 09:44:31,049 DEBUG [org.jboss.ejb.EjbModule] Creating jboss.j2ee:module=failovertest-server.jar,service=EjbModule
2004-06-23 09:44:31,089 DEBUG [org.jboss.ejb.EjbModule] createService, begin
2004-06-23 09:44:31,089 INFO [org.jboss.ejb.EjbModule] Deploying FailoverTestMDB
2004-06-23 09:44:31,109 DEBUG [org.jboss.ejb.EjbModule] Creating WebClassLoader of class org.jboss.web.WebClassLoader
2004-06-23 09:44:31,200 WARN [org.jboss.ejb.EjbModule] Could not load the org.jboss.resource.connectionmanager.CachedConnectionInterceptor interceptor for this container
javax.management.InstanceNotFoundException: jboss.jca:service=CachedConnectionManager is not registered.
at org.jboss.mx.server.registry.BasicMBeanRegistry.get(BasicMBeanRegistry.java:462)
at org.jboss.mx.server.MBeanServerImpl.getAttribute(MBeanServerImpl.java:433)
at org.jboss.resource.connectionmanager.CachedConnectionInterceptor.(CachedConnectionInterceptor.java:72)
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:39)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:27)
at java.lang.reflect.Constructor.newInstance(Constructor.java:274)
at java.lang.Class.newInstance0(Class.java:308)
at java.lang.Class.newInstance(Class.java:261)
at org.jboss.ejb.EjbModule.addInterceptors(EjbModule.java:857)
at org.jboss.ejb.EjbModule.initializeContainer(EjbModule.java:743)
at org.jboss.ejb.EjbModule.createMessageDrivenContainer(EjbModule.java:529)
at org.jboss.ejb.EjbModule.createContainer(EjbModule.java:496)
at org.jboss.ejb.EjbModule.createService(EjbModule.java:316)
at org.jboss.system.ServiceMBeanSupport.jbossInternalCreate(ServiceMBeanSupport.java:237)
at org.jboss.system.ServiceMBeanSupport.jbossInternalLifecycle(ServiceMBeanSupport.java:219)
[...deleted...]
2004-06-23 09:44:31,750 DEBUG [org.jboss.ejb.EjbModule] creating binding for FailoverTestMDB:singleton-message-driven-bean
2004-06-23 09:44:31,790 INFO [org.jboss.ejb.EjbModule] Deploying FailoverTestStatusMDB
2004-06-23 09:44:31,790 DEBUG [org.jboss.ejb.EjbModule] Creating WebClassLoader of class org.jboss.web.WebClassLoader
2004-06-23 09:44:31,811 WARN [org.jboss.ejb.EjbModule] Could not load the org.jboss.resource.connectionmanager.CachedConnectionInterceptor interceptor for this container
javax.management.InstanceNotFoundException: jboss.jca:service=CachedConnectionManager is not registered.
at org.jboss.mx.server.registry.BasicMBeanRegistry.get(BasicMBeanRegistry.java:462)
at org.jboss.mx.server.MBeanServerImpl.getAttribute(MBeanServerImpl.java:433)
at org.jboss.resource.connectionmanager.CachedConnectionInterceptor.(CachedConnectionInterceptor.java:72)
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:39)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:27)
at java.lang.reflect.Constructor.newInstance(Constructor.java:274)
at java.lang.Class.newInstance0(Class.java:308)
at java.lang.Class.newInstance(Class.java:261)
at org.jboss.ejb.EjbModule.addInterceptors(EjbModule.java:857)
at org.jboss.ejb.EjbModule.initializeContainer(EjbModule.java:743)
at org.jboss.ejb.EjbModule.createMessageDrivenContainer(EjbModule.java:529)
at org.jboss.ejb.EjbModule.createContainer(EjbModule.java:496)
at org.jboss.ejb.EjbModule.createService(EjbModule.java:316)
at org.jboss.system.ServiceMBeanSupport.jbossInternalCreate(ServiceMBeanSupport.java:237)
at org.jboss.system.ServiceMBeanSupport.jbossInternalLifecycle(ServiceMBeanSupport.java:219)
[...deleted...]
2004-06-23 09:44:32,552 DEBUG [org.jboss.ejb.EjbModule] creating binding for FailoverTestStatusMDB:singleton-message-driven-bean


Some time later, when the hasingleton deploys the DLQ tries to attach to this queue and generates the following exception:

2004-06-23 09:44:33,092 DEBUG [org.jboss.system.ServiceController] starting service jboss.j2ee:jndiName=local/FailoverTestMDB,service=EJB
2004-06-23 09:44:33,092 DEBUG [org.jboss.ejb.MessageDrivenContainer] Starting jboss.j2ee:jndiName=local/FailoverTestMDB,service=EJB
2004-06-23 09:44:33,092 DEBUG [org.jboss.ejb.MessageDrivenContainer] Begin java:comp/env for EJB: FailoverTestMDB
2004-06-23 09:44:33,092 DEBUG [org.jboss.ejb.MessageDrivenContainer] TCL: org.jboss.util.loading.DelegatingClassLoader@15200be
2004-06-23 09:44:33,102 DEBUG [org.jboss.ejb.MessageDrivenContainer] Binding env resource: jms/FailoverTestStatus to JDNI ENC as: topic/FailoverTestStatus
2004-06-23 09:44:33,102 DEBUG [org.jboss.ejb.MessageDrivenContainer] End java:comp/env for EJB: FailoverTestMDB
2004-06-23 09:44:33,102 DEBUG [org.jboss.ejb.plugins.local.BaseLocalProxyFactory] FailoverTestMDB cannot be Bound, doesn't have local and local home interfaces
2004-06-23 09:44:33,102 DEBUG [org.jboss.system.ServiceController] starting service jboss.j2ee:binding=singleton-message-driven-bean,jndiName=local/FailoverTestMDB,plugin=invoker,service=EJB
2004-06-23 09:44:33,102 DEBUG [org.jboss.ejb.plugins.jms.JMSContainerInvoker] Starting jboss.j2ee:binding=singleton-message-driven-bean,jndiName=local/FailoverTestMDB,plugin=invoker,service=EJB
2004-06-23 09:44:33,102 DEBUG [org.jboss.ejb.plugins.jms.JMSContainerInvoker] Initializing
2004-06-23 09:44:33,102 DEBUG [org.jboss.ejb.plugins.jms.JMSContainerInvoker] Looking up provider adapter: java:/DefaultJMSProvider
2004-06-23 09:44:33,102 DEBUG [org.jboss.ejb.plugins.jms.JMSContainerInvoker] Provider adapter: org.jboss.jms.jndi.JNDIProviderAdapter@5ed853
2004-06-23 09:44:33,112 DEBUG [org.jboss.ejb.plugins.jms.DLQHandler] Creating DLQHandler
2004-06-23 09:44:33,803 DEBUG [org.jboss.ha.framework.server.HAPartitionImpl] dests=[], method_call=HAJNDI.lookupLocally(XAConnectionFactory), mode=2, timeout=60000
2004-06-23 09:44:33,803 DEBUG [org.jboss.ha.framework.server.HAPartitionImpl] destination list is non-null and empty: no need to send message
2004-06-23 09:44:33,813 ERROR [org.jboss.ejb.plugins.jms.DLQHandler] Initialization failed DLQHandler
javax.jms.JMSException: Error creating the dlq connection: XAConnectionFactory
at org.jboss.ejb.plugins.jms.DLQHandler.createService(DLQHandler.java:171)
at org.jboss.system.ServiceMBeanSupport.jbossInternalCreate(ServiceMBeanSupport.java:237)
at org.jboss.system.ServiceMBeanSupport.create(ServiceMBeanSupport.java:164)
at org.jboss.ejb.plugins.jms.JMSContainerInvoker.innerCreate(JMSContainerInvoker.java:542)
at org.jboss.ejb.plugins.jms.JMSContainerInvoker.startService(JMSContainerInvoker.java:764)
at org.jboss.system.ServiceMBeanSupport.jbossInternalStart(ServiceMBeanSupport.java:271)
at org.jboss.system.ServiceMBeanSupport.jbossInternalLifecycle(ServiceMBeanSupport.java:221)
[...deleted...]

This exception is documented on the Wiki as well as the FAQ and the usual reason is that the PersistenceManager is improperly configured or that the PersistenceManager is not correctly configured to work with it. However I used the default hsqldb persistence without change and got these same exceptions.

I have deployed the firebirdsql.rar and the firebird-ds.xml files as per the docs/examples and the tests that I've run are correct: Firebird is correctly deployed and JMS is configured to use the FirebirdDS.

On a slower machine that I'm using as a clustered node the exceptions do not occur, leading me to believe that this is a timing issue and that the CachedConnectionManager has been initialized behind the scenes before the deployer gets to the MDB deployment. I probably need to insert a dependency somewhere to make this happen, but I'm at a loss to discover where. The deployment, regardless of these exceptions, does start up and work correctly. It processes messages to the queue, posts a notification to a topic and fails over correctly. I'm just worried about the exceptions and how to correct them so that they do not mask actual problems.

I tried adding a dependency on CachedConnectionManager into the conf/jboss-service.xml file (dependency from the jboss.ejb:service=EJBDeployer to the CachedConnectionManager), and that worked; this reinforces my opinion that there is a missing dependency somewhere. I just don't think that modifying the "global" conf/jboss-service.xml is the correct thing to do.

Does the XSL deployer fail to rewrite the deployment file correctly with all dependencies maybe? From what I understand the XSL deployer should add elements to my deployment files so that the proper dependencies are listed in accordance with my configuration. Perhaps I'm wrong, but I'd like to know.

Any help that can be provided is most appreciated.

- Drew