Problem publishing message from one Jboss server to another

[jhd]

Environment:

Jboss 4.2.2
Jboss Messaging 1.4.0 SP3
Jdk 1.5.0_06
Windows XP

Two Jboss servers running on different hosts. No clustering is used for Jboss or Jboss Messaging. Server1 has a topic deployed, topic\securityReload. Server two does not.

Use case:

A message may be published from either Server1 or Server2 to topic\securityReload. If the message originates from Server1 message gets consumed successfully. If the message originates from Server2 no exceptions are observed on either server but message never gets through.

I have a stand-alone Java client that is able to send/receive messages successfully to two different queues deployed on Server1.

Why would a JMS client running in another instance of Jboss AS have trouble publishing to a topic on a remote Jboss AS?

Configuration files follow:

twia-destinations-service.xml:

<server>
 <!-- Destination without a configured SecurityManager or without a
 a SecurityConf will default to role guest with read=true, write=true,
 create=false.
 -->

 <mbean code="org.jboss.jms.server.destination.QueueService"
 name="jboss.messaging.destination:service=Queue,name=testQueue"
 xmbean-dd="xmdesc/Queue-xmbean.xml">
 <depends optional-attribute-name="ServerPeer">jboss.messaging:service=ServerPeer</depends>
 <depends>jboss.messaging:service=PostOffice</depends>
 <attribute name="SecurityConfig">
 <security>
 <role name="guest" read="true" write="true"/>
 <role name="publisher" read="true" write="true" create="false"/>
 <role name="noacc" read="false" write="false" create="false"/>
 </security>
 </attribute>
 </mbean>

 <!-- reload destination is hosted on this server so this section is needed -->
 <mbean code="org.jboss.jms.server.destination.TopicService"
 name="jboss.messaging.destination:service=Topic,name=securityReload"
 xmbean-dd="xmdesc/Topic-xmbean.xml">
 <depends optional-attribute-name="ServerPeer">jboss.messaging:service=ServerPeer</depends>
 <depends>jboss.messaging:service=PostOffice</depends>
 </mbean>


 <mbean code="org.jboss.jms.server.destination.QueueService"
 name="jboss.messaging.destination:service=Queue,name=claimBatch"
 xmbean-dd="xmdesc/Queue-xmbean.xml">
 <depends optional-attribute-name="ServerPeer">jboss.messaging:service=ServerPeer</depends>
 <depends>jboss.messaging:service=PostOffice</depends>
 </mbean>

 <mbean code="org.jboss.jms.server.destination.QueueService"
 name="jboss.messaging.destination:service=Queue,name=taskActionExecute"
 xmbean-dd="xmdesc/Queue-xmbean.xml">
 <depends optional-attribute-name="ServerPeer">jboss.messaging:service=ServerPeer</depends>
 <depends>jboss.messaging:service=PostOffice</depends>
 </mbean>

 <mbean code="org.jboss.jms.server.destination.QueueService"
 name="jboss.messaging.destination:service=Queue,name=requestReply"
 xmbean-dd="xmdesc/Queue-xmbean.xml">
 <depends optional-attribute-name="ServerPeer">jboss.messaging:service=ServerPeer</depends>
 <depends>jboss.messaging:service=PostOffice</depends>
 </mbean>

</server>

jms-ds.xml:

<connection-factories>

 <!-- ==================================================================== -->
 <!-- JMS Stuff -->
 <!-- ==================================================================== -->

 <!-- The JMS provider loader -->
 <mbean code="org.jboss.jms.jndi.JMSProviderLoader"
 name="jboss.messaging:service=JMSProviderLoader,name=JMSProvider">
 <attribute name="ProviderName">DefaultJMSProvider</attribute>
 <attribute name="ProviderAdapterClass">
 org.jboss.jms.jndi.JNDIProviderAdapter
 </attribute>
 <!-- The combined connection factory -->
 <attribute name="FactoryRef">java:/XAConnectionFactory</attribute>
 <!-- The queue connection factory -->
 <attribute name="QueueFactoryRef">java:/XAConnectionFactory</attribute>
 <!-- The topic factory -->
 <attribute name="TopicFactoryRef">java:/XAConnectionFactory</attribute>
 <!-- Uncomment to use HAJNDI to access JMS
 <attribute name="Properties">
 java.naming.factory.initial=org.jnp.interfaces.NamingContextFactory
 java.naming.factory.url.pkgs=org.jboss.naming:org.jnp.interfaces
 java.naming.provider.url=localhost:1100
 </attribute>
 -->
 </mbean>

 <!-- The server session pool for Message Driven Beans -->
 <mbean code="org.jboss.jms.asf.ServerSessionPoolLoader"
 name="jboss.messaging:service=ServerSessionPoolMBean,name=StdJMSPool">
 <depends optional-attribute-name="XidFactory">jboss:service=XidFactory</depends>
 <attribute name="PoolName">StdJMSPool</attribute>
 <attribute name="PoolFactoryClass">
 org.jboss.jms.asf.StdServerSessionPoolFactory
 </attribute>
 </mbean>

 <!-- bug 2370 -->
 <mbean code="org.jboss.jms.jndi.JMSProviderLoader"
 name="jboss.messaging:service=JMSProviderLoader,name=RemoteMQProvider,server=localhost">
 <attribute name="ProviderName">QueuehostJMSProvider</attribute>
 <attribute name="ProviderAdapterClass">
 org.jboss.jms.jndi.JNDIProviderAdapter
 </attribute>
 <attribute name="QueueFactoryRef">ConnectionFactory</attribute>
 <attribute name="TopicFactoryRef">ConnectionFactory</attribute>
 <attribute name="Properties">
 java.naming.factory.initial=org.jnp.interfaces.NamingContextFactory
 java.naming.factory.url.pkgs=org.jboss.naming:org.jnp.interfaces
 java.naming.provider.url=jnp://remotehostname:1099
 </attribute>
 </mbean>

 <tx-connection-factory>
 <jndi-name>JmsXARemote</jndi-name>
 <xa-transaction/>
 <rar-name>jms-ra.rar</rar-name>
 <connection-definition>org.jboss.resource.adapter.jms.JmsConnectionFactory</connection-definition>
 <config-property name="SessionDefaultType" type="java.lang.String">javax.jms.Topic</config-property>
 <config-property name="JmsProviderAdapterJNDI" type="java.lang.String">java:/QueuehostJMSProvider</config-property>
 <max-pool-size>20</max-pool-size>
 <security-domain-and-application />
 </tx-connection-factory>
 <!-- bug 2370 -->


 <!-- JMS XA Resource adapter, use this to get transacted JMS in beans -->
 <tx-connection-factory>
 <jndi-name>JmsXA</jndi-name>
 <xa-transaction/>
 <rar-name>jms-ra.rar</rar-name>
 <connection-definition>org.jboss.resource.adapter.jms.JmsConnectionFactory</connection-definition>
 <config-property name="SessionDefaultType" type="java.lang.String">javax.jms.Topic</config-property>
 <config-property name="JmsProviderAdapterJNDI" type="java.lang.String">java:/DefaultJMSProvider</config-property>
 <max-pool-size>20</max-pool-size>
 <security-domain-and-application>JmsXARealm</security-domain-and-application>
 </tx-connection-factory>

</connection-factories>

remoting-bisocket-service.xml:

<server>

 <!-- Standard bisocket connector - the bisocket transport only opens connection from client->server
 so can be used with firewalls where only outgoing connections are allowed.
 For examples of HTTP and SSL transports see docs/examples -->
 <mbean code="org.jboss.remoting.transport.Connector"
 name="jboss.messaging:service=Connector,transport=bisocket"
 display-name="Bisocket Transport Connector">
 <attribute name="Configuration">
 <config>
 <invoker transport="bisocket">

 <!-- There should be no reason to change these parameters - warning!
 Changing them may stop JBoss Messaging working correctly -->
 <attribute name="marshaller" isParam="true">org.jboss.jms.wireformat.JMSWireFormat</attribute>
 <attribute name="unmarshaller" isParam="true">org.jboss.jms.wireformat.JMSWireFormat</attribute>
 <attribute name="dataType" isParam="true">jms</attribute>
 <attribute name="socket.check_connection" isParam="true">false</attribute>
 <attribute name="timeout" isParam="true">0</attribute>
 <attribute name="serverBindAddress">${jboss.bind.address}</attribute>
 <attribute name="serverBindPort">4457</attribute>
 <attribute name="clientSocketClass" isParam="true">org.jboss.jms.client.remoting.ClientSocketWrapper</attribute>
 <attribute name="serverSocketClass">org.jboss.jms.server.remoting.ServerSocketWrapper</attribute>
 <attribute name="numberOfCallRetries" isParam="true">1</attribute>
 <attribute name="pingFrequency" isParam="true">214748364</attribute>
 <attribute name="pingWindowFactor" isParam="true">10</attribute>
 <attribute name="onewayThreadPool">org.jboss.jms.server.remoting.DirectThreadPool</attribute>
 <!-- End immutable parameters -->

 <!-- Periodicity of client pings. Server window by default is twice this figure -->
 <attribute name="clientLeasePeriod" isParam="true">10000</attribute>

 <!-- Number of seconds to wait for a connection in the client pool to become free -->
 <attribute name="numberOfRetries" isParam="true">10</attribute>

 <!-- Max Number of connections in client pool. This should be significantly higher than
 the max number of sessions/consumers you expect -->
 <attribute name="JBM_clientMaxPoolSize" isParam="true">200</attribute>

 <!-- Use these parameters to specify values for binding and connecting control connections to
 work with your firewall/NAT configuration
 <attribute name="secondaryBindPort">xyz</attribute>
 <attribute name="secondaryConnectPort">abc</attribute>
 -->

 </invoker>
 <handlers>
 <handler subsystem="JMS">org.jboss.jms.server.remoting.JMSServerInvocationHandler</handler>
 </handlers>
 </config>
 </attribute>
 </mbean>

</server>

from standardjboss.xml:

 <invoker-proxy-bindings>

 <!-- bug 2370 -->
 <invoker-proxy-binding>
 <name>RemoteSecurityReloadInvoker</name>
 <invoker-mbean>default</invoker-mbean>
 <proxy-factory>org.jboss.ejb.plugins.jms.JMSContainerInvoker</proxy-factory>
 <proxy-factory-config>
 <JMSProviderAdapterJNDI>QueuehostJMSProvider</JMSProviderAdapterJNDI>
 <ServerSessionPoolFactoryJNDI>StdJMSPool</ServerSessionPoolFactoryJNDI>
 <CreateJBossMQDestination>false</CreateJBossMQDestination>
 <!-- WARN: Don't set this to zero until a bug in the pooled executor is fixed -->
 <MinimumSize>1</MinimumSize>
 <MaximumSize>3</MaximumSize>
 <KeepAliveMillis>30000</KeepAliveMillis>
 <MaxMessages>1</MaxMessages>
 <MDBConfig>
 <ReconnectIntervalSec>10</ReconnectIntervalSec>
 <DLQConfig>
 <DestinationQueue>queue/DLQ</DestinationQueue>
 <MaxTimesRedelivered>1</MaxTimesRedelivered>
 <TimeToLive>0</TimeToLive>
 </DLQConfig>
 </MDBConfig>
 </proxy-factory-config>
 </invoker-proxy-binding>
 <!-- bug 2370 -->

...

from MDB deployment - jboss.xml:

 <!-- bug 2370 -->
 <message-driven>
 <ejb-name>SecurityReloadMDB</ejb-name>
 <destination-jndi-name>topic/securityReload</destination-jndi-name>
 <configuration-name>Standard Message Driven Bean</configuration-name>
 <invoker-bindings>
 <invoker>
 <invoker-proxy-binding-name>RemoteSecurityReloadInvoker</invoker-proxy-binding-name>
 </invoker>
 </invoker-bindings>
 </message-driven>
 <!-- bug 2370 -->

This seems like it should be a simple configuration problem but I haven't been able to track it down.

[jhd]

Here is the client code that is executed from either Server1 or Server2:

 private void triggerSecurityReloadForLoginIds(HashSet loginIds)
 {
 if (loginIds.size() > 0)
 {
 TopicConnection topicConnection = null;
 try
 {
 ServiceLocator sl = ServiceLocator.getInstance();

 Context ctx = new InitialContext();
 TopicConnectionFactory cf = (TopicConnectionFactory) ctx
 .lookup("/ConnectionFactory");

 topicConnection = cf.createTopicConnection();
 TopicSession topicSession = topicConnection.createTopicSession(
 false, Session.AUTO_ACKNOWLEDGE);

 String remoteQueueHost = Env.getInstance()
 .getJMSRemoteQueueHost();
 ctx = sl.getInitialContext(remoteQueueHost);
 String localTopicName = Env.getInstance()
 .getReloadSecurityCacheTopicJndi();
 Topic topic = (Topic) ctx.lookup(localTopicName);

 ObjectMessage objectMessage = topicSession
 .createObjectMessage();
 TopicPublisher topicPublisher = topicSession
 .createPublisher(topic);
 objectMessage.setObject(loginIds);
 topicPublisher.publish(objectMessage);
 log
 .info("triggerSecurityReloadForLoginIds sent reload message <"
 + objectMessage.getJMSMessageID()
 + "> - for logonIds: " + loginIds);
 } catch (Exception e)
 {
 // just log the Exception
 String msg = e.getMessage();
 log.fatal(msg);
 } finally
 {
 if (topicConnection != null)
 {
 try
 {
 topicConnection.close();
 } catch (Exception e)
 {
 }
 }
 }
 }

 }

The value of "remoteQueueHost" is different when executed from Server1 or Server2. The property is tokenized through our build process. For Server1 "remoteQueueHost" = localhost. For Server2 "remoteQueueHost" = Server1.

This code executes without errors from either Server1 or Server2 but the message is only propogated successfully from Server1.

[jhd]

I found the problem. Stupid error in the client code. Getting two different InitialContext instances with different URL_PROVIDERs. The first Context was pointing local and used to lookup the ConnectionFactory, the second Context was pointing remote and used to lookup the Topic Name.

I'm not sure why this would fail silently, but it did.

Problem solved.