Sending messages to topic in remote JBoss cluster

[jool77]

Hello!

We are currently running two JBoss (EAP 5.1.2) clusters with two nodes each. Cluster A has web application deployd and cluster B runs EJB application. Cluster B needs to send messages to topic in cluster A. Currently this is working fine with defautl JMS with following configuration:

jms-ds.xml:

<mbean code="org.jboss.jms.jndi.JMSProviderLoader"

       name="jboss.messaging:service=JMSProviderLoader,name=FrontJMSProvider">

   <attribute name="ProviderName">FrontJMSProvider</attribute>

   <attribute name="ProviderAdapterClass">org.jboss.jms.jndi.JNDIProviderAdapter</attribute>

   <attribute name="FactoryRef">/XAConnectionFactory</attribute>

   <attribute name="QueueFactoryRef">/XAConnectionFactory</attribute>

   <attribute name="TopicFactoryRef">/XAConnectionFactory</attribute>

    <attribute name="Properties">

           java.naming.factory.initial=org.jnp.interfaces.NamingContextFactory

           java.naming.factory.url.pkgs=org.jboss.naming:org.jnp.interfaces

           java.naming.provider.url=${jboss.external.naming.address}

    </attribute>

</mbean>

<tx-connection-factory>

    <jndi-name>FrontJmsXA</jndi-name>

    <xa-transaction/>

    <rar-name>jms-ra.rar</rar-name>

    <connection-definition>org.jboss.resource.adapter.jms.JmsConnectionFactory</connection-definition>

    <config-property name="SessionDefaultType" type="java.lang.String">javax.jms.Topic</config-property>

    <config-property name="JmsProviderAdapterJNDI" type="java.lang.String">java:/FrontJMSProvider</config-property>

    <max-pool-size>20</max-pool-size>      

    <depends>jboss.messaging:service=ServerPeer</depends>     

</tx-connection-factory>

Variable "jboss.external.naming.address" contains IPs of both cluster A nodes with HAJndi port.

We are now trying to replace default JMS with HornetQ, but we can't figure out how to achieve above setup with HornetQ. HornetQ seems to ignore "JmsProviderAdapterJNDI" parameter, so there is probably so other way to configure it, or is it even possbile?

Thanks,

Joni

[ataylor]

there is a remote JCA example in the distro, take a look at that, however you need to have something like:

<tx-connection-factory>

      <jndi-name>RemoteJmsXA</jndi-name>

      <xa-transaction/>

      <rar-name>hornetq-ra.rar</rar-name>

      <connection-definition>org.hornetq.ra.HornetQRAConnectionFactory</connection-definition>

      <config-property name="SessionDefaultType" type="java.lang.String">javax.jms.Topic</config-property>

      <!--if we over ride the connector class we must over ride the params too-->

      <config-property name="ConnectorClassName" type="java.lang.String">org.hornetq.core.remoting.impl.netty.NettyConnectorFactory</config-property>

      <config-property name="ConnectionParameters" type="java.lang.String">host=127.0.0.1;port=5446</config-property>

      <max-pool-size>20</max-pool-size>

   </tx-connection-factory>

[jbertram]

As Andy has pointed out, it is certainly possible to do what you want.

The reason it doesn't work for the configuration you pasted is that the JCA resource adapter used by JBoss Messaging is completely different from the one used by (and shipped with) HornetQ.  The JMSProviderLoader is a construct used for both inflow and outflow by the generic JMS JCA RA (i.e. the one used by JBoss Messaging).  It does not apply to the HornetQ JCA RA.

[jool77]

Thanks for response! I have looked that example, but I cannot figure out how I should configure host address as we have two nodes in target cluster. Above configuration probably accepts only one host address, am I right? If one of the nodes is down, connections should be automatically made to other node. In that example NettyConnectorFactory is configured to boht jms-ds.xml and ra.xml, do I need both configurations?

[ataylor]

it takes a comma separated list of possible connections, or you can just use discovery. This is all in the manual by the way!

[jbertram]

Also, the ra.xml applies only to inflow (e.g. MDBs consuming messages).  The jms-ds.xml applies to outflow (i.e. some inVM application sending messages).  Since you only need to send messages then you don't need to change your ra.xml.

[jool77]

Thank you for both! I will try this configuration on next week, hopefully I get it working.

[jool77]

Unfortunately I have still no luck with this. I made configuration Andy suggested to cluster B node (of course I replaced the IP address with corresponding address of cluster A node). Now when cluster B tries to get connection factory it gets following exception:

[org.jboss.resource.connectionmanager.JBossManagedConnectionPool] (WorkerThread#0[172.19.2.11:39054]) Throwable while attempting to get a new connection: null

javax.resource.ResourceException: Error during setup

        at org.hornetq.ra.HornetQRAManagedConnection.<init>(HornetQRAManagedConnection.java:152)

        at org.hornetq.ra.HornetQRAManagedConnectionFactory.createManagedConnection(HornetQRAManagedConnectionFactory.java:173)

        at org.jboss.resource.connectionmanager.InternalManagedConnectionPool.createConnectionEventListener(InternalManagedConnectionPool.java:639)

        at org.jboss.resource.connectionmanager.InternalManagedConnectionPool.getConnection(InternalManagedConnectionPool.java:273)

        at org.jboss.resource.connectionmanager.JBossManagedConnectionPool$BasePool.getConnection(JBossManagedConnectionPool.java:746)

        at org.jboss.resource.connectionmanager.BaseConnectionManager2.getManagedConnection(BaseConnectionManager2.java:404)

        at org.jboss.resource.connectionmanager.TxConnectionManager.getManagedConnection(TxConnectionManager.java:424)

        at org.jboss.resource.connectionmanager.BaseConnectionManager2.allocateConnection(BaseConnectionManager2.java:496)

        at org.jboss.resource.connectionmanager.BaseConnectionManager2$ConnectionManagerProxy.allocateConnection(BaseConnectionManager2.java:941)

        at org.hornetq.ra.HornetQRASessionFactoryImpl.allocateConnection(HornetQRASessionFactoryImpl.java:837)

....

Caused by: javax.resource.ResourceException: Failed to create session factory

        at org.hornetq.ra.HornetQRAManagedConnection.setup(HornetQRAManagedConnection.java:823)

        at org.hornetq.ra.HornetQRAManagedConnection.<init>(HornetQRAManagedConnection.java:141)

        ... 158 more

Caused by: javax.jms.JMSException: Failed to create session factory

        at org.hornetq.jms.client.HornetQConnectionFactory.createConnectionInternal(HornetQConnectionFactory.java:605)

        at org.hornetq.jms.client.HornetQConnectionFactory.createXATopicConnection(HornetQConnectionFactory.java:179)

        at org.hornetq.jms.client.HornetQConnectionFactory.createXATopicConnection(HornetQConnectionFactory.java:174)

        at org.hornetq.ra.HornetQRAManagedConnection.setup(HornetQRAManagedConnection.java:780)

        ... 159 more

Caused by: HornetQException[errorCode=2 message=Cannot connect to server(s). Tried with all available servers.]

        at org.hornetq.core.client.impl.ServerLocatorImpl.createSessionFactory(ServerLocatorImpl.java:779)

        at org.hornetq.jms.client.HornetQConnectionFactory.createConnectionInternal(HornetQConnectionFactory.java:601)

        ... 162 more

2012-02-06 12:13:14,530 ERROR [org.hornetq.ra.HornetQRASessionFactoryImpl] (WorkerThread#0[172.19.2.11:39054]) Could not create session

javax.resource.ResourceException: Unable to get managed connection for FrontJmsXA

        at org.jboss.resource.connectionmanager.BaseConnectionManager2.getManagedConnection(BaseConnectionManager2.java:441)

        at org.jboss.resource.connectionmanager.TxConnectionManager.getManagedConnection(TxConnectionManager.java:424)

        at org.jboss.resource.connectionmanager.BaseConnectionManager2.allocateConnection(BaseConnectionManager2.java:496)

        at org.jboss.resource.connectionmanager.BaseConnectionManager2$ConnectionManagerProxy.allocateConnection(BaseConnectionManager2.java:941)

        at org.hornetq.ra.HornetQRASessionFactoryImpl.allocateConnection(HornetQRASessionFactoryImpl.java:837)

        at org.hornetq.ra.HornetQRASessionFactoryImpl.createTopicSession(HornetQRASessionFactoryImpl.java:313)

Do I need some configuration also to HornetQ in cluster A or what is the problem?

[jool77]

Above error was my own mistake, I used port number from example, but correct port was 5445. However I still cannot get messages sent to both cluster A nodes. I have configured connection parameters of both nodes, but still only one of the nodes receives messages. No errors are displayed in logs.

[ataylor]

connections will only be to one node so messages will only ever get sent to one node, what you need is your 2 servers to be clustered and then (if both queues have consumers) messages will be redistributed between both nodes

[jool77]

Yes, I have clustering enabled. However I have disabled persistence, is it needed for redistribution to work?

[ataylor]

No, if your 2 nodes are clustered and both have the same queue and both queues have consumers then messages will be distributed.

[jool77]

Then it is quite weird. It also seems that remote Hornet connection doesn't recover if I shutdown one remote node (I have configured IP's of both remote nodes to connection params in jms-ds.xml). I also tried to configure connection by using discovery, but then I didn't get connection at all.

[ataylor]

you wont get recovery unless you have a backup node, discovery probably doesnt work because your network dosnt support Multicast or you dont have a loopback address.

If messages arent being distributed its because of one of the following:

1) your nodes arent clustered, im guessing its this because discovery isnt working either.

2) your queues dont have consumers

[jool77]

Okay, so if cluster B has made connection to one of the nodes of cluster A and that node crashes, connection won't be never tried to other node of cluster A without backup node configuration? If I understood correctly backup node configuration requires shared journal. In our setup this is not an option.