Clustering issue.

[raymanf]

My Application(Standalone with Spring3.1) connecting to a topic which configured in a remote jboss cluster(having two nodes sharing that topic).

I am connecting to that topic using jnp.partitionNam and jnp.discoveryGroup

When I shutdown one of the jboss nodes the other nodes takes control.

After that I get exception on my application:

FIX.log.1:2012-10-30 16:24:00,943 org.springframework.jms.connection.CachingConnectionFactory [WARN] Encountered a JMSException - resetting the underlying JMS Connection

The expected behavior is that my application will retry to connect automaticlly to the remote topic and will continue to have messages - But it doesnt!

To prove that our clusterd jboss mechanisem works fine we are restarting the spring application and then we can see that it starting to get messages as normal (from the second node ofcourse)

So I was sure that the problem is in the Spring's listening configuration.

After disuccusign Spring experts they replied me this:

"

the "WARN" message you see is ok, it's not an ERROR. It warns that underlying connection from ConnectionFactory was disconnected, so that you see it in the log that something happened. But the next time a connection is needed it will be retrieved from ConnectionFactory. So it's basically the responsibility of ConnectionFactory to handle this situation. In your case it is "org.hornetq.jms.client.HornetQJMSConnectionFactory" and we do not provide support for this class. However I've noticed a constructor argument called "ha" and you have set it to "false". I'd guess it might be "High Availability" option and you can try setting it to "true". But this is merely my speculation/guess and I do not guarantee this helps. You should contact authors of that class for detailed configuration in case of clustered environment."

this is my connectionFactoryBean:

<bean name="connectionFactory" class="org.hornetq.jms.client.HornetQJMSConnectionFactory">
                    <constructor-arg name="ha" value="false" />
                    <constructor-arg>
                              <bean class="org.hornetq.api.core.TransportConfiguration">
                                        <constructor-arg
                                                  value="org.hornetq.core.remoting.impl.netty.NettyConnectorFactory" />
                                        <constructor-arg>
                                                  <map key-type="java.lang.String" value-type="java.lang.Object">
                                                            <entry key="port" value="5445"></entry>
                                                  </map>
                                        </constructor-arg>
                              </bean>
                    </constructor-arg>
          </bean>

Mybe you have a clue how could I configure it right?

thanks.

ray

[ataylor]

The spring guys are correct, you need to configure your connection factory to use HA, take a look at any of the examples to see what you need to set.

[raymanf]

Could you refer me to an example please?(link will be good also)

[ataylor]

you neeed to use the constructor and set HA to true, heres a snippet from one of the examples:

      <!-- Pause 1 second between connect attempts -->

      <retry-interval>1000</retry-interval>

      <!-- Multiply subsequent reconnect pauses by this multiplier. This can be used to

      implement an exponential back-off. For our purposes we just set to 1.0 so each reconnect

      pause is the same length -->

      <retry-interval-multiplier>1.0</retry-interval-multiplier>

      <!-- Try reconnecting an unlimited number of times (-1 means "unlimited") -->

      <reconnect-attempts>-1</reconnect-attempts>

you need to set these values on the connection factory

[raymanf]

Hi Andy Taylor,

as I got back to this case this example you gave me I guess need to be modified directly in one of hornetq-jms.xml,  hornetq-configuration.xml or any other hornetq-*.xml file.

Could you refer me to the right one?

thanks

[ataylor]

in hornetq-jms.xml

[raymanf]

Hi Andy.

I was trying to modify hornetq-jms.xml as you mention. but still I cant get my server reconnect to the second node in the remote cluster after shutting down the first node.

(it works only if I restart my server manually - something which wont help me much)

(mybe I missed here somewhere to set HA into true?)

thats my configuration file:

<configuration xmlns="urn:hornetq"
          xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
          xsi:schemaLocation="urn:hornetq /schema/hornetq-jms.xsd">




          <connection-factory name="ConnectionFactory">
                    <connectors>
                              <connector-ref connector-name="netty" />
                    </connectors>
                    <entries>
                              <entry name="ConnectionFactory" />
                              <entry name="XAConnectionFactory" />
                    </entries>
                    <retry-interval>1000</retry-interval>
                    <retry-interval-multiplier>1.0</retry-interval-multiplier>
                    <reconnect-attempts>-1</reconnect-attempts>
          </connection-factory>




          <connection-factory name="NettyConnectionFactory">
                    <xa>true</xa>
                    <connectors>
                              <connector-ref connector-name="netty" />
                    </connectors>
                    <entries>
                              <entry name="/XAConnectionFactory" />
                    </entries>
          </connection-factory>


          <connection-factory name="NettyConnectionFactory">
                    <xa>false</xa>
                    <connectors>
                              <connector-ref connector-name="netty" />
                    </connectors>
                    <entries>
                              <entry name="/ConnectionFactory" />
                    </entries>
          </connection-factory>


          <connection-factory name="NettyThroughputConnectionFactory">
                    <xa>true</xa>
                    <connectors>
                              <connector-ref connector-name="netty-throughput" />
                    </connectors>
                    <entries>
                              <entry name="/XAThroughputConnectionFactory" />
                    </entries>
          </connection-factory>


          <connection-factory name="NettyThroughputConnectionFactory">
                    <xa>false</xa>
                    <connectors>
                              <connector-ref connector-name="netty-throughput" />
                    </connectors>
                    <entries>
                              <entry name="/ThroughputConnectionFactory" />
                    </entries>
          </connection-factory>


          <queue name="DLQ">
                    <entry name="/queue/DLQ" />
          </queue>


          <queue name="ExpiryQueue">
                    <entry name="/queue/ExpiryQueue" />
          </queue>




          <topic name="marketMakerTopic">
                    <entry name="/topic/marketMakerTopic" />
          </topic>


          <topic name="mm1sessionsTopic">
                    <entry name="/topic/mm1sessionsTopic" />
          </topic>




          <topic name="mm2sessionsTopic">
                    <entry name="/topic/mm2sessionsTopic" />
          </topic>


          <topic name="mm3sessionsTopic">
                    <entry name="/topic/mm3sessionsTopic" />
          </topic>




          <topic name="tgwTestSenderTopic">
                    <entry name="/topic/tgwTestSenderTopic" />
          </topic>






          <queue name="channelResponseQueue">
                    <entry name="/queue/channelResponseQueue" />
          </queue>




          <topic name="channelRequestTopic">
                    <entry name="/topic/channelRequestTopic" />
          </topic>


</configuration>


thanks.

[ataylor]

have you tried running any of th failover examples, get one of those to work and then migrate to spring, this is probably a spring configuration issue

[raymanf]

The configuration file seems to you configured in the right way?

[raymanf]

I mean.. am I configuring properly the HA configuration at hornetq-jms.xml?

[ataylor]

you may need to add <ha>true</ha>

[ataylor]

look at an example like i said before https://github.com/hornetq/hornetq/tree/master/examples/jms/non-transaction-failover

[raymanf]

After looking at this example I think I am not in the right direction or mybe the quetion is not understood.

What I have here is a Spring client application which listening to a remote cluster. the remote cluster aint hornetQ but jboss messaging.

The Spring client application doesnt reconnect after I shutdown the first remote clust node.

The remote cluster works perfectly fine with other jboss servers.

So I guess adjusting any configuration at hornetq-jms.xml is not realy the issue here.

The thing here is that my spring client using a listener which use a jndiTopicConnFactory. mybe this is where I should lookn into eh?

[ataylor]

Im confued now, if you are using JBoss messaging why are you trying to configure hornetq?

[raymanf]

I am sorry ill clarify myself again.

The jboss messaging is placed in the remote cluster server (where we got two nodes each one has topic and in case of failover the other node's topic takes place)

now the Spring application client has an internal hornetQ as it's internal messaging provider.

beside of having it's own internal messaging system it's also need to connect to a remote topic cluster(the one I just mention) and retrieved messages from it.

In order to connect to the remote cluster I have implement a spring out of the box listener called DMLC (it's like MDB in the ejb world)

that DMLC requires connection factory - so I am using the internal hornetQ messaging connection factory via JNDI .

now this is working. but as I wrote in this post it does problems in a clustering mode when failover occours.(the DMLC doesnt not re-connecting automaticlly to the second node in the DiscoveryGroup.