3 Replies Latest reply on Jun 25, 2016 4:14 PM by jbertram

    Wildfly hornetq clustering using static-connectors, Client fails to connect with errors related to JNDI lookup

    meabhi007

      Hi,

       

      I am trying to configure the HornetQ cluster using static connectors.

      following are configuration for Cluster Node:

      <subsystem xmlns="urn:jboss:domain:messaging:1.2">

      ...

      <connectors>

          <netty-connector name="netty-connector-cluster-node" socket-binding="jms-cluster-node"/>

      </connectors>

      <cluster-connections>

            <cluster-connection name="my-cluster">

            <address>jms</address>

            <connector-ref>netty</connector-ref>

            <static-connectors>

                      <connector-ref>netty-connector-cluster-node</connector-ref>

            </static-connectors>

      </cluster-connection>

      ...

      </subsystem>


      To connect standalone JMS Clients, i am using following piece of code.

      Here in PROVIDER_URL i am providing the comma seperated values of JMS nodes.


      public class JMSConnection

      {

        private static String topicConnectionFactoryName = "testConnectionFactory";

        private static TopicConnectionFactory topicConnectionFactory;

        private String initialContextFactoryName = "org.jboss.naming.remote.client.InitialContextFactory";

        private String providerURL = "http-remoting://node1:9001,http-remoting://node2:9001";


        private static InitialContext initialContext;

       

        public JMSConnection() throws NamingException

        {

        Properties h = setContextProperty(initialContextFactoryName, providerURL);

        initialContext = new InitialContext(h);

        }

       

        /**

        * @param args

        * @throws NamingException

        */

        public static void main(String[] args) throws NamingException

        {

        new JMSConnection();

        initialContext.lookup(topicConnectionFactoryName);

        }


      public static Properties setContextProperty(String contextFactory, String providerUrl)

        {

        Properties prop = new Properties();

        // For Jboss 7 on App Server these properties are passed as blank so we don't need to set

        // them.

        prop.put(Context.INITIAL_CONTEXT_FACTORY, contextFactory);

        prop.put(Context.PROVIDER_URL, providerUrl);

        prop.put("jnp.disableDiscovery", System.getProperty("jboss.global.jnp.disableDiscovery", "true"));

        return prop;

        }

      }

       

      UseCases:

      1. Node-1 - Active and Node-2 Backup

          In this case, JNDI lookup works fine and i get the reference of connection factory.

      2. Node-1 Backup and Node-2 Active

          In this case, JNDI lookup is throwing NamingException that connection factory is not found.

          What i understand here that Since Connection factory is registered into Node-2, while Provider-url is having 1st entry for Node-1 (which is running in backup mode and doesn't have connection factory registered).

          Since Node-1 is up and running so JNDI lookup manages to make connection with node-1 and doesn't find connection factory registered, so throws an exception.


      Can someone please suggest the better way to write the Clients for this scenario, or there is something, which can be done in Wildfly configurations for hornetQ server.?

      even MDB clients are failing to find out the active node.


      <jee:message-driven>

        <jee:ejb-name>Topic1</jee:ejb-name>

      <jee:activation-config>

      ....

      ...

      <jee:activation-config-property>

        <jee:activation-config-property-name>hA</jee:activation-config-property-name>

        <jee:activation-config-property-value>true</jee:activation-config-property-value>

        </jee:activation-config-property>

      <jee:activation-config-property>

        <jee:activation-config-property-name>connectionParameters</jee:activation-config-property-name>

        <jee:activation-config-property-value>host=node1;port=5445,host=node2;port=5445</jee:activation-config-property-value>

        </jee:activation-config-property>

      ...

      ...

      </jee:activation-config>

      </jee:message-driven>


      Note: I didn't see this kind of issue with Multicast based clustering configurations, but i can't use multicast based clustering, since it will not be supported with Amazon Cloud.

      Thanks in advance.


        • 1. Re: Wildfly hornetq clustering using static-connectors, Client fails to connect with errors related to JNDI lookup
          jbertram

          You'll have to write logic in your client to deal with the naming exception and try the alternate URL (or not use JNDI and just instantiate the objects you need).

           

          I would expect the MDBs to work fine.

          • 2. Re: Wildfly hornetq clustering using static-connectors, Client fails to connect with errors related to JNDI lookup
            meabhi007

            Hi Justin,

             

            Thanks for suggestion, i took care of Naming exceptions on client, and yes, MDBs are able to figure out live server.

            Now failover on server crashing is working fine.

             

            But, in case of Network disconnect, split brain occurs, we use to detect the split brain through code and restart the node where network disconnect happened (in backup mode).

            ConnectionFactory lookup manages to connect with correct server but while creating the JMSConnection (through createTopicConnection() api), it throws JMSException.

            Similar exception is coming in JBoss console logs as well (i believe, its while connecting the MDBs).

            Can you please suggest by taking to look to exception, if there can be possible configuration issue?

            NOTE: These issues are not coming in fresh restart of JMS node, just related to failover.

             

            Client Exception:

            javax.jms.JMSException: Failed to create session factory

              at org.hornetq.jms.client.HornetQConnectionFactory.createConnectionInternal(HornetQConnectionFactory.java:673)

              at org.hornetq.jms.client.HornetQConnectionFactory.createTopicConnection(HornetQConnectionFactory.java:193)

              at org.hornetq.jms.client.HornetQConnectionFactory.createTopicConnection(HornetQConnectionFactory.java:188)

              at com.egain.platform.framework.event.Messenger.getTopicConnection(Messenger.java:1357)

              at com.egain.platform.framework.event.Messenger.access$600(Messenger.java:105)

              at com.egain.platform.framework.event.Messenger$ConnectionExceptionListener.onException(Messenger.java:1763)

              at org.hornetq.jms.client.HornetQConnection$JMSFailureListener$1.run(HornetQConnection.java:775)

              at java.lang.Thread.run(Thread.java:745)

            Caused by: HornetQConnectionTimedOutException[errorType=CONNECTION_TIMEDOUT message=HQ119013: Timed out waiting to receive cluster topology. Group:null]

              at org.hornetq.core.client.impl.ServerLocatorImpl.createSessionFactory(ServerLocatorImpl.java:946)

              at org.hornetq.jms.client.HornetQConnectionFactory.createConnectionInternal(HornetQConnectionFactory.java:669)

              ... 7 more

             

            Jboss Console Exeption:

            2016-06-24 22:19:29,851 WARN  [org.hornetq.jms.server] (Periodic Recovery) HQ122008: XA Recovery can not connect to any hornetq server on recovery [XARecoveryConfig [transportConfiguration = [TransportConfiguration(name=netty, factory=org-hornetq-core-remoting-impl-netty-NettyConnectorFactory) ?port=5445&host=10-10-24-76, TransportConfiguration(name=netty, factory=org-hornetq-core-remoting-impl-netty-NettyConnectorFactory) ?port=5445&host=10-10-24-77], discoveryConfiguration = null, username=null, password=****]]

            2016-06-24 22:19:39,852 WARN  [com.arjuna.ats.jta] (Periodic Recovery) ARJUNA016008: Local XARecoveryModule.xaRecovery - caught exception: java.lang.NullPointerException

              at com.arjuna.ats.internal.jta.recovery.arjunacore.XARecoveryModule.xaRecoverySecondPass(XARecoveryModule.java:650) [narayana-jts-jacorb-5.0.0.Final.jar:5.0.0.Final (revision: 9aa71)]

              at com.arjuna.ats.internal.jta.recovery.arjunacore.XARecoveryModule.bottomUpRecovery(XARecoveryModule.java:431) [narayana-jts-jacorb-5.0.0.Final.jar:5.0.0.Final (revision: 9aa71)]

              at com.arjuna.ats.internal.jta.recovery.arjunacore.XARecoveryModule.periodicWorkSecondPass(XARecoveryModule.java:212) [narayana-jts-jacorb-5.0.0.Final.jar:5.0.0.Final (revision: 9aa71)]

              at com.arjuna.ats.internal.arjuna.recovery.PeriodicRecovery.doWorkInternal(PeriodicRecovery.java:789) [narayana-jts-jacorb-5.0.0.Final.jar:5.0.0.Final (revision: 9aa71)]

              at com.arjuna.ats.internal.arjuna.recovery.PeriodicRecovery.run(PeriodicRecovery.java:371) [narayana-jts-jacorb-5.0.0.Final.jar:5.0.0.Final (revision: 9aa71)]


            2016-06-24 22:20:09,870 WARN  [org.hornetq.jms.server] (Periodic Recovery) HQ122015: Can not connect to XARecoveryConfig [transportConfiguration = [TransportConfiguration(name=netty, factory=org-hornetq-core-remoting-impl-netty-NettyConnectorFactory) ?port=5445&host=10-10-24-76, TransportConfiguration(name=netty, factory=org-hornetq-core-remoting-impl-netty-NettyConnectorFactory) ?port=5445&host=10-10-24-77], discoveryConfiguration = null, username=null, password=****] on auto-generated resource recovery: HornetQConnectionTimedOutException[errorType=CONNECTION_TIMEDOUT message=HQ119013: Timed out waiting to receive cluster topology. Group:null]

              at org.hornetq.core.client.impl.ServerLocatorImpl.createSessionFactory(ServerLocatorImpl.java:946) [hornetq-core-client-2.4.5.Final.jar:]

              at org.hornetq.jms.server.recovery.HornetQXAResourceWrapper.connect(HornetQXAResourceWrapper.java:377) [hornetq-jms-server-2.4.5.Final.jar:]

              at org.hornetq.jms.server.recovery.HornetQXAResourceWrapper.getDelegate(HornetQXAResourceWrapper.java:286) [hornetq-jms-server-2.4.5.Final.jar:]

              at org.hornetq.jms.server.recovery.HornetQXAResourceWrapper.recover(HornetQXAResourceWrapper.java:74) [hornetq-jms-server-2.4.5.Final.jar:]

              at com.arjuna.ats.internal.jta.recovery.arjunacore.XARecoveryModule.xaRecoverySecondPass(XARecoveryModule.java:740) [narayana-jts-jacorb-5.0.0.Final.jar:5.0.0.Final (revision: 9aa71)]

              at com.arjuna.ats.internal.jta.recovery.arjunacore.XARecoveryModule.bottomUpRecovery(XARecoveryModule.java:431) [narayana-jts-jacorb-5.0.0.Final.jar:5.0.0.Final (revision: 9aa71)]

              at com.arjuna.ats.internal.jta.recovery.arjunacore.XARecoveryModule.periodicWorkSecondPass(XARecoveryModule.java:212) [narayana-jts-jacorb-5.0.0.Final.jar:5.0.0.Final (revision: 9aa71)]

              at com.arjuna.ats.internal.arjuna.recovery.PeriodicRecovery.doWorkInternal(PeriodicRecovery.java:789) [narayana-jts-jacorb-5.0.0.Final.jar:5.0.0.Final (revision: 9aa71)]

              at com.arjuna.ats.internal.arjuna.recovery.PeriodicRecovery.run(PeriodicRecovery.java:371) [narayana-jts-jacorb-5.0.0.Final.jar:5.0.0.Final (revision: 9aa71)]

            • 3. Re: Wildfly hornetq clustering using static-connectors, Client fails to connect with errors related to JNDI lookup
              jbertram

              You will be highly susceptible to split brain if you just have 2 nodes.  To mitigate the risk of split brain you need to increase the size of your live/backup cluster.

               

              As far as the exceptions you reported I don't really understand the use-case.