3 Replies Latest reply on Mar 27, 2014 10:29 AM by Tomasz Lewandowski

    Live server constantly tries and fails to connect to backup

    Tomasz Lewandowski Newbie

      I have a HornetQ 2.2.14 standalone live/backup static tcp configuration. My OS is Centos 6. Java is Java HotSpot(TM) 64-Bit Server VM (build 20.45-b01, mixed mode)

      After start (both Live and Backup) I see constant messages in Live log:

      2014-03-27 09:21:13,008 Thread-24 (HornetQ-server-HornetQServerImpl::serverUUID=69f8ea44-b346-11e3-a530-dd4c353e1ec9-1951982241) DEBUG [                     impl.netty.NettyConnector] [                     debug] [ 91] : Started Netty Connector version 3.2.5.Final-a96d88c
      2014-03-27 09:21:13,008 Thread-24 (HornetQ-server-HornetQServerImpl::serverUUID=69f8ea44-b346-11e3-a530-dd4c353e1ec9-1951982241) DEBUG [          client.impl.ClientSessionFactoryImpl] [                     debug] [ 91] : Trying to connect at the main server using connector :org-hornetq-core-remoting-impl-netty-NettyConnectorFactory?port=5445&host=192.168.1.2
      2014-03-27 09:21:13,009 Thread-24 (HornetQ-server-HornetQServerImpl::serverUUID=69f8ea44-b346-11e3-a530-dd4c353e1ec9-1951982241) DEBUG [          client.impl.ClientSessionFactoryImpl] [                     debug] [ 91] : Main server is not up. Hopefully there's a backup configured now!
      2014-03-27 09:21:13,009 Thread-24 (HornetQ-server-HornetQServerImpl::serverUUID=69f8ea44-b346-11e3-a530-dd4c353e1ec9-1951982241) DEBUG [          client.impl.ClientSessionFactoryImpl] [                     debug] [ 91] : Trying to connect towards ClientSessionFactoryImpl [serverLocator=ServerLocatorImpl (identity=(main-ClusterConnection::HornetQServerImpl::serverUUID=69f8ea44-b346-11e3-a530-dd4c353e1ec9)) [initialConnectors=[org-hornetq-core-remoting-impl-netty-NettyConnectorFactory?port=5445&host=192.168.1.2], discoveryGroupConfiguration=null], connectorConfig=org-hornetq-core-remoting-impl-netty-NettyConnectorFactory?port=5445&host=192.168.1.2, backupConfig=null]
      2014-03-27 09:21:13,009 Thread-24 (HornetQ-server-HornetQServerImpl::serverUUID=69f8ea44-b346-11e3-a530-dd4c353e1ec9-1951982241) DEBUG [                 client.impl.ServerLocatorImpl] [                     debug] [ 96] : Connector [initialConnector=org-hornetq-core-remoting-impl-netty-NettyConnectorFactory?port=5445&host=192.168.1.2]::Exception on establish connector initial connection

      HornetQException[errorCode=2 message=Unable to connect to server using configuration org-hornetq-core-remoting-impl-netty-NettyConnectorFactory?port=5445&host=192.168.1.2]

          at org.hornetq.core.client.impl.ClientSessionFactoryImpl.connect(ClientSessionFactoryImpl.java:234)
          at org.hornetq.core.client.impl.ServerLocatorImpl$StaticConnector$Connector.tryConnect(ServerLocatorImpl.java:1777)
          at org.hornetq.core.client.impl.ServerLocatorImpl$StaticConnector.connect(ServerLocatorImpl.java:1614)
          at org.hornetq.core.client.impl.ServerLocatorImpl.connect(ServerLocatorImpl.java:587)
          at org.hornetq.core.client.impl.ServerLocatorImpl$3.run(ServerLocatorImpl.java:554)
          at org.hornetq.utils.OrderedExecutorFactory$OrderedExecutor$1.run(OrderedExecutorFactory.java:100)
          at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:895)
          at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:918)
          at java.lang.Thread.run(Thread.java:662)

       

       

       

      I observed that backup server doesn't listen on port 5445, until it becomes Live.

      Is this correct behaviour or have I misconfigured something? This is my config:

       

      Master (LIVE):

       

      <configuration xmlns="urn:hornetq" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="urn:hornetq /schema/hornetq-configuration.xsd">
      
              <paging-directory>${shared.data.dir}/paging</paging-directory>
              <bindings-directory>${shared.data.dir}/bindings</bindings-directory>
              <journal-directory>${shared.data.dir}/journal</journal-directory>
              <journal-min-files>10</journal-min-files>
              <large-messages-directory>${shared.data.dir}/large-messages</large-messages-directory>
      
              <shared-store>true</shared-store>
              <clustered>true</clustered>
      
              <cluster-user>HORNETQ.CLUSTER.ADMIN.USER</cluster-user>
              <cluster-password>abc</cluster-password>
              <failover-on-shutdown>true</failover-on-shutdown>
      
              <acceptors>
                      <acceptor name="netty-acceptor">
                              <factory-class>org.hornetq.core.remoting.impl.netty.NettyAcceptorFactory</factory-class>
                              <param key="port" value="${hornetq.remoting.netty.port.master}" />
                              <param key="host" value="${hornetq.remoting.netty.host.master}" />
                      </acceptor>
              </acceptors>
      
              <connectors>
                      <connector name="netty-connector-master">
                              <factory-class>org.hornetq.core.remoting.impl.netty.NettyConnectorFactory</factory-class>
                              <param key="port" value="${hornetq.remoting.netty.port.master}" />
                              <param key="host" value="${hornetq.remoting.netty.host.master}" />
                      </connector>
                      <connector name="netty-connector-slave">
                              <factory-class>org.hornetq.core.remoting.impl.netty.NettyConnectorFactory</factory-class>
                              <param key="port" value="${hornetq.remoting.netty.port.slave}" />
                              <param key="host" value="${hornetq.remoting.netty.host.slave}" />
                      </connector>
              </connectors>
      
              <cluster-connections>
                      <cluster-connection name="ik-cluster">
                              <address>jms</address>
                              <connector-ref>netty-connector-master</connector-ref>
                              <retry-interval>500</retry-interval>
                              <static-connectors>
                                      <connector-ref>netty-connector-slave</connector-ref>
                              </static-connectors>
                      </cluster-connection>
              </cluster-connections>
      
      </configuration>
      

       

      Slave (BACKUP):

       

      <configuration xmlns="urn:hornetq" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="urn:hornetq /schema/hornetq-configuration.xsd">
      
              <paging-directory>${shared.data.dir}/paging</paging-directory>
              <bindings-directory>${shared.data.dir}/bindings</bindings-directory>
              <journal-directory>${shared.data.dir}/journal</journal-directory>
              <journal-min-files>10</journal-min-files>
              <large-messages-directory>${shared.data.dir}/large-messages</large-messages-directory>
      
              <shared-store>true</shared-store>
              <clustered>true</clustered>
              <backup>true</backup>
              
              <failover-on-shutdown>true</failover-on-shutdown>
      
              <cluster-user>HORNETQ.CLUSTER.ADMIN.USER</cluster-user>
              <cluster-password>abc</cluster-password>
      
              <acceptors>
                      <acceptor name="netty-acceptor">
                              <factory-class>org.hornetq.core.remoting.impl.netty.NettyAcceptorFactory</factory-class>
                              <param key="port" value="${hornetq.remoting.netty.port.slave}" />
                              <param key="host" value="${hornetq.remoting.netty.host.slave}" />
                      </acceptor>
              </acceptors>
      
              <connectors>
                      <connector name="netty-connector-master">
                              <factory-class>org.hornetq.core.remoting.impl.netty.NettyConnectorFactory</factory-class>
                              <param key="port" value="${hornetq.remoting.netty.port.master}" />
                              <param key="host" value="${hornetq.remoting.netty.host.master}" />
                      </connector>
                      <connector name="netty-connector-slave">
                              <factory-class>org.hornetq.core.remoting.impl.netty.NettyConnectorFactory</factory-class>
                              <param key="port" value="${hornetq.remoting.netty.port.slave}" />
                              <param key="host" value="${hornetq.remoting.netty.host.slave}" />
                      </connector>
              </connectors>
      
              <cluster-connections>
                      <cluster-connection name="ik-cluster">
                              <address>jms</address>
                              <connector-ref>netty-connector-slave</connector-ref>
                              <retry-interval>500</retry-interval>
                              <static-connectors>
                                      <connector-ref>netty-connector-master</connector-ref>
                              </static-connectors>
                      </cluster-connection>
              </cluster-connections>
      
      </configuration>
      
        • 1. Re: Live server constantly tries and fails to connect to backup
          Justin Bertram Master

          Couple questions...

           

          • What system properties are you setting on each server?
          • Are all these log messages at DEBUG level?
          • Have you noticed any functional impact on your application?
          • Have you thought about upgrading to a more recent version (e.g. 2.4.0.Final)?
          • 2. Re: Live server constantly tries and fails to connect to backup
            Andy Taylor Master

            yes, these are just debug messages we use for debugging. they are not errors.

            • 3. Re: Live server constantly tries and fails to connect to backup
              Tomasz Lewandowski Newbie

              ad What system properties are you setting on each server?)

              On both master(live) and slave (backup) I set:

              -Dshared.data.dir=/opt/shared/hornetq-data

              -Dhornetq.remoting.netty.host.master=192.168.1.1

              -Dhornetq.remoting.netty.port.master=5445

              -Dhornetq.remoting.netty.host.slave=192.168.1.2

              -Dhornetq.remoting.netty.port.slave=5445

               

              ad Are all these log messages at DEBUG level?)

              Yes, thay are all DEBUG messages, but still I dont feel comfortable with it, as it repeats very often (probably due to retry-interval set to 500)

               

              ad Have you noticed any functional impact on your application?)

              I had some problems (not diagnozed fully yet) that clients temporarly couldn't connect to HornetQ, and I just want to investigate any suspicious logs.

              HornetQ seems to work fine (also tested failover) but still dont feel comfortable with it.

               

              ad Have you thought about upgrading to a more recent version (e.g. 2.4.0.Final)?

              Upgrading would have bigger impact on my system so I don't consider it for now (we have a number of JBoss 7.1 as clients of this standalone HQ, so I expect compatibility problems).