7 Replies Latest reply on Sep 17, 2010 8:13 AM by timfox

    Live server can not recognize backup server

    joydeep.sarkar

      Hi,

       

      When I am starting up the Live server I get the following error message.

       

      2010-08-12 07:24:46,440 ERROR [org.jboss.kernel.plugins.dependency.AbstractKernelController] (main) Error installing to Start: name=JMSServerManager state=Create HornetQException[errorCode=104 message=Connected server is not a backup server]         
      at org.hornetq.core.remoting.impl.ChannelImpl.sendBlocking(ChannelImpl.java:280)         
      at org.hornetq.core.replication.impl.ReplicationManagerImpl.start(ReplicationManagerImpl.java:326)         
      at org.hornetq.core.server.impl.HornetQServerImpl.startReplication(HornetQServerImpl.java:925)         
      at org.hornetq.core.server.impl.HornetQServerImpl.initialisePart2(HornetQServerImpl.java:1019)         
      at org.hornetq.core.server.impl.HornetQServerImpl.start(HornetQServerImpl.java:301)         
      at org.hornetq.jms.server.impl.JMSServerManagerImpl.start(JMSServerManagerImpl.java:179)         
      at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)         
      at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)         
      at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)         
      at java.lang.reflect.Method.invoke(Method.java:585)         
      at org.jboss.reflect.plugins.introspection.ReflectionUtils.invoke(ReflectionUtils.java:59)         
      at org.jboss.reflect.plugins.introspection.ReflectMethodInfoImpl.invoke(ReflectMethodInfoImpl.java:150)         
      at org.jboss.joinpoint.plugins.BasicMethodJoinPoint.dispatch(BasicMethodJoinPoint.java:66)         
      at org.jboss.kernel.plugins.dependency.KernelControllerContextAction$JoinpointDispatchWrapper.execute(KernelControllerContextAction.java:241) 
      

       

      In the backup server I see the following message,

      2010-08-12 06:50:35,780 INFO  [org.hornetq.core.server.impl.HornetQServerImpl] (main) backup server is starting..

      2010-08-12 06:50:35,905 WARN  [org.hornetq.core.persistence.impl.journal.JournalStorageManager] (main) AIO wasn't located on this platform, it will fall back to using pure Java NIO. If your platform is Linux, install LibAIO to enable the AIO journal

      2010-08-12 06:50:35,905 INFO  [org.hornetq.core.persistence.impl.journal.JournalStorageManager] (main) Using NIO Journal

      2010-08-12 06:50:36,686 INFO  [org.hornetq.core.server.impl.HornetQServerImpl] (main) Backup server initialised

       

       

      Also, When I look into the JMX console, I do not see org.hornetq link (Screenshot attached).

       

      Does anyone have any idea what is causing this error?

       

      TIA,

      Joydeep

        • 1. Re: Live server can not recognize backup server
          joydeep.sarkar

          Hi,

           

          Just thought to post the configuration that I have. Hope this could be helpful.

          Please note that rest of the configurations are same as out-of-the box state. I followed chapter 39 of the user manual.

           

          Live Server:

          hornetq-configuration.xml

          <configuration xmlns="urn:hornetq"
                         xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
                         xsi:schemaLocation="urn:hornetq /schema/hornetq-configuration.xsd">
             <clustered>true</clustered>
             <log-delegate-factory-class-name>org.hornetq.integration.logging.Log4jLogDelegateFactory</log-delegate-factory-class-name>
             <bindings-directory>${jboss.server.data.dir}/hornetq/bindings</bindings-directory>
             <journal-directory>${jboss.server.data.dir}/hornetq/journal</journal-directory>
             <large-messages-directory>${jboss.server.data.dir}/hornetq/largemessages</large-messages-directory>
             <paging-directory>${jboss.server.data.dir}/hornetq/paging</paging-directory>
             <backup-connector-ref connector-name="backup-connector"/>
             <connectors>
                <connector name="netty">
                   <factory-class>org.hornetq.integration.transports.netty.NettyConnectorFactory</factory-class>
                   <param key="host"  value="${hornetq.remoting.netty.host:10.1.0.71}"/>
                   <param key="port"  value="${hornetq.remoting.netty.port:5445}"/>
                </connector>
                <connector name="backup-connector">
                  <factory-class>org.hornetq.integration.transports.netty.NettyConnectorFactory</factory-class>
                  <param key="host" value="10.1.0.236"/>
                  <param key="port" value="5445"/>
                </connector>
                <connector name="in-vm">
                   <factory-class>org.hornetq.core.remoting.impl.invm.InVMConnectorFactory</factory-class>
                </connector>
             </connectors>
             <acceptors>
                <acceptor name="netty">
                   <factory-class>org.hornetq.integration.transports.netty.NettyAcceptorFactory</factory-class>
                   <param key="host"  value="${hornetq.remoting.netty.host:10.1.0.71}"/>
                   <param key="port"  value="${hornetq.remoting.netty.port:5445}"/>
                </acceptor>
                <acceptor name="in-vm">
                  <factory-class>org.hornetq.core.remoting.impl.invm.InVMAcceptorFactory</factory-class>
                  <param key="server-id" value="0"/>
                </acceptor>
             </acceptors>
             <broadcast-groups>
                <broadcast-group name="bg-group1">
                   <group-address>231.7.7.7</group-address>
                   <group-port>9876</group-port>
                   <broadcast-period>5000</broadcast-period>
                   <connector-ref connector-name="netty"
                         backup-connector-name="backup-connector"/>
                </broadcast-group>
             </broadcast-groups>
             <discovery-groups>
                <discovery-group name="dg-group1">
                   <group-address>231.7.7.7</group-address>
                   <group-port>9876</group-port>
                   <refresh-timeout>10000</refresh-timeout>
                </discovery-group>
             </discovery-groups>
             <cluster-connections>
                <cluster-connection name="my-cluster">
                   <address>jms</address>
                       <discovery-group-ref discovery-group-name="dg-group1"/>
          <!--    <connector-ref connector-name="netty"
                   backup-connector-name="backup-connector"/> --><!--Failover requires DG-->
                </cluster-connection>
             </cluster-connections>
             <security-settings>
                   <permission type="createTempQueue" roles="guest"/>
                   <permission type="deleteTempQueue" roles="guest"/>
                   <permission type="consume" roles="guest"/>
                   <permission type="send" roles="guest"/>
                </security-setting>
             </security-settings>
             <address-settings>
                <!--default for catch all-->
                <address-setting match="#">
                   <dead-letter-address>jms.queue.DLQ</dead-letter-address>
                   <expiry-address>jms.queue.ExpiryQueue</expiry-address>
                   <redelivery-delay>0</redelivery-delay>
                   <max-size-bytes>-1</max-size-bytes>
                   <page-size-bytes>10485760</page-size-bytes>
                   <message-counter-history-day-limit>10</message-counter-history-day-limit>
                </address-setting>
             </address-settings>
          </configuration>
          


          hornetq-jms.xml

          <configuration xmlns="urn:hornetq"
                      xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
                      xsi:schemaLocation="urn:hornetq /schema/hornetq-jms.xsd">
             <connection-factory name="NettyConnectionFactory">
                <discovery-group-ref discovery-group-name="dg-group1"/>
                <connectors>
                   <connector-ref connector-name="netty"/>
                </connectors>
                <entries>
                   <entry name="/ConnectionFactory"/>
                   <entry name="/XAConnectionFactory"/>
                </entries>
          <!--<retry-interval>1000</retry-interval>
          <retry-interval-multiplier>1.5</retry-interval-multiplier>
          <max-retry-interval>60000</max-retry-interval>
          <reconnect-attempts>1000</reconnect-attempts>-->
             </connection-factory>
          <!--   <connection-factory name="InVMConnectionFactory">
                <connectors>
                   <connector-ref connector-name="in-vm"/>
                </connectors>
                <entries>
                   <entry name="java:/ConnectionFactory"/>
                   <entry name="java:/XAConnectionFactory"/>
                </entries>
             </connection-factory>-->
             <queue name="DLQ">
                <entry name="/queue/DLQ"/>
             </queue>
             <queue name="ExpiryQueue">
                <entry name="/queue/ExpiryQueue"/>
             </queue>
             <queue name="oracleXMLQueue">
                <entry name="/queue/oracleXMLQueue"/>
             </queue>
          </configuration>
          

           

           

          Backup server:

          hornetq-cinfiguration.xml

          <configuration xmlns="urn:hornetq"
                         xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
                         xsi:schemaLocation="urn:hornetq /schema/hornetq-configuration.xsd">
             <clustered>true</clustered>
             <log-delegate-factory-class-name>org.hornetq.integration.logging.Log4jLogDelegateFactory</log-delegate-factory-class-name>
             <bindings-directory>${jboss.server.data.dir}/hornetq/bindings</bindings-directory>
             <journal-directory>${jboss.server.data.dir}/hornetq/journal</journal-directory>
             <large-messages-directory>${jboss.server.data.dir}/hornetq/largemessages</large-messages-directory>
             <paging-directory>${jboss.server.data.dir}/hornetq/paging</paging-directory>
             <backup>true</backup>
             <shared-store>false</shared-store>
             <connectors>
                <connector name="netty">
                   <factory-class>org.hornetq.integration.transports.netty.NettyConnectorFactory</factory-class>
                   <param key="host"  value="${hornetq.remoting.netty.host:10.1.0.236}"/>
                   <param key="port"  value="${hornetq.remoting.netty.port:5445}"/>
                </connector>
                <connector name="in-vm">
                   <factory-class>org.hornetq.core.remoting.impl.invm.InVMConnectorFactory</factory-class>
                </connector>
             </connectors>
             <acceptors>
                <acceptor name="netty">
                   <factory-class>org.hornetq.integration.transports.netty.NettyAcceptorFactory</factory-class>
                   <param key="host"  value="${hornetq.remoting.netty.host:10.1.0.236}"/>
                   <param key="port"  value="${hornetq.remoting.netty.port:5445}"/>
                </acceptor>
                <acceptor name="acceptor">
                  <factory-class>org.hornetq.integration.transports.netty.NettyAcceptorFactory</factory-class>
                  <param key="host" value="${hornetq.remoting.netty.host:10.1.0.236}"/>
                  <param key="port" value="${hornetq.remoting.netty.port:5445}"/>
                </acceptor>
                <acceptor name="in-vm">
                  <factory-class>org.hornetq.core.remoting.impl.invm.InVMAcceptorFactory</factory-class>
                  <param key="server-id" value="0"/>
                </acceptor>
             </acceptors>
             <broadcast-groups>
                <broadcast-group name="bg-group1">
                   <group-address>231.7.7.7</group-address>
                   <group-port>9876</group-port>
                   <broadcast-period>5000</broadcast-period>
                   <connector-ref connector-name="netty"/>
                </broadcast-group>
             </broadcast-groups>
             <discovery-groups>
                <discovery-group name="dg-group1">
                   <group-address>231.7.7.7</group-address>
                   <group-port>9876</group-port>
                   <refresh-timeout>10000</refresh-timeout>
                </discovery-group>
             </discovery-groups>
             <cluster-connections>
                <cluster-connection name="my-cluster">
                   <address>jms</address>
                        <discovery-group-ref discovery-group-name="dg-group1"/>
                </cluster-connection>
             </cluster-connections>
             <security-settings>
                <security-setting match="#">
                   <permission type="createTempQueue" roles="guest"/>
                   <permission type="deleteTempQueue" roles="guest"/>
                   <permission type="consume" roles="guest"/>
                   <permission type="send" roles="guest"/>
                </security-setting>
             </security-settings>
             <address-settings>
                <!--default for catch all-->
                <address-setting match="#">
                   <dead-letter-address>jms.queue.DLQ</dead-letter-address>
                   <expiry-address>jms.queue.ExpiryQueue</expiry-address>
                   <redelivery-delay>0</redelivery-delay>
                   <max-size-bytes>-1</max-size-bytes>
                   <page-size-bytes>10485760</page-size-bytes>
                   <message-counter-history-day-limit>10</message-counter-history-day-limit>
                </address-setting>
             </address-settings>
          </configuration>
          

           

          hornetq-jms.xml

          <configuration xmlns="urn:hornetq"
                      xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
                      xsi:schemaLocation="urn:hornetq /schema/hornetq-jms.xsd">
             <connection-factory name="NettyConnectionFactory">
                <discovery-group-ref discovery-group-name="dg-group1"/>
                <connectors>
                   <connector-ref connector-name="netty"/>
                </connectors>
                <entries>
                   <entry name="/ConnectionFactory"/>
                   <entry name="/XAConnectionFactory"/>
                </entries>
          <retry-interval>1000</retry-interval>
          <retry-interval-multiplier>1.5</retry-interval-multiplier>
          <max-retry-interval>60000</max-retry-interval>
          <reconnect-attempts>1000</reconnect-attempts>
             </connection-factory>
          <!--   <connection-factory name="InVMConnectionFactory">
                <connectors>
                   <connector-ref connector-name="in-vm"/>
                </connectors>
                <entries>
                   <entry name="java:/ConnectionFactory"/>
                   <entry name="java:/XAConnectionFactory"/>
                </entries>
             </connection-factory>-->
             <queue name="DLQ">
                <entry name="/queue/DLQ"/>
             </queue>
             <queue name="ExpiryQueue">
                <entry name="/queue/ExpiryQueue"/>
             </queue>
             <queue name="oracleXMLQueue">
                <entry name="/queue/oracleXMLQueue"/>
             </queue>
          </configuration>
          

           

          I have used following command to start the servers,

          for live: jboss-5.1.0.GA/bin/run.sh -c all-with-hornetq -b 10.1.0.71 -u 224.0.0.1

          for backup: jboss-5.1.0.GA/bin/run.sh -c all-with-hornetq -b 10.1.0.236 -u 224.0.0.1

           

          Please let me know if the configuration setting is required to be changed. I am really at my wit's end.

          Please help.

           

          Regards,

          Joydeep

          • 2. Re: Live server can not recognize backup server
            balrajk

            Any resolutions on this issue? please post it if you were able to find the resolution. I am facing a similar problem as well..

             

            Thanks..

            • 3. Re: Live server can not recognize backup server
              timfox

              You probably have your live and backup servers pointing to the same data directory, which will confuse them.

               

              Take a look at the examples, there's a working live/backup example there. You could copy that and change the hosts/ports

              • 4. Re: Live server can not recognize backup server
                balrajk

                Thanks Tim,

                I shall check on this also

                is it ok to have a MDB listening on this queue in the back up server?

                • 5. Re: Live server can not recognize backup server
                  timfox

                  Balraj Kannan wrote:

                   

                  Thanks Tim,

                  I shall check on this also

                  is it ok to have a MDB listening on this queue in the back up server?

                  Ah, you're using JBoss AS? (I assumed you were running standalone since you didn't mention this).

                   

                  Currently MDBs can't run on backup servers - they will cause failover to occur. This is an issue currently being addresses in the improvements to HA work being done by Andy and Jeff, this should be ready in a few weeks.

                  • 6. Re: Live server can not recognize backup server
                    balrajk

                    Ok this will pose a problem isn't?

                    If the MDB isn't deployed in back-up server then won't the active server fail to come up because the contents are not same in two servers?

                     

                    To solve this,

                    Should there be instances of JBoss which only hosts queues and on other instance host MDB's listening on these queues?

                     

                    Is there any other solution to this?

                     

                    Please advise..

                    • 7. Re: Live server can not recognize backup server
                      timfox

                      Like I said, Andy and Jeff are already working on extensions to HA to cope with this case.

                       

                      For now you could have your MDBs deployed on another server. If the live server goes down the MDB connections will fai lover to the backup.