10 Replies Latest reply on Feb 20, 2014 11:09 PM by veenaonnet

    Hornetq 2.4, replication not able to see in-queue messages on jboss console on failover though after consumption can see them

    veenaonnet

      Hi,

       

      I am using jboss7.2 and HornetQ 2.3 with collocated topology - replication.

      If I send 10 mesages to Node A and then it goes down. Then Node B HornetQ server comes up. My consumer can consume the 10 messages from Node B.

      This means the replication works fine.

      However, I am not able to see the 10 messages in Node B's queue as 'In queue' messages on Jboss admin console before consumption. Also, not able to get them programmatically using Mbean for a queue,

      Any idea why this is happening?

       

      Regards,

      Veena

        • 1. Re: Hornetq 2.4, replication not able to see in-queue messages on jboss console on failover though after consumption can see them
          mnovak

          Hi Veena,

          can you try to check it in CLI:

          cd $JBOSS_HOME/bin
          sh jboss-cli.sh
          connect <nodeB_hostname>
          [standalone@localhost:9999 /] /subsystem=messaging/hornetq-server=backup/jms-queue=in-queue:count-messages // this change per name of backup server and queue in your config
          

           

          I'm not sure whether Jboss Admin Console counts with two HornetQ server (liveB and backupA).

           

          Thanks,

          Mirek

          • 2. Re: Hornetq 2.4, replication not able to see in-queue messages on jboss console on failover though after consumption can see them
            veenaonnet

            Hi Mirek,

             

            In the collocated topology configuration, Node A has 2 hornetq servers - default and backup. Same is the case with Node B.

            Initially I defined queues for default as well backup hornetq servers. But on failback I always got NamingException for the queues. Hence I defined queues only for the default hornetq server. Same is true for the ConnectionFactory.

            So every node has queues and connectionfactories defined for Default hornetQ server. Backup doesn't have queues and connectionfactory defined.

             

            In the scenario I mentioned in my last query, I observed one more thing-

            When failover happened, I did not see any message in Node B's queue as in-queue message. Checked cli, did not find any message for default server. Also as no queue is defined for the backup server could not find the message count for there.

            Then ran the consumer. First time, it did not get any message. But after running it, when I checked default server's message count from cli as well as admin console, found messages there. Next time when consumer ran, it could get the messages.

             

            So can we say that the trigger to get messages in the queue is running the consumer?

            Any idea how this issue can be solved?

             

            Regards,

            Veena

            • 3. Re: Hornetq 2.4, replication not able to see in-queue messages on jboss console on failover though after consumption can see them
              veenaonnet

              Hi,

               

              On further analysis, could find the messages in RuntimeQueue of a backup server

               

              /subsystem=messaging/hornetq-server=backup/runtime-queue=jms.queue.xyz:count-messages -> Gives 10 messages

               

              Once consumer tries to consume them from the jms queue, for the first time it doesn't get anything. At this time, messages get copied to the jms-queue of the default server.

              Can find them in jms queue now.

               

              /subsystem=messaging/hornetq-server=default/jms-queue=xyz:count-messages -> Gves 10 messages

              At this time /subsystem=messaging/hornetq-server=backup/runtime-queue=jms.queue.xyz:count-messages -> Gives 0 messages

               

              Is it as per the design or can use any configuration for messages to get copied to jms-queue without a need to run a consumer once?

               

               

              Regards,

              Veena

              • 4. Re: Hornetq 2.4, replication not able to see in-queue messages on jboss console on failover though after consumption can see them
                mnovak

                I believe the problem is <redistribution-delay>60000</redistribution-delay> in address settings. Can you set it to <redistribution-delay>0</redistribution-delay> in all "address settings" and try.

                 

                Thanks,

                Mirek

                • 5. Re: Hornetq 2.4, replication not able to see in-queue messages on jboss console on failover though after consumption can see them
                  veenaonnet


                  Hi,

                   

                  redistribution-delay is set to 0 only.

                   

                  Regards,

                  Veena

                  • 6. Re: Hornetq 2.4, replication not able to see in-queue messages on jboss console on failover though after consumption can see them
                    mnovak

                    Hi,

                     

                    can you share your config of both of the servers?

                     

                    Thanks,

                    Mirek

                    • 7. Re: Hornetq 2.4, replication not able to see in-queue messages on jboss console on failover though after consumption can see them
                      veenaonnet

                      Hi,

                       

                      Please find below the configuration:

                       

                      Node 1 configuration:

                        <subsystem xmlns="urn:jboss:domain:messaging:1.3">
                         <hornetq-server>
                          <clustered>true</clustered>
                          <persistence-enabled>true</persistence-enabled>
                          <security-enabled>true</security-enabled>
                          <cluster-user>abc</cluster-user>
                          <cluster-password>abc@</cluster-password>
                          <failover-on-shutdown>true</failover-on-shutdown>
                          <shared-store>false</shared-store>
                          <journal-type>ASYNCIO</journal-type>
                          <journal-file-size>102400</journal-file-size>
                          <journal-min-files>2</journal-min-files>
                          <check-for-live-server>true</check-for-live-server>
                          <backup-group-name>10.4.199.75</backup-group-name>
                          <paging-directory path="10.4.199.75/paging" />
                          <bindings-directory path="10.4.199.75/bindings" />
                          <journal-directory path="10.4.199.75/journal" />
                          <large-messages-directory
                           path="10.4.199.75/large-messages" />
                          <max-saved-replicated-journals-size>20
                          </max-saved-replicated-journals-size>
                          <connectors>
                           <connector name="netty">
                            <factory-class>org.hornetq.core.remoting.impl.netty.NettyConnectorFactory
                            </factory-class>
                            <param key="host" value="10.4.199.75" />
                            <param key="port" value="5445" />
                           </connector>
                           <netty-connector name="netty-throughput"
                            socket-binding="messaging-throughput">
                            <param key="batch-delay" value="50" />
                           </netty-connector>
                           <in-vm-connector name="in-vm" server-id="0" />
                          </connectors>

                          <acceptors>
                           <acceptor name="netty">
                                              <factory-class>org.hornetq.core.remoting.impl.netty.NettyAcceptorFactory</factory-class>
                                              <param key="host" value="0.0.0.0"/>
                                              <param key="port" value="5445"/>
                                          </acceptor>
                           <netty-acceptor name="netty-throughput"
                            socket-binding="messaging-throughput">
                            <param key="batch-delay" value="50" />
                            <param key="direct-deliver" value="false" />
                           </netty-acceptor>
                           <in-vm-acceptor name="in-vm" server-id="0" />
                          </acceptors>

                          <broadcast-groups>
                           <broadcast-group name="bg-group">
                            <group-address>228.4.199.76</group-address>
                            <group-port>9876</group-port>
                            <broadcast-period>5000</broadcast-period>
                            <connector-ref>netty</connector-ref>
                           </broadcast-group>
                          </broadcast-groups>
                          <discovery-groups>
                           <discovery-group name="dg-group">
                            <group-address>228.4.199.76</group-address>
                            <group-port>9876</group-port>
                            <refresh-timeout>10000</refresh-timeout>
                           </discovery-group>
                          </discovery-groups>
                          <cluster-connections>
                           <cluster-connection name="msf-cluster-10.4.199.76">
                            <address>jms</address>
                            <connector-ref>netty</connector-ref>
                            <discovery-group-ref discovery-group-name="dg-group" />
                            <retry-interval>1000</retry-interval>
                            <use-duplicate-detection>true</use-duplicate-detection>
                            <forward-when-no-consumers>false</forward-when-no-consumers>
                            <max-hops>1</max-hops>
                           </cluster-connection>
                          </cluster-connections>
                          <security-settings>
                           <security-setting match="#">
                            <permission type="send" roles="guest" />
                            <permission type="consume" roles="guest" />
                            <permission type="createDurableQueue" roles="guest" />
                            <permission type="deleteDurableQueue" roles="guest" />
                            <permission type="createNonDurableQueue" roles="guest" />
                            <permission type="deleteNonDurableQueue" roles="guest" />
                            <permission type="manage" roles="guest" />
                           </security-setting>
                          </security-settings>
                          <address-settings>
                           <!--default for catch all -->
                           <address-setting match="#">
                            <dead-letter-address>jms.queue.DLQ</dead-letter-address>
                            <expiry-address>jms.queue.ExpiryQueue</expiry-address>
                            <redelivery-delay>0</redelivery-delay>
                            <redistribution-delay>0</redistribution-delay>
                            <max-size-bytes>10485760</max-size-bytes>
                            <address-full-policy>BLOCK</address-full-policy>
                            <message-counter-history-day-limit>10
                            </message-counter-history-day-limit>
                           </address-setting>
                          </address-settings>
                          <jms-connection-factories>
                           <connection-factory name="InVmConnectionFactory">
                            <connectors>
                             <connector-ref connector-name="in-vm" />
                            </connectors>
                            <entries>
                             <entry name="java:/ConnectionFactory" />
                            </entries>
                            <ha>true</ha>
                                              <block-on-acknowledge>true</block-on-acknowledge>
                                              <retry-interval>1000</retry-interval>
                                              <retry-interval-multiplier>1.0</retry-interval-multiplier>
                                              <reconnect-attempts>3</reconnect-attempts>
                           </connection-factory>
                           <connection-factory name="RemoteConnectionFactory">
                            <connectors>
                             <connector-ref connector-name="netty" />
                            </connectors>
                            <entries>
                             <entry name="java:jboss/exported/jms/RemoteConnectionFactory" />
                            </entries>
                            <ha>true</ha>
                            <block-on-acknowledge>true</block-on-acknowledge>
                            <retry-interval>1000</retry-interval>
                            <retry-interval-multiplier>1.0</retry-interval-multiplier>
                            <reconnect-attempts>3</reconnect-attempts>
                           </connection-factory>
                           <pooled-connection-factory name="hornetq-ra">
                            <transaction mode="xa" />
                            <connectors>
                             <connector-ref connector-name="in-vm" />
                            </connectors>
                            <entries>
                             <entry name="java:/JmsXA" />
                            </entries>
                           </pooled-connection-factory>
                          </jms-connection-factories>
                          <jms-destinations>
                           <jms-queue name="abc">
                            <entry name="queue/abc" />
                            <durable>true</durable>
                           </jms-queue>
                          
                          </jms-destinations>
                         </hornetq-server>
                         <hornetq-server name="backup">
                          <clustered>true</clustered>
                          <backup>true</backup>
                          <persistence-enabled>true</persistence-enabled>
                          <security-enabled>true</security-enabled>
                          <cluster-user>abc</cluster-user>
                          <cluster-password>abc@</cluster-password>
                          <allow-failback>true</allow-failback>
                          <failover-on-shutdown>true</failover-on-shutdown>
                          <shared-store>false</shared-store>
                          <journal-type>ASYNCIO</journal-type>
                          <journal-file-size>102400</journal-file-size>
                          <journal-min-files>2</journal-min-files>
                          <check-for-live-server>true</check-for-live-server>
                          <backup-group-name>10.4.199.74
                          </backup-group-name>
                          <paging-directory path="10.4.199.74/paging" />
                          <bindings-directory path="10.4.199.74/bindings" />
                          <journal-directory path="10.4.199.74/journal" />
                          <large-messages-directory
                           path="10.4.199.74/large-messages" />
                          <max-saved-replicated-journals-size>20
                          </max-saved-replicated-journals-size>
                          <live-connector-ref>netty</live-connector-ref>
                          <connectors>
                           <connector name="netty-backup">
                            <factory-class>org.hornetq.core.remoting.impl.netty.NettyConnectorFactory
                            </factory-class>
                            <param key="host" value="10.4.199.75" />
                            <param key="port" value="5446" />
                           </connector>
                           <in-vm-connector name="in-vm" server-id="1" />
                          </connectors>

                          <acceptors>
                           <in-vm-acceptor name="in-vm" server-id="1"/>
                                          <acceptor name="netty-backup">
                                              <factory-class>org.hornetq.core.remoting.impl.netty.NettyAcceptorFactory</factory-class>
                                              <param key="host" value="0.0.0.0"/>
                                              <param key="port" value="5446"/>
                                          </acceptor>
                          </acceptors>

                          <broadcast-groups>
                           <broadcast-group name="bg-group-backup">
                            <group-address>228.4.199.76</group-address>
                            <group-port>9876</group-port>
                            <broadcast-period>5000</broadcast-period>
                            <connector-ref>netty-backup</connector-ref>
                           </broadcast-group>
                          </broadcast-groups>
                          <discovery-groups>
                           <discovery-group name="dg-group-backup">
                            <group-address>228.4.199.76</group-address>
                            <group-port>9876</group-port>
                            <refresh-timeout>10000</refresh-timeout>
                           </discovery-group>
                          </discovery-groups>
                          <cluster-connections>
                           <cluster-connection name="msf-cluster-10.4.199.76">
                            <address>jms</address>
                            <connector-ref>netty-backup</connector-ref>
                            <discovery-group-ref discovery-group-name="dg-group-backup" />
                            <retry-interval>1000</retry-interval>
                            <use-duplicate-detection>true</use-duplicate-detection>
                            <forward-when-no-consumers>false</forward-when-no-consumers>
                            <max-hops>1</max-hops>
                           </cluster-connection>
                          </cluster-connections>
                          <security-settings>
                           <security-setting match="#">
                            <permission type="send" roles="guest" />
                            <permission type="consume" roles="guest" />
                            <permission type="createDurableQueue" roles="guest" />
                            <permission type="deleteDurableQueue" roles="guest" />
                            <permission type="createNonDurableQueue" roles="guest" />
                            <permission type="deleteNonDurableQueue" roles="guest" />
                            <permission type="manage" roles="guest" />
                           </security-setting>
                          </security-settings>
                          <address-settings>
                           <!--default for catch all -->
                           <address-setting match="#">
                            <dead-letter-address>jms.queue.DLQ</dead-letter-address>
                            <expiry-address>jms.queue.ExpiryQueue</expiry-address>
                            <redelivery-delay>0</redelivery-delay>
                            <redistribution-delay>0</redistribution-delay>
                            <max-size-bytes>10485760</max-size-bytes>
                            <address-full-policy>BLOCK</address-full-policy>
                            <message-counter-history-day-limit>10
                            </message-counter-history-day-limit>
                           </address-setting>
                          </address-settings>
                         </hornetq-server>
                        </subsystem>

                       

                      Node2 Configuration:

                        <subsystem xmlns="urn:jboss:domain:messaging:1.3">
                         <hornetq-server>
                          <clustered>true</clustered>
                          <persistence-enabled>true</persistence-enabled>
                          <security-enabled>true</security-enabled>
                          <cluster-user>abc</cluster-user>
                          <cluster-password>abc@</cluster-password>
                          <failover-on-shutdown>true</failover-on-shutdown>
                          <shared-store>false</shared-store>
                          <journal-type>ASYNCIO</journal-type>
                          <journal-file-size>102400</journal-file-size>
                          <journal-min-files>2</journal-min-files>
                          <check-for-live-server>true</check-for-live-server>
                          <backup-group-name>10.4.199.74</backup-group-name>
                          <paging-directory path="10.4.199.74/paging" />
                          <bindings-directory path="10.4.199.74/bindings" />
                          <journal-directory path="10.4.199.74/journal" />
                          <large-messages-directory
                           path="10.4.199.74/large-messages" />
                          <max-saved-replicated-journals-size>20
                          </max-saved-replicated-journals-size>
                          <connectors>
                           <connector name="netty">
                            <factory-class>org.hornetq.core.remoting.impl.netty.NettyConnectorFactory
                            </factory-class>
                            <param key="host" value="10.4.199.74" />
                            <param key="port" value="5445" />
                           </connector>
                           <netty-connector name="netty-throughput"
                            socket-binding="messaging-throughput">
                            <param key="batch-delay" value="50" />
                           </netty-connector>
                           <in-vm-connector name="in-vm" server-id="0" />
                          </connectors>

                          <acceptors>
                           <acceptor name="netty">
                                              <factory-class>org.hornetq.core.remoting.impl.netty.NettyAcceptorFactory</factory-class>
                                              <param key="host" value="0.0.0.0"/>
                                              <param key="port" value="5445"/>
                                          </acceptor>
                           <netty-acceptor name="netty-throughput"
                            socket-binding="messaging-throughput">
                            <param key="batch-delay" value="50" />
                            <param key="direct-deliver" value="false" />
                           </netty-acceptor>
                           <in-vm-acceptor name="in-vm" server-id="0" />
                          </acceptors>

                          <broadcast-groups>
                           <broadcast-group name="bg-group">
                            <group-address>228.4.199.76</group-address>
                            <group-port>9876</group-port>
                            <broadcast-period>5000</broadcast-period>
                            <connector-ref>netty</connector-ref>
                           </broadcast-group>
                          </broadcast-groups>
                          <discovery-groups>
                           <discovery-group name="dg-group">
                            <group-address>228.4.199.76</group-address>
                            <group-port>9876</group-port>
                            <refresh-timeout>10000</refresh-timeout>
                           </discovery-group>
                          </discovery-groups>
                          <cluster-connections>
                           <cluster-connection name="msf-cluster-10.4.199.76">
                            <address>jms</address>
                            <connector-ref>netty</connector-ref>
                            <discovery-group-ref discovery-group-name="dg-group" />
                            <retry-interval>1000</retry-interval>
                            <use-duplicate-detection>true</use-duplicate-detection>
                            <forward-when-no-consumers>false</forward-when-no-consumers>
                            <max-hops>1</max-hops>
                           </cluster-connection>
                          </cluster-connections>
                          <security-settings>
                           <security-setting match="#">
                            <permission type="send" roles="guest" />
                            <permission type="consume" roles="guest" />
                            <permission type="createDurableQueue" roles="guest" />
                            <permission type="deleteDurableQueue" roles="guest" />
                            <permission type="createNonDurableQueue" roles="guest" />
                            <permission type="deleteNonDurableQueue" roles="guest" />
                            <permission type="manage" roles="guest" />
                           </security-setting>
                          </security-settings>
                          <address-settings>
                           <!--default for catch all -->
                           <address-setting match="#">
                            <dead-letter-address>jms.queue.DLQ</dead-letter-address>
                            <expiry-address>jms.queue.ExpiryQueue</expiry-address>
                            <redelivery-delay>0</redelivery-delay>
                            <redistribution-delay>0</redistribution-delay>
                            <max-size-bytes>10485760</max-size-bytes>
                            <address-full-policy>BLOCK</address-full-policy>
                            <message-counter-history-day-limit>10
                            </message-counter-history-day-limit>
                           </address-setting>
                          </address-settings>
                          <jms-connection-factories>
                           <connection-factory name="InVmConnectionFactory">
                            <connectors>
                             <connector-ref connector-name="in-vm" />
                            </connectors>
                            <entries>
                             <entry name="java:/ConnectionFactory" />
                            </entries>
                            <ha>true</ha>
                                              <block-on-acknowledge>true</block-on-acknowledge>
                                              <retry-interval>1000</retry-interval>
                                              <retry-interval-multiplier>1.0</retry-interval-multiplier>
                                              <reconnect-attempts>3</reconnect-attempts>
                           </connection-factory>
                           <connection-factory name="RemoteConnectionFactory">
                            <connectors>
                             <connector-ref connector-name="netty" />
                            </connectors>
                            <entries>
                             <entry name="java:jboss/exported/jms/RemoteConnectionFactory" />
                            </entries>
                            <ha>true</ha>
                            <block-on-acknowledge>true</block-on-acknowledge>
                            <retry-interval>1000</retry-interval>
                            <retry-interval-multiplier>1.0</retry-interval-multiplier>
                            <reconnect-attempts>3</reconnect-attempts>
                           </connection-factory>
                           <pooled-connection-factory name="hornetq-ra">
                            <transaction mode="xa" />
                            <connectors>
                             <connector-ref connector-name="in-vm" />
                            </connectors>
                            <entries>
                             <entry name="java:/JmsXA" />
                            </entries>
                           </pooled-connection-factory>
                          </jms-connection-factories>
                          <jms-destinations>
                           <jms-queue name="abc">
                            <entry name="queue/abc" />
                            <durable>true</durable>
                           </jms-queue>
                          
                          </jms-destinations>
                         </hornetq-server>
                         <hornetq-server name="backup">
                          <clustered>true</clustered>
                          <backup>true</backup>
                          <persistence-enabled>true</persistence-enabled>
                          <security-enabled>true</security-enabled>
                          <cluster-user>abc</cluster-user>
                          <cluster-password>abc@</cluster-password>
                          <allow-failback>true</allow-failback>
                          <failover-on-shutdown>true</failover-on-shutdown>
                          <shared-store>false</shared-store>
                          <journal-type>ASYNCIO</journal-type>
                          <journal-file-size>102400</journal-file-size>
                          <journal-min-files>2</journal-min-files>
                          <check-for-live-server>true</check-for-live-server>
                          <backup-group-name>10.4.199.75
                          </backup-group-name>
                          <paging-directory path="10.4.199.75/paging" />
                          <bindings-directory path="10.4.199.75/bindings" />
                          <journal-directory path="10.4.199.75/journal" />
                          <large-messages-directory
                           path="10.4.199.75/large-messages" />
                          <max-saved-replicated-journals-size>20
                          </max-saved-replicated-journals-size>
                          <live-connector-ref>netty</live-connector-ref>
                          <connectors>
                           <connector name="netty-backup">
                            <factory-class>org.hornetq.core.remoting.impl.netty.NettyConnectorFactory
                            </factory-class>
                            <param key="host" value="10.4.199.74" />
                            <param key="port" value="5446" />
                           </connector>
                           <in-vm-connector name="in-vm" server-id="1" />
                          </connectors>

                          <acceptors>
                           <in-vm-acceptor name="in-vm" server-id="1"/>
                                          <acceptor name="netty-backup">
                                              <factory-class>org.hornetq.core.remoting.impl.netty.NettyAcceptorFactory</factory-class>
                                              <param key="host" value="0.0.0.0"/>
                                              <param key="port" value="5446"/>
                                          </acceptor>
                          </acceptors>

                          <broadcast-groups>
                           <broadcast-group name="bg-group-backup">
                            <group-address>228.4.199.76</group-address>
                            <group-port>9876</group-port>
                            <broadcast-period>5000</broadcast-period>
                            <connector-ref>netty-backup</connector-ref>
                           </broadcast-group>
                          </broadcast-groups>
                          <discovery-groups>
                           <discovery-group name="dg-group-backup">
                            <group-address>228.4.199.76</group-address>
                            <group-port>9876</group-port>
                            <refresh-timeout>10000</refresh-timeout>
                           </discovery-group>
                          </discovery-groups>
                          <cluster-connections>
                           <cluster-connection name="msf-cluster-10.4.199.76">
                            <address>jms</address>
                            <connector-ref>netty-backup</connector-ref>
                            <discovery-group-ref discovery-group-name="dg-group-backup" />
                            <retry-interval>1000</retry-interval>
                            <use-duplicate-detection>true</use-duplicate-detection>
                            <forward-when-no-consumers>false</forward-when-no-consumers>
                            <max-hops>1</max-hops>
                           </cluster-connection>
                          </cluster-connections>
                          <security-settings>
                           <security-setting match="#">
                            <permission type="send" roles="guest" />
                            <permission type="consume" roles="guest" />
                            <permission type="createDurableQueue" roles="guest" />
                            <permission type="deleteDurableQueue" roles="guest" />
                            <permission type="createNonDurableQueue" roles="guest" />
                            <permission type="deleteNonDurableQueue" roles="guest" />
                            <permission type="manage" roles="guest" />
                           </security-setting>
                          </security-settings>
                          <address-settings>
                           <!--default for catch all -->
                           <address-setting match="#">
                            <dead-letter-address>jms.queue.DLQ</dead-letter-address>
                            <expiry-address>jms.queue.ExpiryQueue</expiry-address>
                            <redelivery-delay>0</redelivery-delay>
                            <redistribution-delay>0</redistribution-delay>
                            <max-size-bytes>10485760</max-size-bytes>
                            <address-full-policy>BLOCK</address-full-policy>
                            <message-counter-history-day-limit>10
                            </message-counter-history-day-limit>
                           </address-setting>
                          </address-settings>
                         </hornetq-server>
                        </subsystem>

                       

                      Regards,

                      Veena

                      • 8. Re: Hornetq 2.4, replication not able to see in-queue messages on jboss console on failover though after consumption can see them
                        ataylor

                        moved to correct forum

                        • 9. Re: Re: Hornetq 2.4, replication not able to see in-queue messages on jboss console on failover though after consumption can see them
                          mnovak

                          I was not able to find the problem in your config. Anyway here is config which works for me to compare:

                           

                          Node 1:

                           

                                <subsystem xmlns="urn:jboss:domain:messaging:1.4">
                                      <hornetq-server>
                                          <persistence-enabled>true</persistence-enabled>
                                          <security-enabled>false</security-enabled>
                                          <cluster-password>heslo</cluster-password>
                                          <failover-on-shutdown>true</failover-on-shutdown>
                                          <shared-store>false</shared-store>
                                          <journal-type>ASYNCIO</journal-type>
                                          <journal-file-size>10485760</journal-file-size>
                                          <journal-min-files>2</journal-min-files>
                                          <check-for-live-server>true</check-for-live-server>
                                          <backup-group-name>firstPair</backup-group-name>
                                          <paging-directory path="firstPairJournalLive/paging"/>
                                          <bindings-directory path="firstPairJournalLive/bindings"/>
                                          <journal-directory path="firstPairJournalLive/journal"/>
                                          <large-messages-directory path="firstPairJournalLive/large-messages"/>
                          
                                          <connectors>
                                              <netty-connector name="netty" socket-binding="messaging"/>
                                              <netty-connector name="netty-throughput" socket-binding="messaging-throughput">
                                                  <param key="batch-delay" value="50"/>
                                              </netty-connector>
                                              <in-vm-connector name="in-vm" server-id="0"/>
                                          </connectors>
                          
                                          <acceptors>
                                              <netty-acceptor name="netty" socket-binding="messaging"/>
                                              <netty-acceptor name="netty-throughput" socket-binding="messaging-throughput">
                                                  <param key="batch-delay" value="50"/>
                                                  <param key="direct-deliver" value="false"/>
                                              </netty-acceptor>
                                              <in-vm-acceptor name="in-vm" server-id="0"/>
                                          </acceptors>
                          
                                          <broadcast-groups>
                                              <broadcast-group name="bg-group1">
                                                  <socket-binding>messaging-group</socket-binding>
                                                  <broadcast-period>2000</broadcast-period>
                                                  <connector-ref>
                                                      netty
                                                  </connector-ref>
                                              </broadcast-group>
                                          </broadcast-groups>
                          
                                          <discovery-groups>
                                              <discovery-group name="dg-group1">
                                                  <socket-binding>messaging-group</socket-binding>
                                                  <refresh-timeout>10000</refresh-timeout>
                                              </discovery-group>
                                          </discovery-groups>
                          
                                          <cluster-connections>
                                              <cluster-connection name="my-cluster">
                                                  <address>jms</address>
                                                  <connector-ref>netty</connector-ref>
                                                  <retry-interval>1000</retry-interval>
                                                  <use-duplicate-detection>true</use-duplicate-detection>
                                                  <forward-when-no-consumers>false</forward-when-no-consumers>
                                                  <max-hops>1</max-hops>
                                                  <discovery-group-ref discovery-group-name="dg-group1"/>
                                              </cluster-connection>
                                          </cluster-connections>
                          
                                          <security-settings>
                                              <security-setting match="#">
                                                  <permission type="send" roles="guest"/>
                                                  <permission type="consume" roles="guest"/>
                                                  <permission type="createDurableQueue" roles="guest"/>
                                                  <permission type="deleteDurableQueue" roles="guest"/>
                                                  <permission type="createNonDurableQueue" roles="guest"/>
                                                  <permission type="deleteNonDurableQueue" roles="guest"/>
                                                  <permission type="manage" roles="guest"/>
                                              </security-setting>
                                          </security-settings>
                          
                                          <address-settings>
                                              <address-setting match="#">
                                                  <redelivery-delay>0</redelivery-delay>
                                                  <max-delivery-attempts>200</max-delivery-attempts>
                                                  <max-size-bytes>1048576</max-size-bytes>
                                                  <page-size-bytes>524288</page-size-bytes>
                                                  <address-full-policy>PAGE</address-full-policy>
                                                  <redistribution-delay>0</redistribution-delay>
                                              </address-setting>
                                          </address-settings>
                          
                                          <jms-connection-factories>
                                              <connection-factory name="InVmConnectionFactory">
                                                  <connectors>
                                                      <connector-ref connector-name="in-vm"/>
                                                  </connectors>
                                                  <entries>
                                                      <entry name="java:/ConnectionFactory"/>
                                                  </entries>
                                              </connection-factory>
                                              <connection-factory name="RemoteConnectionFactory">
                                                  <connectors>
                                                      <connector-ref connector-name="netty"/>
                                                  </connectors>
                                                  <entries>
                                                      <entry name="java:jboss/exported/jms/RemoteConnectionFactory"/>
                                                  </entries>
                                                  <ha>true</ha>
                                                  <block-on-acknowledge>true</block-on-acknowledge>
                                                  <retry-interval>1000</retry-interval>
                                                  <retry-interval-multiplier>1.0</retry-interval-multiplier>
                                                  <reconnect-attempts>-1</reconnect-attempts>
                                              </connection-factory>
                                              <pooled-connection-factory name="hornetq-ra">
                                                  <transaction mode="xa"/>
                                                  <connectors>
                                                      <connector-ref connector-name="in-vm"/>
                                                  </connectors>
                                                  <entries>
                                                      <entry name="java:/JmsXA"/>
                                                  </entries>
                                                  <failover-on-server-shutdown>true</failover-on-server-shutdown>
                                              </pooled-connection-factory>
                                          </jms-connection-factories>
                          
                                          <jms-destinations>
                                              <jms-queue name="testQueue0">
                                                  <entry name="jms/queue/testQueue0"/>
                                                  <entry name="java:jboss/exported/jms/queue/testQueue0"/>
                                                  <durable>true</durable>
                                              </jms-queue>
                                              <jms-queue name="InQueue">
                                                  <entry name="jms/queue/InQueue"/>
                                                  <entry name="java:jboss/exported/jms/queue/InQueue"/>
                                                  <durable>true</durable>
                                              </jms-queue>
                                              <jms-queue name="OutQueue">
                                                  <entry name="jms/queue/OutQueue"/>
                                                  <entry name="java:jboss/exported/jms/queue/OutQueue"/>
                                                  <durable>true</durable>
                                              </jms-queue>
                                              <jms-topic name="testTopic0">
                                                  <entry name="jms/topic/testTopic0"/>
                                                  <entry name="java:jboss/exported/jms/topic/testTopic0"/>
                                              </jms-topic>
                                          </jms-destinations>
                                      </hornetq-server>
                                      <hornetq-server name="backup">
                                          <persistence-enabled>true</persistence-enabled>
                                          <security-enabled>false</security-enabled>
                                          <cluster-password>heslo</cluster-password>
                                          <backup>true</backup>
                                          <allow-failback>true</allow-failback>
                                          <failover-on-shutdown>true</failover-on-shutdown>
                                          <shared-store>false</shared-store>
                                          <journal-type>ASYNCIO</journal-type>
                                          <journal-file-size>10485760</journal-file-size>
                                          <check-for-live-server>true</check-for-live-server>
                                          <backup-group-name>secondPair</backup-group-name>
                                          <paging-directory path="secondPairJournalBackup/paging"/>
                                          <bindings-directory path="secondPairJournalBackup/bindings"/>
                                          <journal-directory path="secondPairJournalBackup/journal"/>
                                          <large-messages-directory path="secondPairJournalBackup/large-messages"/>
                          
                                          <connectors>
                                              <netty-connector name="netty-backup" socket-binding="messaging-backup"/>
                                              <in-vm-connector name="in-vm" server-id="0"/>
                                          </connectors>
                          
                                          <acceptors>
                                              <netty-acceptor name="netty-backup" socket-binding="messaging-backup"/>
                                          </acceptors>
                          
                                          <broadcast-groups>
                                              <broadcast-group name="bg-group-backup">
                                                  <socket-binding>messaging-group</socket-binding>
                                                  <broadcast-period>2000</broadcast-period>
                                                  <connector-ref>
                                                      netty-backup
                                                  </connector-ref>
                                              </broadcast-group>
                                          </broadcast-groups>
                          
                                          <discovery-groups>
                                              <discovery-group name="dg-group-backup">
                                                  <socket-binding>messaging-group</socket-binding>
                                                  <refresh-timeout>10000</refresh-timeout>
                                              </discovery-group>
                                          </discovery-groups>
                          
                                          <cluster-connections>
                                              <cluster-connection name="my-cluster">
                                                  <address>jms</address>
                                                  <connector-ref>netty-backup</connector-ref>
                                                  <retry-interval>1000</retry-interval>
                                                  <use-duplicate-detection>true</use-duplicate-detection>
                                                  <forward-when-no-consumers>false</forward-when-no-consumers>
                                                  <max-hops>1</max-hops>
                                                  <discovery-group-ref discovery-group-name="dg-group-backup"/>
                                              </cluster-connection>
                                          </cluster-connections>
                          
                                          <security-settings>
                                              <security-setting match="#">
                                                  <permission type="send" roles="guest"/>
                                                  <permission type="consume" roles="guest"/>
                                                  <permission type="createDurableQueue" roles="guest"/>
                                                  <permission type="deleteDurableQueue" roles="guest"/>
                                                  <permission type="createNonDurableQueue" roles="guest"/>
                                                  <permission type="deleteNonDurableQueue" roles="guest"/>
                                                  <permission type="manage" roles="guest"/>
                                              </security-setting>
                                          </security-settings>
                          
                                          <address-settings>
                                              <address-setting match="#">
                                                  <redelivery-delay>0</redelivery-delay>
                                                  <max-delivery-attempts>200</max-delivery-attempts>
                                                  <max-size-bytes>1048576</max-size-bytes>
                                                  <page-size-bytes>524288</page-size-bytes>
                                                  <address-full-policy>PAGE</address-full-policy>
                                                  <redistribution-delay>0</redistribution-delay>
                                              </address-setting>
                                          </address-settings>
                          </hornetq-server>
                          </subsystem>
                          

                           

                          Node 2:

                                  <subsystem xmlns="urn:jboss:domain:messaging:1.4">
                                      <hornetq-server>
                                          <persistence-enabled>true</persistence-enabled>
                                          <security-enabled>false</security-enabled>
                                          <cluster-password>heslo</cluster-password>
                                          <failover-on-shutdown>true</failover-on-shutdown>
                                          <shared-store>false</shared-store>
                                          <journal-type>ASYNCIO</journal-type>
                                          <journal-file-size>10485760</journal-file-size>
                                          <journal-min-files>2</journal-min-files>
                                          <check-for-live-server>true</check-for-live-server>
                                          <backup-group-name>secondPair</backup-group-name>
                                          <paging-directory path="secondPairJournalLive/paging"/>
                                          <bindings-directory path="secondPairJournalLive/bindings"/>
                                          <journal-directory path="secondPairJournalLive/journal"/>
                                          <large-messages-directory path="secondPairJournalLive/large-messages"/>
                          
                                          <connectors>
                                              <netty-connector name="netty" socket-binding="messaging"/>
                                              <netty-connector name="netty-throughput" socket-binding="messaging-throughput">
                                                  <param key="batch-delay" value="50"/>
                                              </netty-connector>
                                              <in-vm-connector name="in-vm" server-id="0"/>
                                          </connectors>
                          
                                          <acceptors>
                                              <netty-acceptor name="netty" socket-binding="messaging"/>
                                              <netty-acceptor name="netty-throughput" socket-binding="messaging-throughput">
                                                  <param key="batch-delay" value="50"/>
                                                  <param key="direct-deliver" value="false"/>
                                              </netty-acceptor>
                                              <in-vm-acceptor name="in-vm" server-id="0"/>
                                          </acceptors>
                          
                                          <broadcast-groups>
                                              <broadcast-group name="bg-group1">
                                                  <socket-binding>messaging-group</socket-binding>
                                                  <broadcast-period>2000</broadcast-period>
                                                  <connector-ref>
                                                      netty
                                                  </connector-ref>
                                              </broadcast-group>
                                          </broadcast-groups>
                          
                                          <discovery-groups>
                                              <discovery-group name="dg-group1">
                                                  <socket-binding>messaging-group</socket-binding>
                                                  <refresh-timeout>10000</refresh-timeout>
                                              </discovery-group>
                                          </discovery-groups>
                          
                                          <cluster-connections>
                                              <cluster-connection name="my-cluster">
                                                  <address>jms</address>
                                                  <connector-ref>netty</connector-ref>
                                                  <retry-interval>1000</retry-interval>
                                                  <use-duplicate-detection>true</use-duplicate-detection>
                                                  <forward-when-no-consumers>false</forward-when-no-consumers>
                                                  <max-hops>1</max-hops>
                                                  <discovery-group-ref discovery-group-name="dg-group1"/>
                                              </cluster-connection>
                                          </cluster-connections>
                          
                                          <security-settings>
                                              <security-setting match="#">
                                                  <permission type="send" roles="guest"/>
                                                  <permission type="consume" roles="guest"/>
                                                  <permission type="createDurableQueue" roles="guest"/>
                                                  <permission type="deleteDurableQueue" roles="guest"/>
                                                  <permission type="createNonDurableQueue" roles="guest"/>
                                                  <permission type="deleteNonDurableQueue" roles="guest"/>
                                                  <permission type="manage" roles="guest"/>
                                              </security-setting>
                                          </security-settings>
                          
                                          <address-settings>
                                              <address-setting match="#">
                                                  <redelivery-delay>0</redelivery-delay>
                                                  <max-delivery-attempts>200</max-delivery-attempts>
                                                  <max-size-bytes>1048576</max-size-bytes>
                                                  <page-size-bytes>524288</page-size-bytes>
                                                  <address-full-policy>PAGE</address-full-policy>
                                                  <redistribution-delay>0</redistribution-delay>
                                              </address-setting>
                                          </address-settings>
                          
                                          <jms-connection-factories>
                                              <connection-factory name="InVmConnectionFactory">
                                                  <connectors>
                                                      <connector-ref connector-name="in-vm"/>
                                                  </connectors>
                                                  <entries>
                                                      <entry name="java:/ConnectionFactory"/>
                                                  </entries>
                                              </connection-factory>
                                              <connection-factory name="RemoteConnectionFactory">
                                                  <connectors>
                                                      <connector-ref connector-name="netty"/>
                                                  </connectors>
                                                  <entries>
                                                      <entry name="java:jboss/exported/jms/RemoteConnectionFactory"/>
                                                  </entries>
                                                  <ha>true</ha>
                                                  <block-on-acknowledge>true</block-on-acknowledge>
                                                  <retry-interval>1000</retry-interval>
                                                  <retry-interval-multiplier>1.0</retry-interval-multiplier>
                                                  <reconnect-attempts>-1</reconnect-attempts>
                                              </connection-factory>
                                              <pooled-connection-factory name="hornetq-ra">
                                                  <transaction mode="xa"/>
                                                  <connectors>
                                                      <connector-ref connector-name="in-vm"/>
                                                  </connectors>
                                                  <entries>
                                                      <entry name="java:/JmsXA"/>
                                                  </entries>
                                                  <failover-on-server-shutdown>true</failover-on-server-shutdown>
                                              </pooled-connection-factory>
                                          </jms-connection-factories>
                          
                                          <jms-destinations>
                                              <jms-queue name="testQueue0">
                                                  <entry name="jms/queue/testQueue0"/>
                                                  <entry name="java:jboss/exported/jms/queue/testQueue0"/>
                                                  <durable>true</durable>
                                              </jms-queue>
                                              <jms-queue name="InQueue">
                                                  <entry name="jms/queue/InQueue"/>
                                                  <entry name="java:jboss/exported/jms/queue/InQueue"/>
                                                  <durable>true</durable>
                                              </jms-queue>
                                              <jms-queue name="OutQueue">
                                                  <entry name="jms/queue/OutQueue"/>
                                                  <entry name="java:jboss/exported/jms/queue/OutQueue"/>
                                                  <durable>true</durable>
                                              </jms-queue>
                                              <jms-topic name="testTopic0">
                                                  <entry name="jms/topic/testTopic0"/>
                                                  <entry name="java:jboss/exported/jms/topic/testTopic0"/>
                                              </jms-topic>
                                          </jms-destinations>
                                      </hornetq-server>
                                      <hornetq-server name="backup">
                                          <persistence-enabled>true</persistence-enabled>
                                          <security-enabled>false</security-enabled>
                                          <cluster-password>heslo</cluster-password>
                                          <backup>true</backup>
                                          <allow-failback>true</allow-failback>
                                          <failover-on-shutdown>true</failover-on-shutdown>
                                          <shared-store>false</shared-store>
                                          <journal-type>ASYNCIO</journal-type>
                                          <journal-file-size>10485760</journal-file-size>
                                          <check-for-live-server>true</check-for-live-server>
                                          <backup-group-name>firstPair</backup-group-name>
                                          <paging-directory path="firstPairJournalBackup/paging"/>
                                          <bindings-directory path="firstPairJournalBackup/bindings"/>
                                          <journal-directory path="firstPairJournalBackup/journal"/>
                                          <large-messages-directory path="firstPairJournalBackup/large-messages"/>
                          
                                          <connectors>
                                              <netty-connector name="netty-backup" socket-binding="messaging-backup"/>
                                              <in-vm-connector name="in-vm" server-id="0"/>
                                          </connectors>
                          
                                          <acceptors>
                                              <netty-acceptor name="netty-backup" socket-binding="messaging-backup"/>
                                          </acceptors>
                          
                                          <broadcast-groups>
                                              <broadcast-group name="bg-group-backup">
                                                  <socket-binding>messaging-group</socket-binding>
                                                  <broadcast-period>2000</broadcast-period>
                                                  <connector-ref>
                                                      netty-backup
                                                  </connector-ref>
                                              </broadcast-group>
                                          </broadcast-groups>
                          
                                          <discovery-groups>
                                              <discovery-group name="dg-group-backup">
                                                  <socket-binding>messaging-group</socket-binding>
                                                  <refresh-timeout>10000</refresh-timeout>
                                              </discovery-group>
                                          </discovery-groups>
                          
                                          <cluster-connections>
                                              <cluster-connection name="my-cluster">
                                                  <address>jms</address>
                                                  <connector-ref>netty-backup</connector-ref>
                                                  <retry-interval>1000</retry-interval>
                                                  <use-duplicate-detection>true</use-duplicate-detection>
                                                  <forward-when-no-consumers>false</forward-when-no-consumers>
                                                  <max-hops>1</max-hops>
                                                  <discovery-group-ref discovery-group-name="dg-group-backup"/>
                                              </cluster-connection>
                                          </cluster-connections>
                          
                                          <security-settings>
                                              <security-setting match="#">
                                                  <permission type="send" roles="guest"/>
                                                  <permission type="consume" roles="guest"/>
                                                  <permission type="createDurableQueue" roles="guest"/>
                                                  <permission type="deleteDurableQueue" roles="guest"/>
                                                  <permission type="createNonDurableQueue" roles="guest"/>
                                                  <permission type="deleteNonDurableQueue" roles="guest"/>
                                                  <permission type="manage" roles="guest"/>
                                              </security-setting>
                                          </security-settings>
                          
                                          <address-settings>
                                              <address-setting match="#">
                                                  <redelivery-delay>0</redelivery-delay>
                                                  <max-delivery-attempts>200</max-delivery-attempts>
                                                  <max-size-bytes>1048576</max-size-bytes>
                                                  <page-size-bytes>524288</page-size-bytes>
                                                  <address-full-policy>PAGE</address-full-policy>
                                                  <redistribution-delay>0</redistribution-delay>
                                              </address-setting>
                                          </address-settings>
                          </hornetq-server>
                          </subsystem>
                          


                          Socket binding are the same for both of the servers:

                                  <socket-binding name="messaging" port="5445"/>
                                  <socket-binding name="messaging-group" port="0" multicast-address="${jboss.messaging.group.address:231.7.7.7}" multicast-port="${jboss.messaging.group.port:9876}"/>
                                  <socket-binding name="messaging-throughput" port="5455"/>
                                  <socket-binding name="messaging-backup" port="5446"/>
                          






                          • 10. Re: Re: Hornetq 2.4, replication not able to see in-queue messages on jboss console on failover though after consumption can see them
                            veenaonnet

                            Thanks.

                            Not finding any differenece in the configuration.

                            Only difference is, I am using InVmConnection factory in the producer and consumer. Not using the RemoteConnectionFactory.

                             

                            Regards,

                            Veena