1 2 Previous Next 17 Replies Latest reply on Dec 10, 2013 12:31 PM by jbertram

    Problem setting the JMS jboss module in cluster

    a_torres
    a_torres Oct 4, 2011 9:27 AM

      Hi Jboss community,

       

      I'm trying to set up a JMS cluster in jboss and, I'm afraid, I can't. I run the example of the hornetq standalone server and it works just fine but translating it to standalone.xml conf just don't work: no UDP port binding in the broadcast port. No error message in the server.log.

       

      This is the example in the hornetq standalone server:

       

       

      <configuration xmlns="urn:hornetq"

                  xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"

                  xsi:schemaLocation="urn:hornetq /schema/hornetq-configuration.xsd">

         <clustered>true</clustered>

       

         <!-- Connectors -->

       

         <connectors>

            <connector name="netty-connector">

               <factory-class>org.hornetq.core.remoting.impl.netty.NettyConnectorFactory</factory-class>

               <param key="port" value="5445"/>

            </connector>

         </connectors>

        

         <!-- Acceptors -->

         <acceptors>

            <acceptor name="netty-acceptor">

               <factory-class>org.hornetq.core.remoting.impl.netty.NettyAcceptorFactory</factory-class>

               <param key="port" value="5445"/>

            </acceptor>

         </acceptors>

        

         <!-- Clustering configuration -->

         <broadcast-groups>

            <broadcast-group name="my-broadcast-group">

               <group-address>231.7.7.7</group-address>

               <group-port>9876</group-port>

               <broadcast-period>100</broadcast-period>

               <connector-ref>netty-connector</connector-ref>

           </broadcast-group>

         </broadcast-groups>

        

         <discovery-groups>

            <discovery-group name="my-discovery-group">

               <group-address>231.7.7.7</group-address>

               <group-port>9876</group-port>

               <refresh-timeout>10000</refresh-timeout>

            </discovery-group>

         </discovery-groups>

        

         <cluster-connections>

            <cluster-connection name="my-cluster">

               <address>jms</address>

               <connector-ref>netty-connector</connector-ref>

               <retry-interval>500</retry-interval>

               <use-duplicate-detection>true</use-duplicate-detection>

               <forward-when-no-consumers>true</forward-when-no-consumers>

               <max-hops>1</max-hops>

               <discovery-group-ref discovery-group-name="my-discovery-group"/>

            </cluster-connection>

         </cluster-connections>

        

         <!-- Other config -->

       

         <security-settings>

            <!--security for example queue-->

            <security-setting match="jms.queue.exampleQueue">

               <permission type="createDurableQueue" roles="guest"/>

               <permission type="deleteDurableQueue" roles="guest"/>

               <permission type="createNonDurableQueue" roles="guest"/>

               <permission type="deleteNonDurableQueue" roles="guest"/>

               <permission type="consume" roles="guest"/>

               <permission type="send" roles="guest"/>

            </security-setting>

         </security-settings>

       

      </configuration>

       

      I translated it to:

       

      <subsystem xmlns="urn:jboss:domain:messaging:1.0">

        <clustered>true</clustered>

        <acceptors>

            <netty-acceptor name="netty" socket-binding="messaging"/>

        </acceptors>

        <connectors>

            <netty-connector name="netty" socket-binding="messaging"/>

        </connectors>

        <broadcast-groups>

           <broadcast-group name="my-broadcast-group">

              <local-bind-port>54321</local-bind-port>

              <group-address>231.7.7.7</group-address>

              <group-port>9876</group-port>

              <broadcast-period>1000</broadcast-period>

              <connector-ref>netty</connector-ref>

           </broadcast-group>

        </broadcast-groups>

         <discovery-groups>

            <discovery-group name="my-discovery-group">

               <group-address>231.7.7.7</group-address>

               <group-port>9876</group-port>

               <refresh-timeout>10000</refresh-timeout>

            </discovery-group>

         </discovery-groups>

      <cluster-connections>

          <cluster-connection name="my-cluster">

             <address>jms</address>

             <connector-ref>netty</connector-ref>

             <retry-interval>500</retry-interval>

             <use-duplicate-detection>true</use-duplicate-detection>

             <forward-when-no-consumers>true</forward-when-no-consumers>

             <max-hops>1</max-hops>

             <discovery-group-ref discovery-group-name="my-discovery-group"/>

          </cluster-connection>

      </cluster-connections>            

         <security-settings>

            <!--security for example queue-->

            <security-setting match="jms.queue.exampleQueue">

               <permission type="createDurableQueue" roles="guest"/>

               <permission type="deleteDurableQueue" roles="guest"/>

               <permission type="createNonDurableQueue" roles="guest"/>

               <permission type="deleteNonDurableQueue" roles="guest"/>

               <permission type="consume" roles="guest"/>

               <permission type="send" roles="guest"/>

            </security-setting>

         </security-settings>

        <jms-connection-factories>

            <connection-factory name="ConnectionFactory">

                <connectors>

                    <connector-ref connector-name="netty"/>

                </connectors>

                <entries>

                    <entry name="ConnectionFactory"/>

                </entries>

            </connection-factory>

        </jms-connection-factories>

        <jms-destinations>

            <jms-queue name="testQueue">

                <entry name="queue/test"/>

            </jms-queue>

        </jms-destinations>

      </subsystem>

       

      in the standalone.xml file.

       

      The server log is:

       

      Calling C:\HP\jboss\bin\standalone.conf.bat

      ===============================================================================

       

        JBoss Bootstrap Environment

       

        JBOSS_HOME: C:\HP\jboss

       

        JAVA: C:\Java\jdk1.6.0_23\bin\java

       

        JAVA_OPTS: -Dprogram.name=standalone.bat -Xms64M -Xmx512M -XX:MaxPermSize=256M -Dsun.rmi.dgc.client.gcInterval=3600000 -Dsun.rmi.dgc.server.gcInterval=3600000 -Dorg.jboss.resolver.warning=true -Djboss.modules.system.pkgs=org.jboss.byteman -server -Xdebug -Xrunjdwp:transport=dt_socket,address=8989,server=y,suspend=n

       

      ===============================================================================

       

      Listening for transport dt_socket at address: 8989

      15:24:52,154 INFO  [org.jboss.modules] JBoss Modules version 1.0.2.GA

      15:24:52,419 INFO  [org.jboss.msc] JBoss MSC version 1.0.1.GA

      15:24:52,482 INFO  [org.jboss.as] JBoss AS 7.0.2.Final "Arc" starting

      15:24:54,607 WARN  [org.jboss.as] No security realm defined for native management service, all access will be unrestricted.

      15:24:55,263 INFO  [org.jboss.as] creating http management service using network interface (management) port (9990)

      15:24:55,279 WARN  [org.jboss.as] No security realm defined for http management service, all access will be unrestricted.

      15:24:55,310 INFO  [org.jboss.as.logging] Removing bootstrap log handlers

      15:24:55,701 INFO  [org.jboss.as.connector.subsystems.datasources] (Controller Boot Thread) Deploying JDBC-compliant driver class oracle.jdbc.OracleDriver (version 11.2)

      15:24:55,966 INFO  [org.jboss.as.clustering.infinispan.subsystem] (Controller Boot Thread) Activating Infinispan subsystem.

      15:24:56,123 INFO  [org.jboss.as.jacorb] (Controller Boot Thread) Activating JacORB Subsystem

      15:24:56,701 INFO  [org.jboss.as.naming] (Controller Boot Thread) JBAS011800: Activating Naming Subsystem

      15:24:56,748 INFO  [org.jboss.as.naming] (MSC service thread 1-4) JBAS011802: Starting Naming Service

      15:24:56,779 INFO  [org.jboss.as.osgi] (Controller Boot Thread) JBAS011910: Activating OSGi Subsystem

      15:24:56,935 INFO  [org.jboss.as.security] (Controller Boot Thread) Activating Security Subsystem

      15:24:57,013 INFO  [org.jboss.remoting] (MSC service thread 1-2) JBoss Remoting version 3.2.0.Beta2

      15:24:57,404 INFO  [org.xnio] (MSC service thread 1-2) XNIO Version 3.0.0.Beta3

      15:24:57,732 INFO  [org.xnio.nio] (MSC service thread 1-2) XNIO NIO Implementation Version 3.0.0.Beta3

      15:25:01,029 INFO  [org.apache.catalina.core.AprLifecycleListener] (MSC service thread 1-1) La biblioteca nativa de Apache Tomcat basada en ARP que permite un rendimiento ¾ptimo en entornos de desarrollo no ha sido hallada en java.library.path: C:\Java\jdk1.6.0_23\bin;.;C:\WINDOWS\Sun\Java\bin;C:\WINDOWS\system32;C:\WINDOWS;C:\app\cerdeno\product\11.2.0\client_1;C:\oracle\product\10.2.0\db_1\bin;C:\WINDOWS\system32;C:\WINDOWS;C:\WINDOWS\System32\Wbem;C:\Program Files\ActivIdentity\ActivClient\;C:\Program Files\TortoiseSVN\bin;C:\Java\jdk1.6.0_23\bin;C:\ant\bin;C:\Program Files\Apache Software Foundation\Maven 1.0.2\bin;C:\Program Files\cvsnt;C:\HP\OpenView\ServiceActivator\bin;C:\WINDOWS\system32\WindowsPowerShell\v1.0;C:\Program Files\Common Files\Hewlett-Packard\HPOvLIC\lib;C:\Program Files\CVSNT\

      15:25:01,029 INFO  [org.jboss.as.webservices] (Controller Boot Thread) Activating WebServices Extension

      15:25:01,857 INFO  [org.jboss.as.ee] (Controller Boot Thread) Activating EE subsystem

      15:25:02,232 INFO  [org.jboss.as.remoting] (MSC service thread 1-1) Listening on /127.0.0.1:9999

      15:25:04,248 INFO  [org.apache.coyote.http11.Http11Protocol] (MSC service thread 1-4) Arrancando Coyote HTTP/1.1 en puerto http-localhost-127.0.0.1-8080

      15:25:05,357 INFO  [org.jboss.as.jmx.JMXConnectorService] (MSC service thread 1-2) Starting remote JMX connector

      15:25:06,248 WARN  [jacorb.codeset] (MSC service thread 1-1) Warning - unknown codeset (Cp1252) - defaulting to ISO-8859-1

      15:25:06,779 INFO  [org.jboss.as.jacorb] (MSC service thread 1-1) CORBA ORB Service Started

      15:25:07,560 INFO  [org.jboss.as.connector] (MSC service thread 1-1) Starting JCA Subsystem (JBoss IronJacamar 1.0.3.Final)

      15:25:08,139 INFO  [org.jboss.ws.common.management.AbstractServerConfig] (MSC service thread 1-3) JBoss Web Services - Stack CXF Server 4.0.0.Beta3

      15:25:09,811 INFO  [org.jboss.as.connector.subsystems.datasources] (MSC service thread 1-4) Bound data source [java:/hpsa/jdbc/inventoryDB]

      15:25:09,826 INFO  [org.jboss.as.connector.subsystems.datasources] (MSC service thread 1-1) Bound data source [java:/hpsa/jdbc/defaultDB]

      15:25:09,811 INFO  [org.jboss.as.connector.subsystems.datasources] (MSC service thread 1-3) Bound data source [java:/hpsa/jdbc/uiDB]

      15:25:09,826 INFO  [org.jboss.as.connector.subsystems.datasources] (MSC service thread 1-1) Bound data source [java:/hpsa/jdbc/mwfmDB]

      15:25:09,826 INFO  [org.jboss.as.connector.subsystems.datasources] (MSC service thread 1-4) Bound data source [java:/hpsa/jdbc/resmgrDB]

      15:25:09,826 INFO  [org.jboss.as.connector.subsystems.datasources] (MSC service thread 1-1) Bound data source [java:/hpsa/jdbc/serviceDB]

      15:25:10,467 INFO  [org.jboss.as.jacorb] (MSC service thread 1-2) CORBA Naming Service Started

      15:25:10,670 WARN  [org.jboss.as.messaging] (MSC service thread 1-1) AIO wasn't located on this platform, it will fall back to using pure Java NIO. If your platform is Linux, install LibAIO to enable the AIO journal

      15:25:11,186 INFO  [org.hornetq.core.server.impl.HornetQServerImpl] (MSC service thread 1-3) live server is starting with configuration HornetQ Configuration (clustered=true,backup=false,sharedStore=true,journalDirectory=C:\HP\jboss\standalone\data\messagingjournal,bindingsDirectory=C:\HP\jboss\standalone\data\messagingbindings,largeMessagesDirectory=C:\HP\jboss\standalone\data\messaginglargemessages,pagingDirectory=C:\HP\jboss\standalone\data\messagingpaging)

      15:25:11,201 INFO  [org.hornetq.core.server.impl.HornetQServerImpl] (MSC service thread 1-3) Waiting to obtain live lock

      15:25:11,498 INFO  [org.hornetq.core.persistence.impl.journal.JournalStorageManager] (MSC service thread 1-3) Using NIO Journal

      15:25:11,545 ADVERTENCIA [org.hornetq.core.server.impl.HornetQServerImpl] (MSC service thread 1-3) Security risk! It has been detected that the cluster admin user and password have not been changed from the installation default. Please see the HornetQ user guide, cluster chapter, for instructions on how to do this.

      15:25:11,764 INFO  [org.hornetq.core.server.impl.FileLockNodeManager] (MSC service thread 1-3) Waiting to obtain live lock

      15:25:11,764 INFO  [org.hornetq.core.server.impl.FileLockNodeManager] (MSC service thread 1-3) Live Server Obtained live lock

      15:25:14,623 INFO  [org.hornetq.core.remoting.impl.netty.NettyAcceptor] (MSC service thread 1-3) Started Netty Acceptor version 3.2.3.Final-r${buildNumber} localhost:5445 for CORE protocol

      15:25:14,623 INFO  [org.hornetq.core.server.impl.HornetQServerImpl] (MSC service thread 1-3) Server is now live

      15:25:14,623 INFO  [org.hornetq.core.server.impl.HornetQServerImpl] (MSC service thread 1-3) HornetQ Server version 2.2.7.Final (HQ_2_2_7_FINAL_AS7, 121) [56caffdb-e840-11e0-87a3-005056c00008] started

      15:25:14,639 INFO  [org.hornetq.core.server.impl.HornetQServerImpl] (MSC service thread 1-1) trying to deploy queue jms.queue.testQueue

      15:25:14,811 INFO  [org.jboss.as.messaging.jms.AS7BindingRegistry] (MSC service thread 1-1) Bound messaging object to jndi name java:/queue/test

      15:25:14,826 INFO  [org.jboss.as.messaging.jms.AS7BindingRegistry] (MSC service thread 1-2) Bound messaging object to jndi name java:/ConnectionFactory

      15:25:14,842 INFO  [org.hornetq.core.server.impl.HornetQServerImpl] (MSC service thread 1-4) trying to deploy queue jms.queue.OVSAQueue

      15:25:14,842 INFO  [org.jboss.as.messaging.jms.AS7BindingRegistry] (MSC service thread 1-4) Bound messaging object to jndi name java:/queue/OVSAQueue

      15:25:14,983 INFO  [org.jboss.as.deployment] (MSC service thread 1-1) Started FileSystemDeploymentService for directory C:\HP\jboss\standalone\deployments

      15:25:14,983 INFO  [org.jboss.as.deployment] (DeploymentScanner-threads - 1) Found hpsa.ear in deployment directory. To trigger deployment create a file called hpsa.ear.dodeploy

      15:25:15,233 INFO  [org.jboss.as] (Controller Boot Thread) JBoss AS 7.0.2.Final "Arc" started in 23360ms - Started 147 of 206 services (59 services are passive or on-demand)

       

       

      Anyone knows what's happening?

       

      Regards,

      Alvaro.

        • 1. Re: Problem setting the JMS jboss module in cluster
          a_torres
          a_torres Oct 6, 2011 11:34 AM (in response to a_torres)

          Nobody has a hint? At least to know if it's a bug or I'm just doing something wrong?

            • Actions
          • 2. Re: Problem setting the JMS jboss module in cluster
            jaikiran
            jaikiran Oct 7, 2011 7:01 AM (in response to a_torres)

            Perhaps related to https://issues.jboss.org/browse/AS7-2037?

              • Actions
            • 3. Re: Problem setting the JMS jboss module in cluster
              a_torres
              a_torres Oct 7, 2011 8:34 AM (in response to jaikiran)

              Thank you for awnser.

               

              I'm not sure if is because of that. In the issues states that it's resolved in 7.1.0 Beta1, but how I can I check it? I didn't found how to download it.

                • Actions
              • 4. Re: Problem setting the JMS jboss module in cluster
                jaikiran
                jaikiran Oct 7, 2011 8:56 AM (in response to a_torres)

                Try out the latest nightly build http://community.jboss.org/thread/167590 later tonight or tomorrow. This issue was fixed just today, so it might not yet have been published in the nightly builds.

                 

                If you still run into issues with the nightly build (tomorrow) then post the details here.

                1 of 1 people found this helpful
                  • Actions
                • 5. Re: Problem setting the JMS jboss module in cluster
                  a_torres
                  a_torres Oct 10, 2011 12:12 PM (in response to jaikiran)

                  It seems somehow related as I tested the nightly build from 7.1.0 and manage to sucessfully configure the discovery port. I test it with a standalone hornetq installation and pass the tests.

                   

                  I'm not sure about the broadcast port as it not listen in the chosen port. As another new UDP port is listen when I startup I think the problem can be because it ignores the local-bind-port tag but I'm not sure.

                   

                  As I have to drop this solution (as it was a workaround becase no JNDI remote invocation was implemented in 7.0.1) I don't go further on that but if I have time and you need it I can take a look to see if the broadcast is working.

                   

                  Regards and thanks,

                  Alvaro Torres

                    • Actions
                  • 6. Re: Problem setting the JMS jboss module in cluster
                    erasmomarciano
                    erasmomarciano Oct 3, 2012 8:33 AM (in response to a_torres)

                    HI

                     

                    I Have an similan problem with the configuration JMS CLUSTER

                     

                     

                    My scenary

                     

                    JBOSS_A

                     

                    <code>

                     

                      <subsystem xmlns="urn:jboss:domain:messaging:1.2">

                     

                     

                                <hornetq-server>

                                    <clustered>true</clustered>

                                    <broadcast-groups>

                                            <broadcast-group name="my-broadcast-group">

                                                    <local-bind-port>54321</local-bind-port>

                                                    <group-address>231.7.7.7</group-address>

                                                    <group-port>9876</group-port>

                                                    <broadcast-period>1000</broadcast-period>

                                                    <connector-ref>netty</connector-ref>

                                            </broadcast-group>

                                    </broadcast-groups>

                                    <discovery-groups>

                                            <discovery-group name="my-discovery-group">

                                                    <group-address>231.7.7.7</group-address>

                                                    <group-port>9876</group-port>

                                                    <refresh-timeout>10000</refresh-timeout>

                                            </discovery-group>

                                    </discovery-groups>

                                    <cluster-connections>

                                            <cluster-connection name="my-cluster">

                                                    <address>jms</address>

                                                    <connector-ref>netty</connector-ref>

                                                    <retry-interval>500</retry-interval>

                                                    <use-duplicate-detection>true</use-duplicate-detection>

                                                    <forward-when-no-consumers>true</forward-when-no-consumers>

                                                    <max-hops>1</max-hops>

                                                    <discovery-group-ref discovery-group-name="my-discovery-group"/>

                                            </cluster-connection>

                                    </cluster-connections>

                         <persistence-enabled>true</persistence-enabled>

                                    <journal-file-size>102400</journal-file-size>

                                    <journal-min-files>2</journal-min-files>

                     

                     

                                    <connectors>

                                        <netty-connector name="netty" socket-binding="messaging"/>

                                        <netty-connector name="netty-throughput" socket-binding="messaging-throughput">

                                            <param key="batch-delay" value="50"/>

                                        </netty-connector>

                                        <in-vm-connector name="in-vm" server-id="0"/>

                                    </connectors>

                     

                     

                                    <acceptors>

                                        <netty-acceptor name="netty" socket-binding="messaging"/>

                                        <netty-acceptor name="netty-throughput" socket-binding="messaging-throughput">

                                            <param key="batch-delay" value="50"/>

                                            <param key="direct-deliver" value="false"/>

                                        </netty-acceptor>

                                        <in-vm-acceptor name="in-vm" server-id="0"/>

                                    </acceptors>

                     

                     

                                    <security-settings>

                                        <security-setting match="#">

                                            <permission type="send" roles="guest"/>

                                            <permission type="consume" roles="guest"/>

                                            <permission type="createNonDurableQueue" roles="guest"/>

                                            <permission type="deleteNonDurableQueue" roles="guest"/>

                                        </security-setting>

                                    </security-settings>

                     

                     

                      <address-settings>

                                        <address-setting match="#">

                                            <dead-letter-address>jms.queue.DLQ</dead-letter-address>

                                            <expiry-address>jms.queue.ExpiryQueue</expiry-address>

                                            <redelivery-delay>0</redelivery-delay>

                                            <max-size-bytes>10485760</max-size-bytes>

                                            <address-full-policy>BLOCK</address-full-policy>

                                            <message-counter-history-day-limit>10</message-counter-history-day-limit>

                                        </address-setting>

                                    </address-settings>

                     

                     

                                    <jms-connection-factories>

                                        <connection-factory name="InVmConnectionFactory">

                                            <connectors>

                                                <connector-ref connector-name="in-vm"/>

                                            </connectors>

                                            <entries>

                                                <entry name="java:/ConnectionFactory"/>

                                            </entries>

                                        </connection-factory>

                                        <connection-factory name="RemoteConnectionFactory">

                                            <connectors>

                                                <connector-ref connector-name="netty"/>

                                            </connectors>

                                            <entries>

                                                <entry name="java:jboss/exported/jms/RemoteConnectionFactory"/>

                                            </entries>

                                        </connection-factory>

                     

                     

                     

                    <pooled-connection-factory name="hornetq-ra">

                                            <transaction mode="xa"/>

                                            <connectors>

                                                <connector-ref connector-name="in-vm"/>

                                            </connectors>

                                            <entries>

                                                <entry name="java:/JmsXA"/>

                                            </entries>

                                        </pooled-connection-factory>

                                    </jms-connection-factories>

                     

                     

                                    <jms-destinations>

                                        <jms-queue name="testQueue">

                                            <entry name="queue/test"/>

                                            <entry name="java:jboss/exported/jms/queue/test"/>

                                        </jms-queue>

                                        <jms-topic name="testTopic">

                                            <entry name="topic/test"/>

                                            <entry name="java:jboss/exported/jms/topic/test"/>

                                        </jms-topic>

                                    </jms-destinations>

                                </hornetq-server>

                            </subsystem>

                     

                    </code>

                     

                    JBOSS_B

                     

                    I have to duplicate this code?

                      • Actions
                    • 7. Re: Problem setting the JMS jboss module in cluster
                      jbertram
                      jbertram Oct 3, 2012 10:57 AM (in response to erasmomarciano)

                      If you want a "symmetric" cluster (the most common use-case) then yes, you must duplicate that configuration on all nodes of the cluster.

                        • Actions
                      • 8. Re: Problem setting the JMS jboss module in cluster
                        klmurty
                        klmurty Dec 9, 2013 3:05 PM (in response to jbertram)

                        Hi Justin,

                         

                        I have a similar kind of issue in my case, i really appreciate if you can resolve it. I go through this thread and tried but not successful. I am using jboss 7.1.0 and enabled messaging subsystem. We have 3 different jboss servers and want to build JMS cluster. My configuration is as follows:

                         

                          <subsystem xmlns="urn:jboss:domain:messaging:1.1">
                                <hornetq-server>
                                    <clustered>true</clustered>
                                    <persistence-enabled>true</persistence-enabled>
                                    <security-enabled>false</security-enabled>
                                    <journal-file-size>102400</journal-file-size>
                                    <journal-min-files>2</journal-min-files>

                         

                                    <connectors>
                                        <netty-connector name="netty" socket-binding="messaging"/>
                                        <netty-connector name="netty-throughput" socket-binding="messaging-throughput">
                                            <param key="batch-delay" value="50"/>
                                        </netty-connector>
                                        <in-vm-connector name="in-vm" server-id="0"/>
                                    </connectors>

                         

                                    <acceptors>
                                        <netty-acceptor name="netty" socket-binding="messaging"/>
                                        <netty-acceptor name="netty-throughput" socket-binding="messaging-throughput">
                                            <param key="batch-delay" value="50"/>
                                            <param key="direct-deliver" value="false"/>
                                        </netty-acceptor>
                                        <in-vm-acceptor name="in-vm" server-id="0"/>
                                    </acceptors>

                         

                                    <broadcast-groups>
                                        <broadcast-group name="jms-broadcast-group">
                                            <group-address>231.7.7.7</group-address>
                                            <group-port>9876</group-port>
                                            <broadcast-period>100</broadcast-period>
                                            <connector-ref>
                                                netty
                                            </connector-ref>
                                        </broadcast-group>
                                    </broadcast-groups>

                         

                                    <discovery-groups>
                                        <discovery-group name="jms-discovery-group">
                                            <group-address>231.7.7.7</group-address>
                                            <group-port>9876</group-port>
                                            <refresh-timeout>10000</refresh-timeout>
                                        </discovery-group>
                                    </discovery-groups>

                         

                                    <cluster-connections>
                                        <cluster-connection name="jms-cluster">
                                            <address>jms</address>
                                            <connector-ref>netty</connector-ref>
                                            <retry-interval>500</retry-interval>
                                            <use-duplicate-detection>true</use-duplicate-detection>
                                            <forward-when-no-consumers>true</forward-when-no-consumers>
                                            <max-hops>1</max-hops>
                                            <discovery-group-ref discovery-group-name="jms-discovery-group"/>
                                        </cluster-connection>
                                    </cluster-connections>

                           

                                    <security-settings>
                                        <security-setting match="#">
                                            <permission type="send" roles="guest"/>
                                            <permission type="consume" roles="guest"/>
                                            <permission type="createNonDurableQueue" roles="guest"/>
                                            <permission type="deleteNonDurableQueue" roles="guest"/>
                                        </security-setting>
                                    </security-settings>

                         

                                    <address-settings>
                                        <address-setting match="#">
                                            <dead-letter-address>jms.queue.DLQ</dead-letter-address>
                                            <expiry-address>jms.queue.ExpiryQueue</expiry-address>
                                            <redelivery-delay>0</redelivery-delay>
                                             <max-size-bytes>10485760</max-size-bytes>
                                            <page-size-bytes>10240</page-size-bytes>
                                            <address-full-policy>PAGE</address-full-policy>
                                            <message-counter-history-day-limit>10</message-counter-history-day-limit>
                                        </address-setting>
                                    </address-settings>

                         

                                    <jms-connection-factories>
                                        <connection-factory name="InVmConnectionFactory">
                                            <discovery-group-ref discovery-group-name="jms-discovery-group"/>
                                            <connectors>
                                                <connector-ref connector-name="netty"/>
                                            </connectors>
                                            <entries>
                                                <entry name="java:/ConnectionFactory"/>
                                                <entry name="java:jboss/exported/ConnectionFactory"/>
                                            </entries>
                                        </connection-factory>
                                        <connection-factory name="RemoteConnectionFactory">
                                            <connectors>
                                                <connector-ref connector-name="netty"/>
                                            </connectors>
                                            <entries>
                                                <entry name="java:/RemoteConnectionFactory"/>
                                                <entry name="java:jboss/exported/RemoteConnectionFactory"/>
                                            </entries>
                                        </connection-factory>
                                        <pooled-connection-factory name="hornetq-ra">
                                            <transaction mode="xa"/>
                                            <connectors>
                                                <connector-ref connector-name="in-vm"/>
                                            </connectors>
                                            <entries>
                                                <entry name="java:/JmsXA"/>
                                                <entry name="java:jboss/exported/JmsXA"/>
                                            </entries>
                                        </pooled-connection-factory>
                                    </jms-connection-factories>

                         

                                    <jms-destinations>
                                        <jms-queue name="AndroidAppQueue">
                                            <entry name="queue/AndroidAppQueue"/>
                                            <entry name="java:jboss/exported/queue/AndroidAppQueue"/>
                                        </jms-queue>
                                        <jms-queue name="BlackBerryAppQueue">
                                            <entry name="queue/BlackBerryAppQueue"/>
                                            <entry name="java:jboss/exported/queue/BlackBerryAppQueue"/>
                                        </jms-queue>
                                        <jms-queue name="IOSAppQueue">
                                            <entry name="queue/IOSAppQueue"/>
                                            <entry name="java:jboss/exported/queue/IOSAppQueue"/>
                                        </jms-queue>
                                        <jms-queue name="AppAnalysis">
                                            <entry name="queue/AppAnalysis"/>
                                            <entry name="java:jboss/exported/queue/AppAnalysis"/>
                                        </jms-queue>
                                    </jms-destinations>
                                </hornetq-server>
                           

                        </subsystem>

                        I have the same configuration is all of the other nodes but still clustering is not working. My goal is once i send millions of messages on to the queue, messages are to be consumed among these 3 different nodes.

                         

                        I am struggling for last few days to resolve this. Just wondering if you can resolve this.

                         

                        Thanks in advance.

                         

                        Thanks

                        Murty

                          • Actions
                        • 9. Re: Problem setting the JMS jboss module in cluster
                          jbertram
                          jbertram Dec 9, 2013 3:24 PM (in response to klmurty)

                          Can you be more specific about how your trying to exercise the clustering functionality and what exactly isn't working?

                           

                          Do you see in the log that the HornetQ cluster bridges are forming between the cluster nodes?

                           

                          Can you confirm that multicast works in the environment where these servers are located?

                            • Actions
                          • 10. Re: Problem setting the JMS jboss module in cluster
                            klmurty
                            klmurty Dec 9, 2013 4:30 PM (in response to jbertram)

                            "Can you be more specific about how your trying to exercise the clustering functionality and what exactly isn't working?"

                            1. I have 3 virtual machines with Jboss 7.1.0 installed, on the same network and can ping each other.
                            2. The application is deployed on all the 3 servers.
                            3. I can push the messages on to the queue from any of the servers and all the servers has message consumers too.
                            4. When i push the messages on to the queue that have to be distributes across all the 3 servers and should consume messages
                            5. I don't see this behaviour now. When i push the messages from one server, it consumes messages from that server only. I don't see any active log statements on the other servers.
                            6. I gave the messaging subsystem XML fragment in this thread. I will attach the complete standalone.xml for here for your reference.

                             

                            "Do you see in the log that the HornetQ cluster bridges are forming between the cluster nodes?"

                             

                            Please explain this that how can i check this? what kind of log messages i would see in the log. I will attach one of log messages as well.

                             

                            "Can you confirm that multicast works in the environment where these servers are located?"


                            Please explain this that how can i check this? what kind of log messages i would see in the log. I will attach one of log messages as well.



                              • Actions
                            • 11. Re: Re: Problem setting the JMS jboss module in cluster
                              jbertram
                              jbertram Dec 9, 2013 5:19 PM (in response to klmurty)

                              When i push the messages on to the queue that have to be distributes across all the 3 servers and should consume messages

                              I'm not sure what you mean here.

                               

                              I don't see this behaviour now. When i push the messages from one server, it consumes messages from that server only. I don't see any active log statements on the other servers.

                              If you're consuming from a queue then it's appropriate that only one consumer would receive the message.  That's the basic semantics of a queue.

                               

                              Please explain this that how can i check this? what kind of log messages i would see in the log. I will attach one of log messages as well.

                              You should see something like this:

                               

                              INFO  [org.hornetq.core.server] (Thread-23 (HornetQ-server-HornetQServerImpl::serverUUID=14061e28-1a39-11e3-8e71-d5692c5fe453-5951057)) HQ221027: Bridge ClusterConnectionBridge@fdaad6 [name=sf.my-cluster.14064539-1a39-11e3-8e71-53f3b9c0818d, queue=QueueImpl[name=sf.my-cluster.14064539-1a39-11e3-8e71-53f3b9c0818d, postOffice=PostOfficeImpl [server=HornetQServerImpl::serverUUID=14061e28-1a39-11e3-8e71-d5692c5fe453]]@1dad25e targetConnector=ServerLocatorImpl (identity=(Cluster-connection-bridge::ClusterConnectionBridge@fdaad6 [name=sf.my-cluster.14064539-1a39-11e3-8e71-53f3b9c0818d, queue=QueueImpl[name=sf.my-cluster.14064539-1a39-11e3-8e71-53f3b9c0818d, postOffice=PostOfficeImpl [server=HornetQServerImpl::serverUUID=14061e28-1a39-11e3-8e71-d5692c5fe453]]@1dad25e targetConnector=ServerLocatorImpl [initialConnectors=[TransportConfiguration(name=netty, factory=org-hornetq-core-remoting-impl-netty-NettyConnectorFactory) ?port=5445&host=127-0-0-2], discoveryGroupConfiguration=null]]::ClusterConnectionImpl@2711370[nodeUUID=14061e28-1a39-11e3-8e71-d5692c5fe453, connector=TransportConfiguration(name=netty, factory=org-hornetq-core-remoting-impl-netty-NettyConnectorFactory) ?port=5445&host=localhost, address=jms, server=HornetQServerImpl::serverUUID=14061e28-1a39-11e3-8e71-d5692c5fe453])) [initialConnectors=[TransportConfiguration(name=netty, factory=org-hornetq-core-remoting-impl-netty-NettyConnectorFactory) ?port=5445&host=127-0-0-2], discoveryGroupConfiguration=null]] is connected

                               

                              Please explain this that how can i check this? what kind of log messages i would see in the log. I will attach one of log messages as well.

                              Check with your network administrator to know if multicast is enabled in your environment or not.

                                • Actions
                              • 12. Re: Problem setting the JMS jboss module in cluster
                                klmurty
                                klmurty Dec 9, 2013 5:33 PM (in response to jbertram)

                                Justin Bertram wrote:

                                 

                                When i push the messages on to the queue that have to be distributes across all the 3 servers and should consume messages

                                I'm not sure what you mean here.

                                Murty: I mean, when you push the messages from any one of the servers, all the servers are equally participated in consuming the messages as well. because both producer and consumer code exists in the same application.

                                 

                                I don't see this behaviour now. When i push the messages from one server, it consumes messages from that server only. I don't see any active log statements on the other servers.

                                If you're consuming from a queue then it's appropriate that only one consumer would receive the message.  That's the basic semantics of a queue.

                                Murty: True. It is working as local to that application server.

                                 

                                Please explain this that how can i check this? what kind of log messages i would see in the log. I will attach one of log messages as well.

                                You should see something like this:

                                 

                                1. INFO  [org.hornetq.core.server] (Thread-23 (HornetQ-server-HornetQServerImpl::serverUUID=14061e28-1a39-11e3-8e71-d5692c5fe453-5951057)) HQ221027: Bridge ClusterConnectionBridge@fdaad6 [name=sf.my-cluster.14064539-1a39-11e3-8e71-53f3b9c0818d, queue=QueueImpl[name=sf.my-cluster.14064539-1a39-11e3-8e71-53f3b9c0818d, postOffice=PostOfficeImpl [server=HornetQServerImpl::serverUUID=14061e28-1a39-11e3-8e71-d5692c5fe453]]@1dad25e targetConnector=ServerLocatorImpl (identity=(Cluster-connection-bridge::ClusterConnectionBridge@fdaad6 [name=sf.my-cluster.14064539-1a39-11e3-8e71-53f3b9c0818d, queue=QueueImpl[name=sf.my-cluster.14064539-1a39-11e3-8e71-53f3b9c0818d, postOffice=PostOfficeImpl [server=HornetQServerImpl::serverUUID=14061e28-1a39-11e3-8e71-d5692c5fe453]]@1dad25e targetConnector=ServerLocatorImpl [initialConnectors=[TransportConfiguration(name=netty, factory=org-hornetq-core-remoting-impl-netty-NettyConnectorFactory) ?port=5445&host=127-0-0-2], discoveryGroupConfiguration=null]]::ClusterConnectionImpl@2711370[nodeUUID=14061e28-1a39-11e3-8e71-d5692c5fe453, connector=TransportConfiguration(name=netty, factory=org-hornetq-core-remoting-impl-netty-NettyConnectorFactory) ?port=5445&host=localhost, address=jms, server=HornetQServerImpl::serverUUID=14061e28-1a39-11e3-8e71-d5692c5fe453])) [initialConnectors=[TransportConfiguration(name=netty, factory=org-hornetq-core-remoting-impl-netty-NettyConnectorFactory) ?port=5445&host=127-0-0-2], discoveryGroupConfiguration=null]] is connected 

                                 

                                Murty: I see this message. Please find the log below:

                                Please explain this that how can i check this? what kind of log messages i would see in the log. I will attach one of log messages as well.

                                Check with your network administrator to know if multicast is enabled in your environment or not.

                                 

                                 

                                 

                                15:35:11,692 INFO  [org.hornetq.core.server.impl.HornetQServerImpl] (MSC service thread 1-4) live server is starting with configuration HornetQ Configuration (clustered=true,backup=false,sharedStore=true,journalDirectory=/opt/msp/pkg/jboss-shared/standalone/data/messagingjournal,bindingsDirectory=/opt/msp/pkg/jboss-shared/standalone/data/messagingbindings,largeMessagesDirectory=/opt/msp/pkg/jboss-shared/standalone/data/messaginglargemessages,pagingDirectory=/opt/msp/pkg/jboss-shared/standalone/data/messagingpaging)

                                15:35:11,695 INFO  [org.hornetq.core.server.impl.HornetQServerImpl] (MSC service thread 1-4) Waiting to obtain live lock

                                15:35:11,714 INFO  [org.hornetq.core.persistence.impl.journal.JournalStorageManager] (MSC service thread 1-4) Using AIO Journal

                                15:35:11,868 INFO  [org.hornetq.core.server.impl.AIOFileLockNodeManager] (MSC service thread 1-4) Waiting to obtain live lock

                                15:35:11,869 INFO  [org.hornetq.core.server.impl.AIOFileLockNodeManager] (MSC service thread 1-4) Live Server Obtained live lock

                                15:35:12,216 WARN  [org.hornetq.core.server.cluster.impl.BroadcastGroupImpl] (MSC service thread 1-4) local-bind-address specified for broadcast group but no local-bind-port specified so socket will NOT be bound to a local address/port

                                15:35:12,272 INFO  [org.hornetq.core.remoting.impl.netty.NettyAcceptor] (MSC service thread 1-4) Started Netty Acceptor version 3.2.5.Final-a96d88c 0.0.0.0:5445 for CORE protocol

                                15:35:12,275 INFO  [org.hornetq.core.remoting.impl.netty.NettyAcceptor] (MSC service thread 1-4) Started Netty Acceptor version 3.2.5.Final-a96d88c 0.0.0.0:5455 for CORE protocol

                                15:35:12,276 INFO  [org.hornetq.core.server.impl.HornetQServerImpl] (MSC service thread 1-4) Server is now live

                                15:35:12,277 INFO  [org.hornetq.core.server.impl.HornetQServerImpl] (MSC service thread 1-4) HornetQ Server version 2.2.11.Final (HQ_2_2_11_FINAL_AS7, 122) [6b22cb2b-5ddb-11e3-89cf-00163e146ad0]) started

                                15:35:12,305 INFO  [org.jboss.as.messaging] (MSC service thread 1-2) JBAS011601: Bound messaging object to jndi name java:/RemoteConnectionFactory

                                15:35:12,306 INFO  [org.jboss.as.messaging] (MSC service thread 1-2) JBAS011601: Bound messaging object to jndi name java:jboss/exported/RemoteConnectionFactory

                                15:35:12,307 INFO  [org.hornetq.core.server.impl.HornetQServerImpl] (MSC service thread 1-2) trying to deploy queue jms.queue.BlackBerryAppQueue

                                15:35:12,311 INFO  [org.jboss.as.messaging] (MSC service thread 1-2) JBAS011601: Bound messaging object to jndi name java:jboss/exported/queue/BlackBerryAppQueue

                                15:35:12,312 INFO  [org.jboss.as.messaging] (MSC service thread 1-2) JBAS011601: Bound messaging object to jndi name java:/queue/BlackBerryAppQueue

                                15:35:12,313 INFO  [org.hornetq.core.server.impl.HornetQServerImpl] (MSC service thread 1-2) trying to deploy queue jms.queue.AppAnalysis

                                15:35:12,313 INFO  [org.jboss.as.messaging] (MSC service thread 1-2) JBAS011601: Bound messaging object to jndi name java:/queue/AppAnalysis

                                15:35:12,314 INFO  [org.jboss.as.messaging] (MSC service thread 1-2) JBAS011601: Bound messaging object to jndi name java:jboss/exported/queue/AppAnalysis

                                15:35:12,314 INFO  [org.hornetq.core.server.impl.HornetQServerImpl] (MSC service thread 1-2) trying to deploy queue jms.queue.AndroidAppQueue

                                15:35:12,315 INFO  [org.jboss.as.messaging] (MSC service thread 1-2) JBAS011601: Bound messaging object to jndi name java:/queue/AndroidAppQueue

                                15:35:12,316 INFO  [org.jboss.as.messaging] (MSC service thread 1-2) JBAS011601: Bound messaging object to jndi name java:jboss/exported/queue/AndroidAppQueue

                                15:35:12,324 INFO  [org.jboss.as.messaging] (MSC service thread 1-4) JBAS011601: Bound messaging object to jndi name java:jboss/exported/ConnectionFactory

                                15:35:12,325 INFO  [org.jboss.as.messaging] (MSC service thread 1-4) JBAS011601: Bound messaging object to jndi name java:/ConnectionFactory

                                15:35:12,325 INFO  [org.hornetq.core.server.impl.HornetQServerImpl] (MSC service thread 1-3) trying to deploy queue jms.queue.IOSAppQueue

                                15:35:12,326 INFO  [org.jboss.as.messaging] (MSC service thread 1-3) JBAS011601: Bound messaging object to jndi name java:/queue/IOSAppQueue

                                15:35:12,327 INFO  [org.jboss.as.messaging] (MSC service thread 1-3) JBAS011601: Bound messaging object to jndi name java:jboss/exported/queue/IOSAppQueue

                                15:35:12,471 INFO  [org.jboss.as.deployment.connector] (MSC service thread 1-1) JBAS010406: Registered connection factory java:/JmsXA

                                15:35:12,479 INFO  [org.hornetq.ra.HornetQResourceAdapter] (MSC service thread 1-1) HornetQ resource adaptor started

                                15:35:12,480 INFO  [org.jboss.as.connector.services.ResourceAdapterActivatorService$ResourceAdapterActivator] (MSC service thread 1-1) IJ020002: Deployed: file://RaActivatorhornetq-ra

                                15:35:12,482 INFO  [org.jboss.as.deployment.connector] (MSC service thread 1-1) JBAS010401: Bound JCA ConnectionFactory [java:/JmsXA]

                                15:35:12,550 INFO  [org.hornetq.core.server.cluster.impl.BridgeImpl] (Thread-12 (HornetQ-server-HornetQServerImpl::serverUUID=6b22cb2b-5ddb-11e3-89cf-00163e146ad0-179455452)) Bridge ClusterConnectionBridge@5a0d2bee [name=sf.jms-cluster.f8cfb0de-624d-11e1-bee9-2e1020524153, queue=QueueImpl[name=sf.jms-cluster.f8cfb0de-624d-11e1-bee9-2e1020524153, postOffice=PostOfficeImpl [server=HornetQServerImpl::serverUUID=6b22cb2b-5ddb-11e3-89cf-00163e146ad0]]@16bf4472 targetConnector=ServerLocatorImpl (identity=(Cluster-connection-bridge::ClusterConnectionBridge@5a0d2bee [name=sf.jms-cluster.f8cfb0de-624d-11e1-bee9-2e1020524153, queue=QueueImpl[name=sf.jms-cluster.f8cfb0de-624d-11e1-bee9-2e1020524153, postOffice=PostOfficeImpl [server=HornetQServerImpl::serverUUID=6b22cb2b-5ddb-11e3-89cf-00163e146ad0]]@16bf4472 targetConnector=ServerLocatorImpl [initialConnectors=[org-hornetq-core-remoting-impl-netty-NettyConnectorFactory?port=5445&host=0-0-0-0], discoveryGroupConfiguration=null]]::ClusterConnectionImpl@842453046[nodeUUID=6b22cb2b-5ddb-11e3-89cf-00163e146ad0, connector=org-hornetq-core-remoting-impl-netty-NettyConnectorFactory?port=5445&host=0-0-0-0, address=jms, server=HornetQServerImpl::serverUUID=6b22cb2b-5ddb-11e3-89cf-00163e146ad0])) [initialConnectors=[org-hornetq-core-remoting-impl-netty-NettyConnectorFactory?port=5445&host=0-0-0-0], discoveryGroupConfiguration=null]] is connected

                                15:35:12,633 INFO  [org.jboss.as] (Controller Boot Thread) JBAS015874: JBoss AS 7.1.0.Final "Thunder" started in 3767ms - Started 160 of 231 services (71 services are passive or on-demand)

                                15:35:12,653 INFO  [org.jboss.as.server.deployment] (MSC service thread 1-3) JBAS015876: Starting deployment of "beachberry.war"

                                15:35:12,757 INFO  [org.hornetq.core.server.cluster.impl.BridgeImpl] (Thread-10 (HornetQ-server-HornetQServerImpl::serverUUID=6b22cb2b-5ddb-11e3-89cf-00163e146ad0-179455452)) Bridge ClusterConnectionBridge@2e7b2e05 [name=sf.jms-cluster.4f55d8ed-5dfa-11e3-8b25-00163e280009, queue=QueueImpl[name=sf.jms-cluster.4f55d8ed-5dfa-11e3-8b25-00163e280009, postOffice=PostOfficeImpl [server=HornetQServerImpl::serverUUID=6b22cb2b-5ddb-11e3-89cf-00163e146ad0]]@190a621a targetConnector=ServerLocatorImpl (identity=(Cluster-connection-bridge::ClusterConnectionBridge@2e7b2e05 [name=sf.jms-cluster.4f55d8ed-5dfa-11e3-8b25-00163e280009, queue=QueueImpl[name=sf.jms-cluster.4f55d8ed-5dfa-11e3-8b25-00163e280009, postOffice=PostOfficeImpl [server=HornetQServerImpl::serverUUID=6b22cb2b-5ddb-11e3-89cf-00163e146ad0]]@190a621a targetConnector=ServerLocatorImpl [initialConnectors=[org-hornetq-core-remoting-impl-netty-NettyConnectorFactory?port=5445&host=0-0-0-0], discoveryGroupConfiguration=null]]::ClusterConnectionImpl@842453046[nodeUUID=6b22cb2b-5ddb-11e3-89cf-00163e146ad0, connector=org-hornetq-core-remoting-impl-netty-NettyConnectorFactory?port=5445&host=0-0-0-0, address=jms, server=HornetQServerImpl::serverUUID=6b22cb2b-5ddb-11e3-89cf-00163e146ad0])) [initialConnectors=[org-hornetq-core-remoting-impl-netty-NettyConnectorFactory?port=5445&host=0-0-0-0], discoveryGroupConfiguration=null]] is connected

                                  • Actions
                                • 13. Re: Problem setting the JMS jboss module in cluster
                                  jbertram
                                  jbertram Dec 9, 2013 5:54 PM (in response to klmurty)

                                  I can see you're binding your sever to 0.0.0.0 which will cause problems for HornetQ in the version you're using.  See https://issues.jboss.org/browse/HORNETQ-952 for more details.  In your case it will prevent clustering from working properly, despite the fact that you see the expected message.  You should either not bind to 0.0.0.0 (i.e. bind to a specific IP address or hostname) or upgrade to a later version of AS7 or, preferably, Wildfly.  I think that will enable your cluster to form properly.

                                   

                                  In any event, I'm still not clear on what you're trying to accomplish.  It seems like you have producers running on every cluster node and those producers are sending messages to the local HornetQ broker, but you want those messages to be forwarded to a remote cluster node.  Is that true?  If so, what do you gain by this extra network hop for the message?

                                    • Actions
                                  • 14. Re: Problem setting the JMS jboss module in cluster
                                    klmurty
                                    klmurty Dec 9, 2013 6:56 PM (in response to jbertram)

                                    1) How do i overcome this issue (replacing 0.0.0.0 by actual IP address in standalone.xml) by keeping Jboss 7.1.0. I can't change jboss now because it has lot of other impacts. Please provide the solution. I cant upgrade to higher version now to resolve this issue for now.

                                     

                                    "In any event, I'm still not clear on what you're trying to accomplish.  It seems like you have producers running on every cluster node and those producers are sending messages to the local HornetQ broker, but you want those messages to be forwarded to a remote cluster node.  Is that true?  If so, what do you gain by this extra network hop for the message?"

                                     

                                    yes you are correct. But Can you please suggest for my requirement?

                                     

                                    I want to push messages on to HornetQ and all the nodes consume messages. The messages count is more thats why i want to distribute among the nodes.

                                     

                                    Is my approach correct or do you suggest any good solution for this.

                                      • Actions
                                    1 2 Previous Next