1 2 3 Previous Next 32 Replies Latest reply on Jul 22, 2011 2:08 PM by pczeus

    Cannot configure a UDP based cluster for a JBoss embedded HornetQ.

    swapnonil.mukherjee

      Hi Everybody,

       

      I am trying to set up a hornetq cluster. I am running hornetq 2.0 GA inside JBoss5.1.I am using hornetq's JMS API.

      I have learnt that IP Multicasting is enabled on the RHEL servers on our network. Based on the directions given in the HorentQ Manual Chapter 38, I am trying to form a cluster using two servers located on 10.1.0.74 and 10.1.0.39.

       

      However after doing all that I get the following error.

       

      javax.jms.JMSException: Timed out waiting to receive initial broadcast from discovery group

       

      I am providing the configuration details for the server located at 10.1.0.74. The configuration for the other server is almost identical expect for the connector and acceptor IP addresses.

       

      The following is the hornetq-configuration.xml

       

      <configuration xmlns="urn:hornetq"
                     xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
                     xsi:schemaLocation="urn:hornetq /schema/hornetq-configuration.xsd">
      
         <clustered>true</clustered>
      
         <log-delegate-factory-class-name>org.hornetq.integration.logging.Log4jLogDelegateFactory</log-delegate-factory-class-name>
      
         <bindings-directory>${jboss.server.data.dir}/hornetq/bindings</bindings-directory>
      
         <journal-directory>${jboss.server.data.dir}/hornetq/journal</journal-directory>
      
         <large-messages-directory>${jboss.server.data.dir}/hornetq/largemessages</large-messages-directory>
      
         <paging-directory>${jboss.server.data.dir}/hornetq/paging</paging-directory>
      
         <connectors>
            <connector name="netty">
               <factory-class>org.hornetq.integration.transports.netty.NettyConnectorFactory</factory-class>
               <param key="host"  value="10.1.0.74"/>
               <param key="port"  value="5445"/>
            </connector>
      
            <connector name="in-vm">
               <factory-class>org.hornetq.core.remoting.impl.invm.InVMConnectorFactory</factory-class>
            </connector>
      
         </connectors>
      
         <acceptors>
            <acceptor name="netty">
               <factory-class>org.hornetq.integration.transports.netty.NettyAcceptorFactory</factory-class>
               <param key="host"  value="10.1.0.74"/>
               <param key="port"  value="5445"/>
            </acceptor>
      
            <acceptor name="in-vm">
              <factory-class>org.hornetq.core.remoting.impl.invm.InVMAcceptorFactory</factory-class>
              <param key="server-id" value="0"/>
            </acceptor>
      
          </acceptors>
          
          <broadcast-groups>
            <broadcast-group name="bg-group1">
               <group-address>224.0.0.1</group-address>
               <group-port>9876</group-port>
               <broadcast-period>500</broadcast-period>
               <connector-ref connector-name="netty"/>
            </broadcast-group>
         </broadcast-groups>
      
         <discovery-groups>
            <discovery-group name="dg-group1">
               <group-address>224.0.0.1</group-address>
               <group-port>9876</group-port>
               <refresh-timeout>10000</refresh-timeout>
            </discovery-group>
         </discovery-groups>
      
         <cluster-connections>
            <cluster-connection name="my-cluster">
               <address>jms</address>
                    <discovery-group-ref discovery-group-name="dg-group1"/>
            </cluster-connection>
         </cluster-connections>
      
         <security-settings>
            <security-setting match="#">
               <permission type="createTempQueue" roles="guest"/>
               <permission type="deleteTempQueue" roles="guest"/>
               <permission type="consume" roles="guest"/>
               <permission type="send" roles="guest"/>
            </security-setting>
         </security-settings>
      
         <address-settings>
            <!--default for catch all-->
            <address-setting match="#">
               <dead-letter-address>jms.queue.DLQ</dead-letter-address>
               <expiry-address>jms.queue.ExpiryQueue</expiry-address>
               <redelivery-delay>0</redelivery-delay>
               <max-size-bytes>-1</max-size-bytes>
               <page-size-bytes>10485760</page-size-bytes>
               <message-counter-history-day-limit>10</message-counter-history-day-limit>
            </address-setting>
         </address-settings>
      
      </configuration>
      

       

      The following is hornetq-jms.xml

       

      <configuration xmlns="urn:hornetq"
                  xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
                  xsi:schemaLocation="urn:hornetq /schema/hornetq-jms.xsd">
      
         <connection-factory name="NettyConnectionFactory">
            <connectors>
               <connector-ref connector-name="netty"/>
            </connectors>
            <entries>
               <entry name="/ConnectionFactory"/>
               <entry name="/XAConnectionFactory"/>
            </entries>
         </connection-factory>
         <connection-factory name="ClusteredConnectionFactory">
              <discovery-group-ref discovery-group-name="dg-group1"/>
              <entries>
                      <entry name="/ClusteredConnectionFactory"/>
              </entries>
         </connection-factory>
         <connection-factory name="InVMConnectionFactory">
            <connectors>
               <connector-ref connector-name="in-vm"/>
            </connectors>
            <entries>
               <entry name="java:/ConnectionFactory"/>
               <entry name="java:/XAConnectionFactory"/>
            </entries>
         </connection-factory>
      
         <queue name="DLQ">
            <entry name="/queue/DLQ"/>
         </queue>
      
         <queue name="ExpiryQueue">
            <entry name="/queue/ExpiryQueue"/>
         </queue>
         <queue name="testQueue">
            <entry name="/queue/testQueue"/>
         </queue>
      </configuration>
      

       

      I use JNDI to look up the Connection Factory named ClusteredConnectionFactory

       

      public static Context getInitialContext()throws javax.naming.NamingException
      {
      
              Properties p = new Properties();
              p.put(Context.INITIAL_CONTEXT_FACTORY, "org.jnp.interfaces.NamingContextFactory");
              p.put(Context.URL_PKG_PREFIXES, " org.jboss.naming:org.jnp.interfaces");
              p.put(Context.PROVIDER_URL, "jnp://10.1.0.74:1099");
              return new javax.naming.InitialContext(p);
      }
      

       

      And finally the Connection Creation code is the following.

       

      ConnectionFactory cf = (ConnectionFactory) ic.lookup("/ClusteredConnectionFactory");

       

      What am I doing wrong? I know that multicasting is enabled , but I don't know if the

       

      <group-address>224.0.0.1</group-address>

       

      IP for configuring broadcast and discovery groups is correct or not?

       

      Any Help would be Appreciated.

        • 1. Re: Cannot configure a UDP based cluster for a JBoss embedded HornetQ.
          timfox

          The first thing to do would be to see if the two nodes have formed a cluster.

           

          This can be done by invoking the getNodes() method on the cluster connection control.

           

          Have a search in this forum, it has been discussed several times before.

          • 2. Re: Cannot configure a UDP based cluster for a JBoss embedded HornetQ.
            swapnonil.mukherjee

            Hi Tim,

             

            Thanks for pointing me out to the ClusterConnection JMX attributes. I can indeed see that my first server 10.1.0.74 has knowledge about second server which is 10.1.0.39

             

            NodesRjava.util.MapAttribute exposed for management
            {70cc5a76-55e0-11df-a5af-f1c14af8e790=/10.1.0.39:5445}    

             

            and my second server 10.1.0.39 also has knowledge about first server which was 10.1.0.74

             

            NodesRjava.util.MapAttribute exposed for management
            {a7ea84b2-55da-11df-92f0-df47d213cb48=/10.1.0.74:5445}    

             

            But despite this I am still not able to connect to the "/ClusteredConnectionFactory". It is still giving me the same error

             

            Exception in thread "main" javax.jms.JMSException: Timed out waiting to receive initial broadcast from discovery group

             

            I am posting my entire Java Client which I use to send message.

             

            import java.util.Properties;
            import javax.jms.*;
            import javax.naming.Context;
            
            public class QueueExample
            {
            
                public void example() throws Exception
                {
                    String destinationName = "queue/testQueue";
            
                    Context ic = null;
                    ConnectionFactory cf = null;
                    Connection connection = null;
            
                    try
                    {
                        ic = getInitialContext();
            
                        cf = (ConnectionFactory) ic.lookup("ClusteredConnectionFactory");
                        Queue queue = (Queue) ic.lookup(destinationName);
            
                        connection = cf.createConnection();
                        Session session = connection.createSession(false, Session.AUTO_ACKNOWLEDGE);
                        MessageProducer publisher = session.createProducer(queue);
                        ///MessageConsumer subscriber = session.createConsumer(queue);
            
                        connection.start();
            
                        TextMessage message = session.createTextMessage("Hello!");
                        publisher.send(message);
                        System.out.println("Message sent!");
            
            
                    }
                    finally
                    {
                        if (ic != null)
                        {
                            try
                            {
                                ic.close();
                            }
                            catch (Exception e)
                            {
                                throw e;
                            }
                        }
            
                        closeConnection(connection);
                    }
                }
            
                private void closeConnection(Connection con)
                {
                    try
                    {
                        if (con != null)
                        {
                            con.close();
                        }
                    }
                    catch (JMSException jmse)
                    {
                        System.out.println("Could not close connection " + con + " exception was " + jmse);
                    }
                }
            
            
                public static void main(String[] args) throws Exception
                {
                    new QueueExample().example();
                }
            
                public static Context getInitialContext()throws javax.naming.NamingException
                {
            
                    Properties p = new Properties();
                    p.put(Context.INITIAL_CONTEXT_FACTORY, "org.jnp.interfaces.NamingContextFactory");
                    p.put(Context.URL_PKG_PREFIXES, " org.jboss.naming:org.jnp.interfaces");
                    p.put(Context.PROVIDER_URL, "jnp://10.1.0.74:1099");
                    return new javax.naming.InitialContext(p);
                }
            }
            

             

            Just for the record.

             

            I am running the client code from Windows Box located inside same network on IP 10.1.148.201.

            • 3. Re: Cannot configure a UDP based cluster for a JBoss embedded HornetQ.

              I am having 100% same issue, did you get it to work?
              Regards

              Ravi Gupta

              • 4. Re: Cannot configure a UDP based cluster for a JBoss embedded HornetQ.
                swapnonil.mukherjee

                No not yet Ravi,

                 

                The problem is that I have followed everything that there is in Chapter 38 of HornetQ Manual and it is still not working.

                 

                I am trying to deploy a very simple Message Driven Bean in a cluster which I actually copied pasted from this example.

                http://www.mastertheboss.com/en/jboss-server/236-hornetq-simple-tutorial.html

                • 5. Re: Cannot configure a UDP based cluster for a JBoss embedded HornetQ.

                  Thanks, I have been trying this for days, I really need that to work,

                  Let me know if you find something,

                  Regards

                  Ravi Gupta

                  • 6. Re: Cannot configure a UDP based cluster for a JBoss embedded HornetQ.
                    timfox

                    So you get:

                     

                    javax.jms.JMSException: Timed out waiting to receive initial broadcast from discovery group

                    Have you looked up what this means in the user manual, and tried increasing it?
                    • 7. Re: Cannot configure a UDP based cluster for a JBoss embedded HornetQ.

                      Thanks, can you shed some light on this pls?

                      Running two instances on two physical machines.

                       

                      2:42:38,015 INFO  [TomcatDeployer] deploy, ctxPath=/TestLoggerChicago, warUrl=.../tmp/deploy/tmp4310934260415146148TestLoggerChicago-exp.war/
                      2:42:38,312 INFO  [TomcatDeployer] deploy, ctxPath=/jmx-console, warUrl=.../deploy/jmx-console.war/
                      2:42:38,781 INFO  [Http11Protocol] Starting Coyote HTTP/1.1 on http-127.0.0.1-8080
                      2:42:38,843 INFO  [AjpProtocol] Starting Coyote AJP/1.3 on ajp-127.0.0.1-8009
                      2:42:38,906 INFO  [Server] JBoss (MX MicroKernel) [4.3.0.GA_CP03 (build: SVNTag=JBPAPP_4_3_0_GA_CP03 date=200810241616)] Started in 58s:547ms
                      2:43:04,968 WARN  [DiscoveryGroupImpl] There seem to be more than one broadcasters on the network broadcasting the same node id
                      2:43:05,500 WARN  [DiscoveryGroupImpl] There seem to be more than one broadcasters on the network broadcasting the same node id
                      2:43:06,000 WARN  [DiscoveryGroupImpl] There seem to be more than one broadcasters on the network broadcasting the same node id
                      2:43:06,500 WARN  [DiscoveryGroupImpl] There seem to be more than one broadcasters on the network broadcasting the same node id
                      2:43:07,000 WARN  [DiscoveryGroupImpl] There seem to be more than one broadcasters on the network broadcasting the same node id
                      2:43:07,500 WARN  [DiscoveryGroupImpl] There seem to be more than one broadcasters on the network broadcasting the same node id
                      2:43:08,000 WARN  [DiscoveryGroupImpl] There seem to be more than one broadcasters on the network broadcasting the same node id
                      2:43:08,515 WARN  [DiscoveryGroupImpl] There seem to be more than one broadcasters on the network broadcasting the same node id
                      2:43:08,562 INFO  [TreeCache] viewAccepted(): [10.220.135.96:1182|1] [10.220.135.96:1182, 10.220.135.125:2042]
                      2:43:08,765 INFO  [TreeCache] locking the subtree at / to transfer state
                      2:43:08,796 INFO  [StateTransferGenerator_140] returning the state for tree rooted in /(1024 bytes)
                      2:43:09,015 WARN  [DiscoveryGroupImpl] There seem to be more than one broadcasters on the network broadcasting the same node id
                      2:43:09,515 WARN  [DiscoveryGroupImpl] There seem to be more than one broadcasters on the network broadcasting the same node id
                      2:43:10,015 WARN  [DiscoveryGroupImpl] There seem to be more than one broadcasters on the network broadcasting the same node id
                      2:43:10,515 WARN  [DiscoveryGroupImpl] There seem to be more than one broadcasters on the network broadcasting the same node id
                      2:43:11,015 WARN  [DiscoveryGroupImpl] There seem to be more than one broadcasters on the network broadcasting the same node id
                      2:43:11,515 WARN  [DiscoveryGroupImpl] There seem to be more than one broadcasters on the network broadcasting the same node id
                      2:43:12,015 WARN  [DiscoveryGroupImpl] There seem to be more than one broadcasters on the network broadcasting the same node id
                      2:43:12,515 WARN  [DiscoveryGroupImpl] There seem to be more than one broadcasters on the network broadcasting the same node id
                      2:43:13,015 WARN  [DiscoveryGroupImpl] There seem to be more than one broadcasters on the network broadcasting the same node id
                      2:43:13,515 WARN  [DiscoveryGroupImpl] There seem to be more than one broadcasters on the network broadcasting the same node id
                      2:43:14,015 WARN  [DiscoveryGroupImpl] There seem to be more than one broadcasters on the network broadcasting the same node id
                      2:43:14,515 WARN  [DiscoveryGroupImpl] There seem to be more than one broadcasters on the network broadcasting the same node id
                      2:43:15,015 WARN  [DiscoveryGroupImpl] There seem to be more than one broadcasters on the network broadcasting the same node id
                      2:43:15,515 WARN  [DiscoveryGroupImpl] There seem to be more than one broadcasters on the network broadcasting the same node id

                      • 8. Re: Cannot configure a UDP based cluster for a JBoss embedded HornetQ.
                        timfox

                        http://hornetq.sourceforge.net/docs/hornetq-2.0.0.GA/user-manual/en/html_single/index.html#clusters-discovery.groups.clientside

                         

                        "There is also a further parameter settable on the connection factory using the                     setter method setInitialWaitTimeout(). If the connection                     factory is used immediately after creation then it may not have had enough time                     to received broadcasts from all the nodes in the cluster. On first usage, the                     connection factory will make sure it waits this long since creation before                     creating the first connection. The default value for this parameter is 2000 milliseconds."

                        • 9. Re: Cannot configure a UDP based cluster for a JBoss embedded HornetQ.
                          timfox

                          So I guess you didn't look in JIRA?

                           

                          https://jira.jboss.org/jira/browse/HORNETQ-291

                           

                          Have you tried TRUNK?

                          • 10. Re: Cannot configure a UDP based cluster for a JBoss embedded HornetQ.
                            swapnonil.mukherjee

                            Hi Tim,

                             

                            I have read that documentation, but I am not getting it.

                             

                            Where, in which class is the setInitialWaitTimeout() method defined?

                            • 11. Re: Cannot configure a UDP based cluster for a JBoss embedded HornetQ.
                              swapnonil.mukherjee

                              Also had a look under the examples/jms/symmetric-cluster directory.

                               

                              The example clearly states that they are not using JNDI look up to get Connection Factories.  There is even a comment in the hornetq-jms.xml which says that "connection factory is not used in the example"

                               

                              <configuration xmlns="urn:hornetq"
                                          xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
                                          xsi:schemaLocation="urn:hornetq /schema/hornetq-jms.xsd">
                                          
                                 <!--This connection factory is not actually used in the example, but we keep it here for good
                                 measure -->
                                 <connection-factory name="ConnectionFactory">
                                    <discovery-group-ref discovery-group-name="my-discovery-group"/>
                                    <entries>
                                       <entry name="ConnectionFactory"/>
                                    </entries>
                                 </connection-factory>
                                 
                                 <queue name="exampleQueue">
                                    <entry name="/queue/exampleQueue"/>
                                 </queue>
                                 
                                 <topic name="exampleTopic">
                                    <entry name="/topic/exampleTopic"/>
                                 </topic>
                               
                              </configuration>
                              

                               

                              So this brings me to the question. Do Connection Factories and JNDI look up even work with symmetric clustering?

                              • 12. Re: Cannot configure a UDP based cluster for a JBoss embedded HornetQ.
                                clebert.suconic

                                "Do Connection Factories and JNDI look up even work with symmetric clustering?"

                                 

                                 

                                It's your choice. JNDI is just a way to deliver the factory to the client.

                                • 13. Re: Cannot configure a UDP based cluster for a JBoss embedded HornetQ.
                                  swapnonil.mukherjee

                                  Hi Clebert,

                                   

                                  I would love to use JNDI but if this does not work with the symmetric cluster setup, I could use the following

                                   

                                  final String groupAddress = "231.7.7.7";
                                  final int groupPort = 9876;
                                  ConnectionFactory jmsConnectionFactory = HornetQJMSClient.createConnectionFactory(groupAddress, groupPort);
                                  Connection jmsConnection1 = jmsConnectionFactory.createConnection();
                                  

                                   

                                  as mentioned in the documentation, but even with this I am seeing the same "

                                  Timed out waiting to receive initial broadcast from discovery group

                                  " problem.

                                   

                                  Is there a way to specify the setInitialWaitTimeout somewhere so that I can get over this problem?

                                  • 14. Re: Cannot configure a UDP based cluster for a JBoss embedded HornetQ.
                                    timfox

                                    Did you see my post on the other thread?

                                     

                                    I guess it's too much to actually expect people to read the documentation.

                                     

                                    Surprise, surprise this is explained in the chapter on clustering, the same chapter that deals with UDP discovery:

                                     

                                    http://hornetq.sourceforge.net/docs/hornetq-2.0.0.GA/user-manual/en/html_single/index.html#clusters-discovery.groups.clientside

                                    1 2 3 Previous Next