4 Replies Latest reply on Oct 12, 2010 6:19 AM by Younes Yahyaoui

    JBossESB 4.8 clustering (with shared jUDDI registry)

    daniele sacchetti Newbie

      I'm trying to implement the ESB clustering as described in the Architecture presented in Chaptyer 7 (Advanced Topics) of the JBossESB 4.8 Programmers Guide.

       

      The first issue I'm facing is the setup of a jUDDI registry shared between two instances of JBoss ESB. I'm running two instances of jbossesb-server 4.8.

       

      For the first I've just unzipped the distribution without any changes.

       

      For the second instance I've modified the following configuration files:
      -jbossesb-server-4.8/server/default/conf/jndi.properties
          added aline:     
                 java.naming.provider.url=jnp://172.17.4.31:1099
      - jbossesb-server-4.8/server/default/deploy/jbossesb.sar/esb.juddi.client.xml
          uncommented the RMI section

       


      I start the first (IP address = 172.17.4.31) with command:
      ./run.sh  -c default --host=0.0.0.0

       


      I start the second with command:
      ./run.sh  -Djboss.esb.bind.address=172.17.4.31 -c default --host=0.0.0.0

       

      And on the console of the second instance I get the following exceptions:

       

      16:27:56,335 WARN  [ServiceController] Problem starting service jboss.jmx:type=adaptor,name=Invoker,protocol=jrmp,service=proxyFactory
      javax.naming.NameAlreadyBoundException
      16:27:43,771 WARN  [ServiceController] Problem starting service jboss:service=proxyFactory,target=ClientUserTransactionFactory
      javax.naming.NameAlreadyBoundException
      16:28:04,021 ERROR [JNDIRegistration]
      javax.naming.NameAlreadyBoundException; remaining name 'juddiv3'
      16:28:05,095 ERROR [ExceptionUtil] Queue[/queue/DLQ] startService
      javax.jms.InvalidDestinationException: Destination DLQ already exists
      16:28:05,101 ERROR [ExceptionUtil] Queue[/queue/ExpiryQueue] startService
      javax.jms.InvalidDestinationException: Destination ExpiryQueue already exists
      16:28:05,121 ERROR [ExceptionUtil] Topic[/topic/testTopic] startService
      javax.jms.InvalidDestinationException: Destination testTopic already exists
      16:28:05,125 ERROR [ExceptionUtil] Topic[/topic/securedTopic] startService
      javax.jms.InvalidDestinationException: Destination securedTopic already exists
      16:28:05,145 ERROR [ExceptionUtil] Topic[/topic/testDurableTopic] startService
      javax.jms.InvalidDestinationException: Destination testDurableTopic already exists
      16:28:05,167 ERROR [ExceptionUtil] Queue[/queue/testQueue] startService
      javax.jms.InvalidDestinationException: Destination testQueue already exists
      .....
      .....
      16:28:05,258 ERROR [ExceptionUtil] Queue[/queue/PrivateDLQ] startService
      javax.jms.InvalidDestinationException: Destination PrivateDLQ already exists
      16:28:05,289 ERROR [ExceptionUtil] Queue[/queue/QueueWithOwnDLQAndExpiryQueue] startService
      javax.jms.InvalidDestinationException: Destination QueueWithOwnDLQAndExpiryQueue already exists
      16:28:12,149 ERROR [ExceptionUtil] Queue[/queue/DeadMessageQueue] startService
      javax.jms.InvalidDestinationException: Destination DeadMessageQueue already exists

       


      At the end of the startup phase of the second instance I get this message:

       

      16:28:20,559 ERROR [URLDeploymentScanner] Incomplete Deployment listing:
      16:28:21,168 ERROR [Server] Root deployment has missing dependencies; continuing Incomplete Deployment listing:

       

      --- MBEANS THAT ARE THE ROOT CAUSE OF THE PROBLEM ---
      ObjectName: jboss.jmx:type=adaptor,name=Invoker,protocol=jrmp,service=proxyFactory
        State: FAILED
        Reason: javax.naming.NameAlreadyBoundException
        I Depend On:
          jboss:service=invoker,type=jrmp
          jboss.jmx:type=adaptor,name=Invoker
          jboss:service=Naming
        Depends On Me:
          jboss.admin:service=PluginManager
          jboss.jmx:type=adaptor,name=MBeanProxyRemote,protocol=jrmp

       

      ObjectName: jboss:service=proxyFactory,target=ClientUserTransactionFactory
        State: FAILED
        Reason: javax.naming.NameAlreadyBoundException
        I Depend On:
          jboss:service=invoker,type=unified
        Depends On Me:
          jboss:service=ClientUserTransaction

       

       

       

      Are there any on line documents that could help me to properly implement my clustered architecture?
      Am I missing any other configuration files to properly setup the connection between the two instances and the jUDDI registry?

       

      Thank you,
      Daniele

        • 1. Re: JBossESB 4.8 clustering (with shared jUDDI registry)
          Nikos Massios Novice

          Hello Danielle,

           

          I am trying to cluster JBossESB 4.6 and I am getting same problems.

           

          I have clustered JBoss 5.1GA with Apache and Mod-cluster but now I am trying to cluster the ESB and I am getting problems with the queues

          It is trying to create the same queues again (even though they are already created by the first instance going up) and this is creating mayhem.

           

          Have you managed to get a solution?

           

          Nikos

          • 2. Re: JBossESB 4.8 clustering (with shared jUDDI registry)
            daniele sacchetti Newbie

            Hello Nikos,

             

            Still not fixed but I've solved some issues and I've moved forward:

             

            Now I've two hosts with AS5.1.0+ESB4.6, a third host configured as jUDDI registry (AS5.1.0) and a forth host with PostgreSQL 8.4.4 (used for JBMessaging clustering).

             

            For ESB clustering I'm not using Apache and Mod-cluster but JGroups+JBMessaging clustering.

             

            I've found some documentation and links to help deploying clustering based on JGroups+JBMessaging clustering:

            - Setup JBMessaging clustering:

            http://community.jboss.org/message/552180#552180

             

            - Replace hsqldb with other DB for message persistence:

            http://community.jboss.org/wiki/JBossESBSwitchDatabase

            http://community.jboss.org/wiki/configdatasources

             

            Concerning the exception in my post, I think the problem is due to JNDI problem: I'm using the same JNDI registry located at 172.17.4.31:1099

            (java.naming.provider.url=jnp://172.17.4.31:1099) for both AS+ESB hosts and I think (I have not solved this issue yet) this is not the correct solution to have a centralize jUDDI registry. I think the correct solution is as described in steps 2 and 4 of http://community.jboss.org/wiki/JBossESBSwitchDatabase,that is, just based on a shared DB.

             

            Daniele

            • 3. Re: JBossESB 4.8 clustering (with shared jUDDI registry)
              Nikos Massios Novice

              Hello Daniele,

               

              I got it working. I can call the ESB either by HTTP via the apache mod-cluster and by JMS gateway.

               

              I did also some basic steps copied from Ryan Hochstetler (thanks Ryan)

              http://community.jboss.org/message/552180#552180

               

              I am using oracle for a db, clustered queues and a clustered post office as explained by Ryan

              I am using the -g -u options at the start up. For example for node 2 I do

               

              run.bat -c node2 -gLocalDevMassios -u 239.255.100.100 -b <YOUR IP HERE> -Djboss.messaging.ServerPeerID=2 -Djboss.service.binding.set=ports-01 -Djboss.mod_cluster.jvmRoute=node2

               

              In the example above I am setting the serverpeerid for jboss messaging. I am also setting the jvm route for mod proxy. You do not need to set the jvm_route if you do not cluster http

               

              After having done all those changes I was still getting the errors you mentioned. The extra changes that fixed the problem were on

              C:\jboss-5.1.0.GA-ESB4.6\server\node2\deployers\esb.deployer\esb.juddi.xml

              C:\jboss-5.1.0.GA-ESB4.6\server\node2\deployers\esb.deployer\jbossesb-properties.xml

              These files are copied on startup to

              C:\jboss-5.1.0.GA-ESB4.6\server\node2\data\

              These files contained the ip's of node1. I changed those ip's to contain the local ones. I think this is something that could be improved in the esb because it took me quite some time to find out what was going on. These files should be fixed for each node of the ESB.

               

              In terms of configuration I have 4 identical ESB nodes each with its own JUDDI, messaging, admin consoles etc.  (Although they all work with the same db)

               

              Our ESB does service routing for a single request. Because of that and because of the clustering some of the processing happens on one node while some happens on another. So the clustering works even within the same request which I am not sure if it is a good idea or not. I would like some answer on whether this can be disabled somehow.

               

              Nikos

              • 4. Re: JBossESB 4.8 clustering (with shared jUDDI registry)
                Younes Yahyaoui Novice

                Hi daniele,

                 

                I start the first (IP address = 172.17.4.31) with command:

                ./run.sh  -c default --host=0.0.0.0

                 

                with default you canno't start a JBossESB Cluster , you need all configuration because default configuration don't support clustering.

                 

                --Regards

                 

                younes