6 Replies Latest reply on Nov 17, 2011 8:44 PM by amathewjboss1

    second node won't start in clustering

    pxiaoott

      Hello,

       

      I have setup jboss 5.1 cluster with two nodes (on two separate servers). Start first node is always no problem. Everytime when i started the second node, the second node throwed the following exception. And the second node started but with problem. I'm running jboss5.1 on Solaris10. It happened after having installed an identity manager application.

       

      Please help. Thanks in advance.

       

       

       

      2011-11-03 16:39:44,770 INFO  [org.jboss.web.tomcat.service.deployers.TomcatDeployment] (main) deploy, ctxPath=/admin-console
      2011-11-03 16:39:44,955 INFO  [javax.enterprise.resource.webcontainer.jsf.config] (main) Initializing Mojarra (1.2_12-b01-FCS)
      for context '/admin-console'
      2011-11-03 16:39:50,031 INFO  [org.jboss.web.tomcat.service.deployers.TomcatDeployment] (main) deploy, ctxPath=/
      2011-11-03 16:39:50,143 INFO  [org.jboss.web.tomcat.service.deployers.TomcatDeployment] (main) deploy, ctxPath=/jmx-console
      2011-11-03 16:40:30,706 ERROR [org.jboss.system.server.profileservice.repository.ScopedProfileServiceController] (main) Error
      installing to Create: name=ProfileKey@26d560[domain=default, server=default, name=farm] state=Configured mode=On Demand requiredState=Installed
      java.lang.reflect.InvocationTargetException
              at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
              at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
              at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
              at java.lang.reflect.Method.invoke(Method.java:597)
              at org.jboss.system.server.profileservice.repository.AbstractProfileLifeCycleAction.invoke(AbstractProfileLifeCycleAct
      ion.java:97)
              at org.jboss.system.server.profileservice.repository.AbstractProfileLifeCycleAction.invoke(AbstractProfileLifeCycleAct
      ion.java:77)
              at org.jboss.system.server.profileservice.repository.AbstractProfileLifeCycleAction.install(AbstractProfileLifeCycleAc
      tion.java:49)
              at org.jboss.system.server.profileservice.repository.AbstractProfileAction.install(AbstractProfileAction.java:53)
              at org.jboss.system.server.profileservice.repository.AbstractProfileService.install(AbstractProfileService.java:361)
              at org.jboss.dependency.plugins.AbstractControllerContext.install(AbstractControllerContext.java:348)
              at org.jboss.dependency.plugins.AbstractController.install(AbstractController.java:1631)
              at org.jboss.dependency.plugins.AbstractController.incrementState(AbstractController.java:934)
              at org.jboss.dependency.plugins.AbstractController.resolveContexts(AbstractController.java:1082)
              at org.jboss.dependency.plugins.AbstractController.resolveContexts(AbstractController.java:984)
              at org.jboss.dependency.plugins.AbstractController.change(AbstractController.java:822)
              at org.jboss.dependency.plugins.AbstractController.change(AbstractController.java:553)
              at org.jboss.system.server.profileservice.repository.AbstractProfileService.activateProfile(AbstractProfileService.jav
      a:306)
              at org.jboss.system.server.profileservice.ProfileServiceBootstrap.start(ProfileServiceBootstrap.java:271)
              at org.jboss.bootstrap.AbstractServerImpl.start(AbstractServerImpl.java:461)
              at org.jboss.Main.boot(Main.java:221)
              at org.jboss.Main$1.run(Main.java:556)
              at java.lang.Thread.run(Thread.java:662)
      Caused by: java.lang.RuntimeException: java.io.FileNotFoundException: /disk/apps/jboss-5.1.0.GA/server/all/farm/iam_im.ear/config/iam_migration.trigger (No such file or directory)
              at org.jboss.profileservice.cluster.repository.DefaultRepositoryClusteringHandler.rethrowAsUnchecked(DefaultRepository
      ClusteringHandler.java:951)
              at org.jboss.profileservice.cluster.repository.DefaultRepositoryClusteringHandler.installModifications(DefaultReposito
      ryClusteringHandler.java:630)
              at org.jboss.profileservice.cluster.repository.DefaultRepositoryClusteringHandler.synchronizeContent(DefaultRepository
      ClusteringHandler.java:324)
              at org.jboss.system.server.profileservice.repository.clustered.ClusteredDeploymentRepository.load(ClusteredDeploymentR
      epository.java:255)
              at org.jboss.system.server.profile.repository.AbstractProfile.create(AbstractProfile.java:158)
              ... 22 more
      Caused by: java.io.FileNotFoundException: /disk/apps/jboss-5.1.0.GA/server/all/farm/iam_im.ear/config/iam_migration.trigger (N
      o such file or directory)
              at java.io.FileInputStream.open(Native Method)
              at java.io.FileInputStream.<init>(FileInputStream.java:106)
              at org.jboss.system.server.profileservice.repository.clustered.local.file.FileReadAction.getInputStream(FileReadAction
      .java:164)
              at org.jboss.system.server.profileservice.repository.clustered.local.file.FileReadAction.getNextBytes(FileReadAction.j
      ava:81)
              at org.jboss.profileservice.cluster.repository.DefaultRepositoryClusteringHandler$RpcTarget.pullBytes(DefaultRepositor
      yClusteringHandler.java:1097)
              at sun.reflect.GeneratedMethodAccessor413.invoke(Unknown Source)
              at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
              at java.lang.reflect.Method.invoke(Method.java:597)
              at org.jgroups.blocks.MethodCall.invoke(MethodCall.java:331)
              at org.jboss.ha.framework.server.ClusterPartition$RpcHandler.handle(ClusterPartition.java:2266)
              at org.jgroups.blocks.RequestCorrelator.handleRequest(RequestCorrelator.java:637)
              at org.jgroups.blocks.RequestCorrelator.receiveMessage(RequestCorrelator.java:545)
              at org.jgroups.blocks.RequestCorrelator.receive(RequestCorrelator.java:368)
              at org.jgroups.blocks.MessageDispatcher$ProtocolAdapter.up(MessageDispatcher.java:775)
              at org.jgroups.JChannel.up(JChannel.java:1339)
              at org.jgroups.stack.ProtocolStack.up(ProtocolStack.java:462)
              at org.jgroups.protocols.pbcast.FLUSH.up(FLUSH.java:435)
              at org.jgroups.protocols.pbcast.STATE_TRANSFER.up(STATE_TRANSFER.java:144)
              at org.jgroups.protocols.FRAG2.up(FRAG2.java:188)
              at org.jgroups.protocols.FC.up(FC.java:473)
              at org.jgroups.protocols.pbcast.GMS.up(GMS.java:824)
              at org.jgroups.protocols.pbcast.STABLE.up(STABLE.java:233)
              at org.jgroups.protocols.UNICAST.handleDataReceived(UNICAST.java:616)
              at org.jgroups.protocols.UNICAST.up(UNICAST.java:282)
              at org.jgroups.protocols.pbcast.NAKACK.up(NAKACK.java:758)
              at org.jgroups.protocols.VERIFY_SUSPECT.up(VERIFY_SUSPECT.java:167)
              at org.jgroups.protocols.FD.up(FD.java:284)
              at org.jgroups.protocols.FD_SOCK.up(FD_SOCK.java:309)
              at org.jgroups.protocols.MERGE2.up(MERGE2.java:144)
              at org.jgroups.protocols.Discovery.up(Discovery.java:264)
              at org.jgroups.protocols.TP$ProtocolAdapter.up(TP.java:2327)
              at org.jgroups.protocols.TP.passMessageUp(TP.java:1261)
              at org.jgroups.protocols.TP.access$100(TP.java:49)
              at org.jgroups.protocols.TP$IncomingPacket.handleMyMessage(TP.java:1838)
              at org.jgroups.protocols.TP$IncomingPacket.run(TP.java:1812)
              at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
              at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
              ... 1 more
      2011-11-03 16:40:30,710 ERROR [STDERR] (main) Failed to boot JBoss:
      2011-11-03 16:40:30,710 ERROR [STDERR] (main) java.lang.IllegalStateException: Incompletely deployed:

      *** PROFILES IN ERROR: Name -> Error
      Profile: ProfileKey@26d560[domain=default, server=default, name=farm] in error due to java.lang.reflect.InvocationTargetException

      *** PROFILES MISSING DEPENDENCIES: Name -> Dependency{Required State:Actual State}
      ProfileKey@1f85a86[domain=default, server=default, name=all] is missing following dependencies: ProfileKey@26d560[domain=defau
      lt, server=default, name=farm]{Installed:**ERROR**}
      2011-11-03 16:40:30,710 ERROR [STDERR] (main)   at org.jboss.system.server.profileservice.repository.AbstractProfileService.lo
      gErrors(AbstractProfileService.java:567)
      2011-11-03 16:40:30,710 ERROR [STDERR] (main)   at org.jboss.system.server.profileservice.repository.AbstractProfileService.va
      lidate(AbstractProfileService.java:348)
      2011-11-03 16:40:30,710 ERROR [STDERR] (main)   at org.jboss.system.server.profileservice.repository.AbstractProfileService.va
      lidateProfile(AbstractProfileService.java:331)
      2011-11-03 16:40:30,710 ERROR [STDERR] (main)   at org.jboss.system.server.profileservice.ProfileServiceBootstrap.start(Profil
      eServiceBootstrap.java:272)
      2011-11-03 16:40:30,710 ERROR [STDERR] (main)   at org.jboss.bootstrap.AbstractServerImpl.start(AbstractServerImpl.java:461)
      2011-11-03 16:40:30,710 ERROR [STDERR] (main)   at org.jboss.Main.boot(Main.java:221)
      2011-11-03 16:40:30,711 ERROR [STDERR] (main)   at org.jboss.Main$1.run(Main.java:556)
      2011-11-03 16:40:30,711 ERROR [STDERR] (main)   at java.lang.Thread.run(Thread.java:662)
      (END)

        • 1. Re: second node won't start in clustering
          wdfink

          How do you install and start the nodes?

          You should start with two fresh unziped JBoss instances and start only a cluster without other changes.

          If you do your changes step-by-step it will be easier to find out what's going wrong when your config is changed or you deploy something.

           

          BTW you should mark your threads if you found an answer

          • 2. Re: second node won't start in clustering
            rhusar

            Hey pxiaoott, the cause is quite obvious:

             

            Caused by: java.io.FileNotFoundException: /disk/apps/jboss-5.1.0.GA/server/all/farm/iam_im.ear/config/iam_migration.trigger (N

            o such file or directory)

            Address that first.

             

            Rado

            • 3. Re: second node won't start in clustering
              pxiaoott

              Thanks for the responses.

               

              After wipe databases and start do fresh install, The error above went away. But I have different error.

               

              1) unzip jboss zip and start as cluster mode on node 1

              2) unzip jboss zip and start as cluster mode on node 2

              3) both nodes are running ok

              4) now install application (identiymanager) on node 1, start as cluster mode on node1, success

              5) now install same application on node 2. start jboss as cluster mode on node 2. Failed. The node2 showed the following errors.

              6) or vice versa, start node2 first with no issue. when starting node1, the node1 throwed same error as below.

               

              We're using jboss 5.1 (on solaris10) and cluster is on tcp. Can someone provide a direction to troubleshoot such cluster problem? Your help is much appreciated.

               

              server.log

              ---------------------------------------------------------
              GMS: address is 7.33.44.147:7600 (cluster=IdmPartitionTRANS-CTRL)
              ---------------------------------------------------------
              2011-11-17 10:59:21,804 INFO  [org.jboss.messaging.core.impl.postoffice.GroupMember] (main) org.jboss.messaging.core.impl.postoffice.GroupMember$ControlMembershipListener@1b9e0fd got new view [7.33.45.147:7600|1] [7.33.45.147:7600, 7.33.44.147:7600], old view is null
              2011-11-17 10:59:21,804 INFO  [org.jboss.messaging.core.impl.postoffice.GroupMember] (main) I am (7.33.44.147:7600)
              2011-11-17 10:59:21,805 INFO  [org.jboss.messaging.core.impl.postoffice.GroupMember] (main) New Members : 2 ([7.33.45.147:7600, 7.33.44.147:7600])
              2011-11-17 10:59:21,805 INFO  [org.jboss.messaging.core.impl.postoffice.GroupMember] (main) All Members : 2 ([7.33.45.147:7600, 7.33.44.147:7600])
              2011-11-17 10:59:22,146 INFO  [STDOUT] (main)
              ---------------------------------------------------------
              GMS: address is 7.33.44.147:7900 (cluster=IdmPartitionTRANS-DATA)
              ---------------------------------------------------------
              2011-11-17 10:59:25,303 ERROR [org.jboss.messaging.util.ExceptionUtil] (main) org.jboss.messaging.core.jmx.MessagingPostOffice
              Service@18ae49a startService
              java.lang.IllegalArgumentException: 7.33.45.147:7900 is not a member of the group [7.33.44.147:7900] (enabled_members=[])
                      at org.jgroups.protocols.UNICAST.down(UNICAST.java:332)
                      at org.jgroups.protocols.pbcast.STABLE.down(STABLE.java:316)
                      at org.jgroups.protocols.VIEW_SYNC.down(VIEW_SYNC.java:204)
                      at org.jgroups.protocols.pbcast.GMS.down(GMS.java:905)
                      at org.jgroups.stack.ProtocolStack.down(ProtocolStack.java:475)
                      at org.jgroups.JChannel.down(JChannel.java:1517)
                      at org.jgroups.JChannel.send(JChannel.java:626)
                      at org.jboss.messaging.core.impl.postoffice.GroupMember.unicastData(GroupMember.java:320)
                      at org.jboss.messaging.core.impl.postoffice.MessagingPostOffice.requestDeliveries(MessagingPostOffice.java:1716)
                      at org.jboss.messaging.core.impl.postoffice.MessagingPostOffice.loadBindings(MessagingPostOffice.java:2583)
                      at org.jboss.messaging.core.impl.postoffice.MessagingPostOffice.start(MessagingPostOffice.java:394)
                      at org.jboss.messaging.core.jmx.MessagingPostOfficeService.startService(MessagingPostOfficeService.java:462)
                      at org.jboss.system.ServiceMBeanSupport.jbossInternalStart(ServiceMBeanSupport.java:376)
                      at org.jboss.system.ServiceMBeanSupport.start(ServiceMBeanSupport.java:269)
                      at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
                      at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
                      at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
                      at java.lang.reflect.Method.invoke(Method.java:597)
                      at org.jboss.mx.interceptor.ReflectedDispatcher.invoke(ReflectedDispatcher.java:157)
                      at org.jboss.mx.server.Invocation.dispatch(Invocation.java:96)
                      at org.jboss.mx.interceptor.AbstractInterceptor.invoke(AbstractInterceptor.java:138)
                      at org.jboss.mx.server.Invocation.invoke(Invocation.java:90)
                      at org.jboss.mx.interceptor.ModelMBeanOperationInterceptor.invoke(ModelMBeanOperationInterceptor.java:140)
                      at org.jboss.mx.server.Invocation.invoke(Invocation.java:90)
                      at org.jboss.mx.server.AbstractMBeanInvoker.invoke(AbstractMBeanInvoker.java:264)
                      at org.jboss.mx.server.MBeanServerImpl.invoke(MBeanServerImpl.java:668)
                      at org.jboss.system.microcontainer.ServiceProxy.invoke(ServiceProxy.java:206)
                      at $Proxy38.start(Unknown Source)
                      at org.jboss.system.microcontainer.StartStopLifecycleAction.installAction(StartStopLifecycleAction.java:42)
                      at org.jboss.system.microcontainer.StartStopLifecycleAction.installAction(StartStopLifecycleAction.java:37)
                      at org.jboss.dependency.plugins.action.SimpleControllerContextAction.simpleInstallAction(SimpleControllerContextAction
              .java:62)
                      at org.jboss.dependency.plugins.action.AccessControllerContextAction.install(AccessControllerContextAction.java:71)
                      at org.jboss.dependency.plugins.AbstractControllerContextActions.install(AbstractControllerContextActions.java:51)
                      at org.jboss.dependency.plugins.AbstractControllerContext.install(AbstractControllerContext.java:348)
                      at org.jboss.system.microcontainer.ServiceControllerContext.install(ServiceControllerContext.java:286)
                      at org.jboss.dependency.plugins.AbstractController.install(AbstractController.java:1631)
                      at org.jboss.dependency.plugins.AbstractController.incrementState(AbstractController.java:934)
                      at org.jboss.dependency.plugins.AbstractController.resolveContexts(AbstractController.java:1082)
                      at org.jboss.dependency.plugins.AbstractController.resolveContexts(AbstractController.java:984)
                      at org.jboss.dependency.plugins.AbstractController.change(AbstractController.java:822)
                      at org.jboss.dependency.plugins.AbstractController.change(AbstractController.java:553)
                      at org.jboss.system.ServiceController.doChange(ServiceController.java:688)
                      at org.jboss.system.ServiceController.start(ServiceController.java:460)
                      at org.jboss.system.deployers.ServiceDeployer.start(ServiceDeployer.java:163)
                      at org.jboss.system.deployers.ServiceDeployer.deploy(ServiceDeployer.java:99)
                      at org.jboss.system.deployers.ServiceDeployer.deploy(ServiceDeployer.java:46)
                      at org.jboss.deployers.spi.deployer.helpers.AbstractSimpleRealDeployer.internalDeploy(AbstractSimpleRealDeployer.java:62)
                      at org.jboss.deployers.spi.deployer.helpers.AbstractRealDeployer.deploy(AbstractRealDeployer.java:50)
                      at org.jboss.deployers.plugins.deployers.DeployerWrapper.deploy(DeployerWrapper.java:171)
                      at org.jboss.deployers.plugins.deployers.DeployersImpl.doDeploy(DeployersImpl.java:1439)
                      at org.jboss.deployers.plugins.deployers.DeployersImpl.doInstallParentFirst(DeployersImpl.java:1157)
                      at org.jboss.deployers.plugins.deployers.DeployersImpl.doInstallParentFirst(DeployersImpl.java:1178)
                      at org.jboss.deployers.plugins.deployers.DeployersImpl.install(DeployersImpl.java:1098)
                      at org.jboss.dependency.plugins.AbstractControllerContext.install(AbstractControllerContext.java:348)
                      at org.jboss.dependency.plugins.AbstractController.install(AbstractController.java:1631)
                      at org.jboss.dependency.plugins.AbstractController.incrementState(AbstractController.java:934)
                      at org.jboss.dependency.plugins.AbstractController.resolveContexts(AbstractController.java:1082)
                      at org.jboss.dependency.plugins.AbstractController.resolveContexts(AbstractController.java:984)
                      at org.jboss.dependency.plugins.AbstractController.change(AbstractController.java:822)
                      at org.jboss.dependency.plugins.AbstractController.change(AbstractController.java:553)
                      at org.jboss.deployers.plugins.deployers.DeployersImpl.process(DeployersImpl.java:781)
                      at org.jboss.deployers.plugins.main.MainDeployerImpl.process(MainDeployerImpl.java:702)
                      at org.jboss.system.server.profileservice.repository.MainDeployerAdapter.process(MainDeployerAdapter.java:117)
                      at org.jboss.system.server.profileservice.repository.ProfileDeployAction.install(ProfileDeployAction.java:70)
                      at org.jboss.system.server.profileservice.repository.AbstractProfileAction.install(AbstractProfileAction.java:53)
                      at org.jboss.system.server.profileservice.repository.AbstractProfileService.install(AbstractProfileService.java:361)
                      at org.jboss.dependency.plugins.AbstractControllerContext.install(AbstractControllerContext.java:348)
                      at org.jboss.dependency.plugins.AbstractController.install(AbstractController.java:1631)
                      at org.jboss.dependency.plugins.AbstractController.incrementState(AbstractController.java:934)
                      at org.jboss.dependency.plugins.AbstractController.resolveContexts(AbstractController.java:1082)
                      at org.jboss.dependency.plugins.AbstractController.resolveContexts(AbstractController.java:984)
                      at org.jboss.dependency.plugins.AbstractController.change(AbstractController.java:822)
                      at org.jboss.dependency.plugins.AbstractController.change(AbstractController.java:553)
                      at org.jboss.system.server.profileservice.repository.AbstractProfileService.activateProfile(AbstractProfileService.jav
              a:306)
                      at org.jboss.system.server.profileservice.ProfileServiceBootstrap.start(ProfileServiceBootstrap.java:271)
                      at org.jboss.bootstrap.AbstractServerImpl.start(AbstractServerImpl.java:461)
                      at org.jboss.Main.boot(Main.java:221)
                      at org.jboss.Main$1.run(Main.java:556)
                      at java.lang.Thread.run(Thread.java:662)
              2011-11-17 10:59:25,309 ERROR [org.jboss.kernel.plugins.dependency.AbstractKernelController] (main) Error installing to Start:
              name=jboss.messaging:service=PostOffice state=Create mode=Manual requiredState=Installed
              java.lang.IllegalArgumentException: 7.33.45.147:7900 is not a member of the group [7.33.44.147:7900] (enabled_members=[])
                      at org.jgroups.protocols.UNICAST.down(UNICAST.java:332)
                      at org.jgroups.protocols.pbcast.STABLE.down(STABLE.java:316)
                      at org.jgroups.protocols.VIEW_SYNC.down(VIEW_SYNC.java:204)
                      at org.jgroups.protocols.pbcast.GMS.down(GMS.java:905)
                      at org.jgroups.stack.ProtocolStack.down(ProtocolStack.java:475)
                      at org.jgroups.JChannel.down(JChannel.java:1517)
                      at org.jgroups.JChannel.send(JChannel.java:626)
                      at org.jboss.messaging.core.impl.postoffice.GroupMember.unicastData(GroupMember.java:320)
                      at org.jboss.messaging.core.impl.postoffice.MessagingPostOffice.requestDeliveries(MessagingPostOffice.java:1716)
                      at org.jboss.messaging.core.impl.postoffice.MessagingPostOffice.loadBindings(MessagingPostOffice.java:2583)
                      at org.jboss.messaging.core.impl.postoffice.MessagingPostOffice.start(MessagingPostOffice.java:394)
                      at org.jboss.messaging.core.jmx.MessagingPostOfficeService.startService(MessagingPostOfficeService.java:462)
                      at org.jboss.system.ServiceMBeanSupport.jbossInternalStart(ServiceMBeanSupport.java:376)
                      at org.jboss.system.ServiceMBeanSupport.start(ServiceMBeanSupport.java:269)
                      at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
                      at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
                      at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
                      at java.lang.reflect.Method.invoke(Method.java:597)
                      at org.jboss.mx.interceptor.ReflectedDispatcher.invoke(ReflectedDispatcher.java:157)
                      at org.jboss.mx.server.Invocation.dispatch(Invocation.java:96)
                      at org.jboss.mx.interceptor.AbstractInterceptor.invoke(AbstractInterceptor.java:138)
                      at org.jboss.mx.server.Invocation.invoke(Invocation.java:90)
                      at org.jboss.mx.interceptor.ModelMBeanOperationInterceptor.invoke(ModelMBeanOperationInterceptor.java:140)
                      at org.jboss.mx.server.Invocation.invoke(Invocation.java:90)
                      at org.jboss.mx.server.AbstractMBeanInvoker.invoke(AbstractMBeanInvoker.java:264)
                      at org.jboss.mx.server.MBeanServerImpl.invoke(MBeanServerImpl.java:668)
                      at org.jboss.system.microcontainer.ServiceProxy.invoke(ServiceProxy.java:206)
                      at $Proxy38.start(Unknown Source)
                      at org.jboss.system.microcontainer.StartStopLifecycleAction.installAction(StartStopLifecycleAction.java:42)
                      at org.jboss.system.microcontainer.StartStopLifecycleAction.installAction(StartStopLifecycleAction.java:37)
                      at org.jboss.dependency.plugins.action.SimpleControllerContextAction.simpleInstallAction(SimpleControllerContextAction
              .java:62)
                      at org.jboss.dependency.plugins.action.AccessControllerContextAction.install(AccessControllerContextAction.java:71)
                      at org.jboss.dependency.plugins.AbstractControllerContextActions.install(AbstractControllerContextActions.java:51)
                      at org.jboss.dependency.plugins.AbstractControllerContext.install(AbstractControllerContext.java:348)
                      at org.jboss.system.microcontainer.ServiceControllerContext.install(ServiceControllerContext.java:286)
                      at org.jboss.dependency.plugins.AbstractController.install(AbstractController.java:1631)
                      at org.jboss.dependency.plugins.AbstractController.incrementState(AbstractController.java:934)
                      at org.jboss.dependency.plugins.AbstractController.resolveContexts(AbstractController.java:1082)
                      at org.jboss.dependency.plugins.AbstractController.resolveContexts(AbstractController.java:984)
                      at org.jboss.dependency.plugins.AbstractController.change(AbstractController.java:822)
                      at org.jboss.dependency.plugins.AbstractController.change(AbstractController.java:553)
                      at org.jboss.system.ServiceController.doChange(ServiceController.java:688)
                      at org.jboss.system.ServiceController.start(ServiceController.java:460)
                      at org.jboss.system.deployers.ServiceDeployer.start(ServiceDeployer.java:163)
                      at org.jboss.system.deployers.ServiceDeployer.deploy(ServiceDeployer.java:99)
                      at org.jboss.system.deployers.ServiceDeployer.deploy(ServiceDeployer.java:46)
                      at org.jboss.deployers.spi.deployer.helpers.AbstractSimpleRealDeployer.internalDeploy(AbstractSimpleRealDeployer.java:
              62)
                      at org.jboss.deployers.spi.deployer.helpers.AbstractRealDeployer.deploy(AbstractRealDeployer.java:50)
                      at org.jboss.deployers.plugins.deployers.DeployerWrapper.deploy(DeployerWrapper.java:171)
                      at org.jboss.deployers.plugins.deployers.DeployersImpl.doDeploy(DeployersImpl.java:1439)
                      at org.jboss.deployers.plugins.deployers.DeployersImpl.doInstallParentFirst(DeployersImpl.java:1157)
                      at org.jboss.deployers.plugins.deployers.DeployersImpl.doInstallParentFirst(DeployersImpl.java:1178)
                      at org.jboss.deployers.plugins.deployers.DeployersImpl.install(DeployersImpl.java:1098)
                      at org.jboss.dependency.plugins.AbstractControllerContext.install(AbstractControllerContext.java:348)
                      at org.jboss.dependency.plugins.AbstractController.install(AbstractController.java:1631)
                      at org.jboss.dependency.plugins.AbstractController.incrementState(AbstractController.java:934)
                      at org.jboss.dependency.plugins.AbstractController.resolveContexts(AbstractController.java:1082)
                      at org.jboss.dependency.plugins.AbstractController.resolveContexts(AbstractController.java:984)
                      at org.jboss.dependency.plugins.AbstractController.change(AbstractController.java:822)
                      at org.jboss.dependency.plugins.AbstractController.change(AbstractController.java:553)
                      at org.jboss.deployers.plugins.deployers.DeployersImpl.process(DeployersImpl.java:781)
                      at org.jboss.deployers.plugins.main.MainDeployerImpl.process(MainDeployerImpl.java:702)
                      at org.jboss.system.server.profileservice.repository.MainDeployerAdapter.process(MainDeployerAdapter.java:117)
                      at org.jboss.system.server.profileservice.repository.ProfileDeployAction.install(ProfileDeployAction.java:70)
                      at org.jboss.system.server.profileservice.repository.AbstractProfileAction.install(AbstractProfileAction.java:53)
                      at org.jboss.system.server.profileservice.repository.AbstractProfileService.install(AbstractProfileService.java:361)
                      at org.jboss.dependency.plugins.AbstractControllerContext.install(AbstractControllerContext.java:348)
                      at org.jboss.dependency.plugins.AbstractController.install(AbstractController.java:1631)
                      at org.jboss.dependency.plugins.AbstractController.incrementState(AbstractController.java:934)
                      at org.jboss.dependency.plugins.AbstractController.resolveContexts(AbstractController.java:1082)
                      at org.jboss.dependency.plugins.AbstractController.resolveContexts(AbstractController.java:984)
                      at org.jboss.dependency.plugins.AbstractController.change(AbstractController.java:822)
                      at org.jboss.dependency.plugins.AbstractController.change(AbstractController.java:553)
                      at org.jboss.system.server.profileservice.repository.AbstractProfileService.activateProfile(AbstractProfileService.jav
              a:306)
                      at org.jboss.system.server.profileservice.ProfileServiceBootstrap.start(ProfileServiceBootstrap.java:271)
                      at org.jboss.bootstrap.AbstractServerImpl.start(AbstractServerImpl.java:461)
                      at org.jboss.Main.boot(Main.java:221)
                      at org.jboss.Main$1.run(Main.java:556)
                      at java.lang.Thread.run(Thread.java:662)
              2011-11-17 10:59:25,326 INFO  [org.jboss.resource.connectionmanager.ConnectionFactoryBindingService] (main) Bound ConnectionMaager 'jboss.jca:service=ConnectionFactoryBinding,name=JmsXA' to JNDI name 'java:JmsXA'
              ...

              2011-11-17 10:59:25,522 INFO  [org.jboss.web.tomcat.service.deployers.TomcatDeployment] (main) deploy, ctxPath=/castylesr5.1.1
              2011-11-17 10:59:25,857 INFO  [org.jboss.ejb.deployers.EjbDeployer] (main) installing bean: ejb/iam_im_identityminder_ejb.jar#WorkFlowCallBackEJB,uid18109388
              ...

              ...

              2011-11-17 10:59:44,736 WARN  [org.jboss.resource.adapter.jms.inflow.JmsActivation] (WorkManager(2)-5) Failure in jms activation org.jboss.resource.adapter.jms.inflow.JmsActivationSpec@1c07397(ra=org.jboss.resource.adapter.jms.JmsResourceAdapter@89eb51
              destination=queue/iam/im/jms/queue/wpServAutoActQueue destinationType=javax.jms.Queue tx=true durable=false reconnect=10 provider=DefaultJMSProvider user=null maxMessages=1 minSession=1 maxSession=15 keepAlive=30000 useDLQ=true DLQHandler=org.jboss.resource.adapter.jms.inflow.dlq.GenericDLQHandler DLQJndiName=queue/DLQ DLQUser=null DLQMaxResent=10)
              javax.naming.NameNotFoundException: DLQ not bound
                      at org.jnp.server.NamingServer.getBinding(NamingServer.java:771)
                      at org.jnp.server.NamingServer.getBinding(NamingServer.java:779)
              :

               

               

              • 4. Re: second node won't start in clustering
                wdfink

                How do you start the servers?

                Do you use CLI options to set TCP or do you change the configuration.

                What kind of app is the "identitymanager"?

                 

                At the moment it is unclear to me why the msg system fail in case of deploy the application.

                Do you have jboss jar files included in your app?

                 

                Also please attach logs and stacktraces instead of pasting into the thread.

                • 5. Re: second node won't start in clustering
                  pxiaoott

                  Here is my start scripts

                  Node1:

                  /disk/apps/jboss-5.1.0.GA/bin/run.sh -c all -g IdmPartitionTRANS -Djboss.messaging.ServerPeerID=200 -b 7.33.45.147 -Djboss.messaging.groupname=IdmPartitionTRANS -Djboss.default.jgroups.stack=tcp -Djgroups.tcpping.initial_hosts=7.33.45.147[7600],7.33.44.147[7600]

                   

                  Node2:

                  /disk/apps/jboss-5.1.0.GA/bin/run.sh -c all -g IdmPartitionTRANS -Djboss.messaging.ServerPeerID=201 -b 7.33.44.147 -Djboss.messaging.groupname=IdmPartitionTRANS -Djboss.default.jgroups.stack=tcp -Djgroups.tcpping.initial_hosts=7.33.44.147[7600],7.33.45.147[7600]

                   

                  i changed oracle-persistence-service.xml

                        <attribute name="ControlChannelName">tcp</attribute>
                  And uncomment  the TCPPING settings below in section "stack name="tcp" and "tcp-sync" in jgroups-channelfactory-stacks.xml

                              <TCPPING timeout="3000"
                                       initial_hosts="${jgroups.tcpping.initial_hosts:7.33.45.147[7600],7.33.44.147[7601]}"
                                       port_range="1"
                                       num_initial_members="3"/>

                  I'm trying to upload the server.log.

                   

                  thanks

                  Ping Xiao

                  • 6. Re: second node won't start in clustering
                    amathewjboss1

                    This seems to be a Network issue.

                     

                    [org.jboss.messaging.core.impl.postoffice.GroupMember] (main) New Members : 2 ([7.33.45.147:7600, 7.33.44.147:7600])

                    Notice 7.33.45 & 7.33.44. Seems like both servers are in different network. There could be some firewall and/or UDP connection issues.