JBoss 5 UDP Clustering issues
Hi,
I am trying to cluster two machines with JBoss5 and Udp. I startup first machine then second one with same udp group ip and port etc settings. It does join the cluster initially and i see this:
2010-04-28 10:52:50,734 INFO [org.jboss.ha.framework.interfaces.HAPartition.MyClusterPartition] Number of cluster members: 2
2010-04-28 10:52:50,734 INFO [org.jboss.ha.framework.interfaces.HAPartition.MyClusterPartition] Other members: 1
Then it tries to transfer the state and it fails with following exception:
2010-04-28 10:54:21,496 DEBUG [org.jgroups.protocols.FD_SOCK] pinger thread terminated
2010-04-28 10:54:21,501 DEBUG [org.jboss.ha.framework.server.ClusterPartition] Starting failed jboss:service=HAPartition,partition=MyClusterPartition
java.lang.IllegalStateException: Initial serviceState transfer failed: Channel.getState() returned false
at org.jboss.ha.framework.server.ClusterPartition.fetchState(ClusterPartition.java:667)
at org.jboss.ha.framework.server.ClusterPartition.startService(ClusterPartition.java:422)
at org.jboss.system.ServiceMBeanSupport.jbossInternalStart(ServiceMBeanSupport.java:376)
at org.jboss.system.ServiceMBeanSupport.pojoStart(ServiceMBeanSupport.java:216)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
at java.lang.reflect.Method.invoke(Unknown Source)
at org.jboss.reflect.plugins.introspection.ReflectionUtils.invoke(ReflectionUtils.java:59)
at org.jboss.reflect.plugins.introspection.ReflectMethodInfoImpl.invoke(ReflectMethodInfoImpl.java:150)
at org.jboss.joinpoint.plugins.BasicMethodJoinPoint.dispatch(BasicMethodJoinPoint.java:66)
at org.jboss.kernel.plugins.dependency.KernelControllerContextAction$JoinpointDispatchWrapper.execute(KernelControllerContextAction.java:241)
at org.jboss.kernel.plugins.dependency.ExecutionWrapper.execute(ExecutionWrapper.java:47)
at org.jboss.kernel.plugins.dependency.KernelControllerContextAction.dispatchExecutionWrapper(KernelControllerContextAction.java:109)
at org.jboss.kernel.plugins.dependency.KernelControllerContextAction.dispatchJoinPoint(KernelControllerContextAction.java:70)
at org.jboss.kernel.plugins.dependency.LifecycleAction.installActionInternal(LifecycleAction.java:221)
at org.jboss.kernel.plugins.dependency.InstallsAwareAction.installAction(InstallsAwareAction.java:54)
at org.jboss.kernel.plugins.dependency.InstallsAwareAction.installAction(InstallsAwareAction.java:42)
at org.jboss.dependency.plugins.action.SimpleControllerContextAction.simpleInstallAction(SimpleControllerContextAction.java:62)
at org.jboss.dependency.plugins.action.AccessControllerContextAction.install(AccessControllerContextAction.java:71)
at org.jboss.dependency.plugins.AbstractControllerContextActions.install(AbstractControllerContextActions.java:51)
at org.jboss.dependency.plugins.AbstractControllerContext.install(AbstractControllerContext.java:348)
at org.jboss.dependency.plugins.AbstractController.install(AbstractController.java:1631)
at org.jboss.dependency.plugins.AbstractController.incrementState(AbstractController.java:934)
at org.jboss.dependency.plugins.AbstractController.resolveContexts(AbstractController.java:1082)
at org.jboss.dependency.plugins.AbstractController.resolveContexts(AbstractController.java:984)
at org.jboss.dependency.plugins.AbstractController.install(AbstractController.java:774)
at org.jboss.dependency.plugins.AbstractController.install(AbstractController.java:540)
at org.jboss.deployers.vfs.deployer.kernel.BeanMetaDataDeployer.deploy(BeanMetaDataDeployer.java:121)
at org.jboss.deployers.vfs.deployer.kernel.BeanMetaDataDeployer.deploy(BeanMetaDataDeployer.java:51)
at org.jboss.deployers.spi.deployer.helpers.AbstractSimpleRealDeployer.internalDeploy(AbstractSimpleRealDeployer.java:62)
at org.jboss.deployers.spi.deployer.helpers.AbstractRealDeployer.deploy(AbstractRealDeployer.java:50)
at org.jboss.deployers.plugins.deployers.DeployerWrapper.deploy(DeployerWrapper.java:171)
at org.jboss.deployers.plugins.deployers.DeployersImpl.doDeploy(DeployersImpl.java:1439)
at org.jboss.deployers.plugins.deployers.DeployersImpl.doInstallParentFirst(DeployersImpl.java:1157)
at org.jboss.deployers.plugins.deployers.DeployersImpl.doInstallParentFirst(DeployersImpl.java:1178)
at org.jboss.deployers.plugins.deployers.DeployersImpl.install(DeployersImpl.java:1098)
at org.jboss.dependency.plugins.AbstractControllerContext.install(AbstractControllerContext.java:348)
at org.jboss.dependency.plugins.AbstractController.install(AbstractController.java:1631)
at org.jboss.dependency.plugins.AbstractController.incrementState(AbstractController.java:934)
at org.jboss.dependency.plugins.AbstractController.resolveContexts(AbstractController.java:1082)
at org.jboss.dependency.plugins.AbstractController.resolveContexts(AbstractController.java:984)
at org.jboss.dependency.plugins.AbstractController.change(AbstractController.java:822)
at org.jboss.dependency.plugins.AbstractController.change(AbstractController.java:553)
at org.jboss.deployers.plugins.deployers.DeployersImpl.process(DeployersImpl.java:781)
at org.jboss.deployers.plugins.main.MainDeployerImpl.process(MainDeployerImpl.java:702)
at org.jboss.system.server.profileservice.repository.MainDeployerAdapter.process(MainDeployerAdapter.java:117)
at org.jboss.system.server.profileservice.repository.ProfileDeployAction.install(ProfileDeployAction.java:70)
at org.jboss.system.server.profileservice.repository.AbstractProfileAction.install(AbstractProfileAction.java:53)
at org.jboss.system.server.profileservice.repository.AbstractProfileService.install(AbstractProfileService.java:361)
at org.jboss.dependency.plugins.AbstractControllerContext.install(AbstractControllerContext.java:348)
at org.jboss.dependency.plugins.AbstractController.install(AbstractController.java:1631)
at org.jboss.dependency.plugins.AbstractController.incrementState(AbstractController.java:934)
at org.jboss.dependency.plugins.AbstractController.resolveContexts(AbstractController.java:1082)
at org.jboss.dependency.plugins.AbstractController.resolveContexts(AbstractController.java:984)
at org.jboss.dependency.plugins.AbstractController.change(AbstractController.java:822)
at org.jboss.dependency.plugins.AbstractController.change(AbstractController.java:553)
at org.jboss.system.server.profileservice.repository.AbstractProfileService.activateProfile(AbstractProfileService.java:306)
at org.jboss.system.server.profileservice.ProfileServiceBootstrap.start(ProfileServiceBootstrap.java:271)
at org.jboss.bootstrap.AbstractServerImpl.start(AbstractServerImpl.java:461)
at org.jboss.Main.boot(Main.java:221)
at org.jboss.Main$1.run(Main.java:556)
at java.lang.Thread.run(Unknown Source)
2010-04-28 10:54:21,507 DEBUG [org.jboss.ha.framework.server.ClusterPartition] Destroying jboss:service=HAPartition,partition=MyClusterPartition
2010-04-28 10:54:21,507 DEBUG [org.jboss.ha.framework.interfaces.HAPartition.MyClusterPartition] Destroying HAPartition: MyClusterPartition
2010-04-28 10:54:21,507 INFO [org.jboss.ha.framework.interfaces.HAPartition.MyClusterPartition] Partition MyClusterPartition destroyed.
And I see the other server reporting a membership change event with 1 dead member . Don't know why it is happening I've tried to increase the timeout to 90000 ms but in vain. Can anyone please let me know where does jboss store this state? And what does it actually do to transfer the state? It looks like the udp address and port are all allowed in our network and It joins the cluster initially but fails at state transfer? We are also running a separate cluster of jboss4 using UDP and it is working fine. This cluster was also working fine till last week and we started getting this error when trying to join machines in cluster.
Any help will be highly appreciated.
Thank you.
