JBAS-4717 - Problem with @JMX for clustering
adrian.brock Sep 13, 2007 10:01 AMCurrently clustering's use of @JMX is causing spurious errors at shutdown:
e.g.
15:32:50,651 WARN [AbstractLifecycleCallbackItem] Ignored error uninstalling context JChannelFactoryJMXWrapper; callback=JMXAdvice javax.management.InstanceNotFoundException: jboss.jgroups:service=MultiplexerChannelFactory is not registered. at org.jboss.mx.server.registry.BasicMBeanRegistry.get(BasicMBeanRegistry.java:529) at org.jboss.mx.server.MBeanServerImpl.unregisterMBean(MBeanServerImpl.java:383) at org.jboss.aop.microcontainer.aspects.jmx.JMXLifecycleCallback.uninstall(JMXLifecycleCallback.java:73) at sun.reflect.GeneratedMethodAccessor79.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:585) at org.jboss.reflect.plugins.introspection.ReflectionUtils.invoke(ReflectionUtils.java:55) at org.jboss.reflect.plugins.introspection.ReflectMethodInfoImpl.invoke(ReflectMethodInfoImpl.java:108) at org.jboss.joinpoint.plugins.BasicMethodJoinPoint.dispatch(BasicMethodJoinPoint.java:66) at org.jboss.beans.info.plugins.AbstractBeanInfo.invoke(AbstractBeanInfo.java:255) at org.jboss.kernel.plugins.dependency.AbstractKernelControllerContext.invoke(AbstractKernelControllerContext.java:234) at org.jboss.dependency.plugins.AbstractLifecycleCallbackItem.uninstall(AbstractLifecycleCallbackItem.java:107) at org.jboss.dependency.plugins.AbstractController.handleLifecycleCallbacks(AbstractController.java:1245) at org.jboss.dependency.plugins.AbstractController.handleUninstallLifecycleCallbacks(AbstractController.java:1231) at org.jboss.dependency.plugins.AbstractController.uninstallContext(AbstractController.java:1007) at org.jboss.dependency.plugins.AbstractController.uninstallContext(AbstractController.java:936) at org.jboss.dependency.plugins.AbstractController.uninstallContext(AbstractController.java:986) at org.jboss.dependency.plugins.AbstractController.uninstallContext(AbstractController.java:936) at org.jboss.dependency.plugins.AbstractController.uninstall(AbstractController.java:463) at org.jboss.dependency.plugins.AbstractController.uninstall(AbstractController.java:426) at org.jboss.deployers.vfs.deployer.kernel.BeanMetaDataDeployer.undeploy(BeanMetaDataDeployer.java:79) at org.jboss.deployers.vfs.deployer.kernel.BeanMetaDataDeployer.undeploy(BeanMetaDataDeployer.java:42) at org.jboss.deployers.spi.deployer.helpers.AbstractSimpleRealDeployer.undeploy(AbstractSimpleRealDeployer.java:73) at org.jboss.deployers.plugins.deployers.DeployerWrapper.undeploy(DeployerWrapper.java:187) at org.jboss.deployers.plugins.deployers.DeployersImpl.doUninstallParentLast(DeployersImpl.java:947) at org.jboss.deployers.plugins.deployers.DeployersImpl.doUninstallParentLast(DeployersImpl.java:940) at org.jboss.deployers.plugins.deployers.DeployersImpl.uninstall(DeployersImpl.java:902) at org.jboss.dependency.plugins.AbstractControllerContext.uninstall(AbstractControllerContext.java:333) at org.jboss.dependency.plugins.AbstractController.uninstall(AbstractController.java:1323) at org.jboss.dependency.plugins.AbstractController.uninstallContext(AbstractController.java:1009) at org.jboss.dependency.plugins.AbstractController.change(AbstractController.java:627) at org.jboss.dependency.plugins.AbstractController.change(AbstractController.java:411) at org.jboss.deployers.plugins.deployers.DeployersImpl.process(DeployersImpl.java:420) at org.jboss.deployers.plugins.main.MainDeployerImpl.process(MainDeployerImpl.java:354) at org.jboss.deployers.plugins.main.MainDeployerImpl.shutdown(MainDeployerImpl.java:382) at org.jboss.system.server.profileservice.ProfileServiceBootstrap.shutdown(ProfileServiceBootstrap.java:151) at org.jboss.bootstrap.AbstractServerImpl.shutdownServer(AbstractServerImpl.java:482) at org.jboss.bootstrap.AbstractServerImpl$ShutdownHook.run(AbstractServerImpl.java:778)
This is because the MBeans implement ServiceMBeanSupport
which means they will be lazily registered with the ServiceController
and thus unregistered twice (once by the JMXLifecycle callback
and once when the ServiceController shuts down).
What I want to do is change the lifecycle to use the ServiceController
instead of the MBeanServer directly.
That way it will be removed from the ServiceController only (which
also removes it from the MBeanServer) unlike now where there is
no unregisteration from the ServiceController unless you shutdown the server.
This has another advantage that you can have proper dependencies on the
JMX name as well which will give some backwards compatibilty.
But the dual lifecycle (bean and jmx) is causing start() to be invoked twice.
15:26:45,657 ERROR [AbstractKernelController] Error installing to Create: name=HAPartition state=Configured java.lang.RuntimeException: Failure creating multiplexed Channel at org.jboss.ha.framework.server.ClusterPartition.createMuxChannel(ClusterPartition.java:225) at org.jboss.ha.framework.server.ClusterPartition.createService(ClusterPartition.java:253) at org.jboss.system.ServiceMBeanSupport.jbossInternalCreate(ServiceMBeanSupport.java:260) at org.jboss.system.ServiceMBeanSupport.create(ServiceMBeanSupport.java:188) at AOPContainerProxy$18.create(AOPContainerProxy$18.java) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:585) at org.jboss.reflect.plugins.introspection.ReflectionUtils.invoke(ReflectionUtils.java:55) at org.jboss.reflect.plugins.introspection.ReflectMethodInfoImpl.invoke(ReflectMethodInfoImpl.java:108) at org.jboss.joinpoint.plugins.BasicMethodJoinPoint.dispatch(BasicMethodJoinPoint.java:66) at org.jboss.kernel.plugins.dependency.KernelControllerContextAction.dispatchJoinPoint(KernelControllerContextAction.java:103) at org.jboss.kernel.plugins.dependency.LifecycleAction.installActionInternal(LifecycleAction.java:221) at org.jboss.kernel.plugins.dependency.KernelControllerContextAction.installAction(KernelControllerContextAction.java:143) at org.jboss.kernel.plugins.dependency.KernelControllerContextAction.installAction(KernelControllerContextAction.java:53) at org.jboss.dependency.plugins.action.SimpleControllerContextAction.simpleInstallAction(SimpleControllerContextAction.java:62) at org.jboss.dependency.plugins.action.AccessControllerContextAction.install(AccessControllerContextAction.java:71) at org.jboss.dependency.plugins.AbstractControllerContextActions.install(AbstractControllerContextActions.java:51) at org.jboss.dependency.plugins.AbstractControllerContext.install(AbstractControllerContext.java:327) at org.jboss.dependency.plugins.AbstractController.install(AbstractController.java:1309) at org.jboss.dependency.plugins.AbstractController.incrementState(AbstractController.java:734) at org.jboss.dependency.plugins.AbstractController.resolveContexts(AbstractController.java:862) at org.jboss.dependency.plugins.AbstractController.resolveContexts(AbstractController.java:784) at org.jboss.dependency.plugins.AbstractController.install(AbstractController.java:574) at org.jboss.dependency.plugins.AbstractController.install(AbstractController.java:398) at org.jboss.system.ServiceController.doInstall(ServiceController.java:641) at org.jboss.system.ServiceController.register(ServiceController.java:344) at org.jboss.system.ServiceController.start(ServiceController.java:415) at org.jboss.system.microcontainer.jmx.ServiceControllerLifecycleCallback.install(ServiceControllerLifecycleCallback.java:93) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:585) at org.jboss.reflect.plugins.introspection.ReflectionUtils.invoke(ReflectionUtils.java:55) at org.jboss.reflect.plugins.introspection.ReflectMethodInfoImpl.invoke(ReflectMethodInfoImpl.java:108) at org.jboss.joinpoint.plugins.BasicMethodJoinPoint.dispatch(BasicMethodJoinPoint.java:66) at org.jboss.beans.info.plugins.AbstractBeanInfo.invoke(AbstractBeanInfo.java:255) at org.jboss.kernel.plugins.dependency.AbstractKernelControllerContext.invoke(AbstractKernelControllerContext.java:234) at org.jboss.dependency.plugins.AbstractLifecycleCallbackItem.install(AbstractLifecycleCallbackItem.java:82) at org.jboss.dependency.plugins.AbstractController.handleLifecycleCallbacks(AbstractController.java:1243) at org.jboss.dependency.plugins.AbstractController.handleInstallLifecycleCallbacks(AbstractController.java:1224) at org.jboss.dependency.plugins.AbstractController.incrementState(AbstractController.java:743) at org.jboss.dependency.plugins.AbstractController.resolveContexts(AbstractController.java:862) at org.jboss.dependency.plugins.AbstractController.resolveContexts(AbstractController.java:784) at org.jboss.dependency.plugins.AbstractController.install(AbstractController.java:574) at org.jboss.dependency.plugins.AbstractController.install(AbstractController.java:398) at org.jboss.system.ServiceController.doInstall(ServiceController.java:641) at org.jboss.system.ServiceController.register(ServiceController.java:344) at org.jboss.system.ServiceController.start(ServiceController.java:415) at org.jboss.system.microcontainer.jmx.ServiceControllerLifecycleCallback.install(ServiceControllerLifecycleCallback.java:93) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:585) at org.jboss.reflect.plugins.introspection.ReflectionUtils.invoke(ReflectionUtils.java:55) at org.jboss.reflect.plugins.introspection.ReflectMethodInfoImpl.invoke(ReflectMethodInfoImpl.java:108) at org.jboss.joinpoint.plugins.BasicMethodJoinPoint.dispatch(BasicMethodJoinPoint.java:66) at org.jboss.beans.info.plugins.AbstractBeanInfo.invoke(AbstractBeanInfo.java:255) at org.jboss.kernel.plugins.dependency.AbstractKernelControllerContext.invoke(AbstractKernelControllerContext.java:234) at org.jboss.dependency.plugins.AbstractLifecycleCallbackItem.install(AbstractLifecycleCallbackItem.java:82) at org.jboss.dependency.plugins.AbstractController.handleLifecycleCallbacks(AbstractController.java:1243) at org.jboss.dependency.plugins.AbstractController.handleInstallLifecycleCallbacks(AbstractController.java:1224) at org.jboss.dependency.plugins.AbstractController.incrementState(AbstractController.java:743) at org.jboss.dependency.plugins.AbstractController.resolveContexts(AbstractController.java:862) at org.jboss.dependency.plugins.AbstractController.resolveContexts(AbstractController.java:784) at org.jboss.dependency.plugins.AbstractController.install(AbstractController.java:574) at org.jboss.dependency.plugins.AbstractController.install(AbstractController.java:398) at org.jboss.system.ServiceController.doInstall(ServiceController.java:641) at org.jboss.system.ServiceController.register(ServiceController.java:344) at org.jboss.system.ServiceController.start(ServiceController.java:415) at org.jboss.system.microcontainer.jmx.ServiceControllerLifecycleCallback.install(ServiceControllerLifecycleCallback.java:93) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:585) at org.jboss.reflect.plugins.introspection.ReflectionUtils.invoke(ReflectionUtils.java:55) at org.jboss.reflect.plugins.introspection.ReflectMethodInfoImpl.invoke(ReflectMethodInfoImpl.java:108) at org.jboss.joinpoint.plugins.BasicMethodJoinPoint.dispatch(BasicMethodJoinPoint.java:66) at org.jboss.beans.info.plugins.AbstractBeanInfo.invoke(AbstractBeanInfo.java:255) at org.jboss.kernel.plugins.dependency.AbstractKernelControllerContext.invoke(AbstractKernelControllerContext.java:234) at org.jboss.dependency.plugins.AbstractLifecycleCallbackItem.install(AbstractLifecycleCallbackItem.java:82) at org.jboss.dependency.plugins.AbstractController.handleLifecycleCallbacks(AbstractController.java:1243) at org.jboss.dependency.plugins.AbstractController.handleInstallLifecycleCallbacks(AbstractController.java:1224) at org.jboss.dependency.plugins.AbstractController.incrementState(AbstractController.java:743) at org.jboss.dependency.plugins.AbstractController.resolveContexts(AbstractController.java:862) at org.jboss.dependency.plugins.AbstractController.resolveContexts(AbstractController.java:784) at org.jboss.dependency.plugins.AbstractController.install(AbstractController.java:574) at org.jboss.dependency.plugins.AbstractController.install(AbstractController.java:398) at org.jboss.deployers.vfs.deployer.kernel.BeanMetaDataDeployer.deploy(BeanMetaDataDeployer.java:68) at org.jboss.deployers.vfs.deployer.kernel.BeanMetaDataDeployer.deploy(BeanMetaDataDeployer.java:42) at org.jboss.deployers.spi.deployer.helpers.AbstractSimpleRealDeployer.deploy(AbstractSimpleRealDeployer.java:65) at org.jboss.deployers.plugins.deployers.DeployerWrapper.deploy(DeployerWrapper.java:169) at org.jboss.deployers.plugins.deployers.DeployersImpl.doInstallParentFirst(DeployersImpl.java:728) at org.jboss.deployers.plugins.deployers.DeployersImpl.doInstallParentFirst(DeployersImpl.java:749) at org.jboss.deployers.plugins.deployers.DeployersImpl.install(DeployersImpl.java:669) at org.jboss.dependency.plugins.AbstractControllerContext.install(AbstractControllerContext.java:327) at org.jboss.dependency.plugins.AbstractController.install(AbstractController.java:1309) at org.jboss.dependency.plugins.AbstractController.incrementState(AbstractController.java:734) at org.jboss.dependency.plugins.AbstractController.resolveContexts(AbstractController.java:862) at org.jboss.dependency.plugins.AbstractController.resolveContexts(AbstractController.java:784) at org.jboss.dependency.plugins.AbstractController.change(AbstractController.java:622) at org.jboss.dependency.plugins.AbstractController.change(AbstractController.java:411) at org.jboss.deployers.plugins.deployers.DeployersImpl.process(DeployersImpl.java:495) at org.jboss.deployers.plugins.main.MainDeployerImpl.process(MainDeployerImpl.java:354) at org.jboss.system.server.profileservice.ProfileServiceBootstrap.loadProfile(ProfileServiceBootstrap.java:245) at org.jboss.system.server.profileservice.ProfileServiceBootstrap.start(ProfileServiceBootstrap.java:131) at org.jboss.bootstrap.AbstractServerImpl.start(AbstractServerImpl.java:382) at org.jboss.Main.boot(Main.java:210) at org.jboss.Main$1.run(Main.java:529) at java.lang.Thread.run(Thread.java:595) Caused by: java.lang.Exception: service ID "DefaultPartition" is already registered, cannot register duplicate ID at org.jgroups.mux.Multiplexer.createMuxChannel(Multiplexer.java:403) at org.jgroups.JChannelFactory.createMultiplexerChannel(JChannelFactory.java:297) at org.jboss.ha.framework.server.JChannelFactory.createMultiplexerChannel(JChannelFactory.java:68) at org.jboss.ha.framework.server.JChannelFactory.createMultiplexerChannel(JChannelFactory.java:82) at org.jboss.ha.framework.server.ClusterPartition.createMuxChannel(ClusterPartition.java:217) ... 110 more
What would fix the problem would to change the name from the none automatic name
"start()" to one that only the MC recognises such that the method is invoked only for
the bean view.
e.g.
@org.jboss.beans.metadata.plugins.annotations.Start public void startPartition()
and the same for stop.
You can try the new ServiceController based version if you change the config
in deployers/jboss-aop-jboss5.deployer/META-INF/jboss-aspect-library-beans.xml
swap this config
<!-- Add the JMX --> <!--aop:lifecycle-configure xmlns:aop="urn:jboss:aop-beans:1.0" name="JMXAdvice" class="org.jboss.system.microcontainer.jmx.ServiceControllerLifecycleCallback" classes="@org.jboss.aop.microcontainer.aspects.jmx.JMX" manager-bean="AspectManager" manager-property="aspectManager"> <property name="serviceController"><inject bean="JMXKernel" property="serviceController"/></property> </aop:lifecycle-configure--> <aop:lifecycle-configure xmlns:aop="urn:jboss:aop-beans:1.0" name="JMXAdvice" class="org.jboss.aop.microcontainer.aspects.jmx.JMXLifecycleCallback" classes="@org.jboss.aop.microcontainer.aspects.jmx.JMX" manager-bean="AspectManager" manager-property="aspectManager"> <property name="mbeanServer"><inject bean="JMXKernel" property="mbeanServer"/></property> </aop:lifecycle-configure>