messaging is preventing shutdown of server

[starksm64]

After running the org.jboss.test.mdb.test.MDBUnitTestCase a few times, I tried to shutdown the server and it ended up hanging:

Thread: JBoss Shutdown Hook : priority:5, demon:true, threadId:11, threadState:WAITING, lockName:EDU.oswego.cs.dl.util.concurrent.WriterPreferenceReadWriteLock$WriterLock@2495223b

 java.lang.Object.wait(Native Method)
 java.lang.Object.wait(Object.java:474)
 EDU.oswego.cs.dl.util.concurrent.WriterPreferenceReadWriteLock$WriterLock.acquire(WriterPreferenceReadWriteLock.java:240)
 org.jboss.messaging.core.plugin.postoffice.DefaultPostOffice.unbindQueue(DefaultPostOffice.java:232)
 org.jboss.jms.server.destination.TopicService.stopService(TopicService.java:157)
 org.jboss.system.ServiceMBeanSupport.jbossInternalStop(ServiceMBeanSupport.java:315)
 org.jboss.system.ServiceMBeanSupport.stop(ServiceMBeanSupport.java:206)
 sun.reflect.GeneratedMethodAccessor122.invoke(Unknown Source)
 sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
 java.lang.reflect.Method.invoke(Method.java:585)
 org.jboss.mx.interceptor.ReflectedDispatcher.invoke(ReflectedDispatcher.java:157)
 org.jboss.mx.server.Invocation.dispatch(Invocation.java:96)
 org.jboss.mx.interceptor.AbstractInterceptor.invoke(AbstractInterceptor.java:138)
 org.jboss.mx.server.Invocation.invoke(Invocation.java:90)
 org.jboss.mx.interceptor.ModelMBeanOperationInterceptor.invoke(ModelMBeanOperationInterceptor.java:140)
 org.jboss.mx.server.Invocation.invoke(Invocation.java:90)
 org.jboss.mx.server.AbstractMBeanInvoker.invoke(AbstractMBeanInvoker.java:264)
 org.jboss.mx.server.MBeanServerImpl.invoke(MBeanServerImpl.java:668)
 org.jboss.system.microcontainer.ServiceProxy.invoke(ServiceProxy.java:184)
 $Proxy0.stop(Unknown Source)
 org.jboss.system.microcontainer.StartStopLifecycleAction.uninstallAction(StartStopLifecycleAction.java:56)
 org.jboss.system.microcontainer.ServiceControllerContextAction.uninstall(ServiceControllerContextAction.java:90)
 org.jboss.dependency.plugins.AbstractControllerContextActions.uninstall(AbstractControllerContextActions.java:58)
 org.jboss.dependency.plugins.AbstractControllerContext.uninstall(AbstractControllerContext.java:240)
 org.jboss.dependency.plugins.AbstractController.uninstall(AbstractController.java:738)
 org.jboss.dependency.plugins.AbstractController.uninstallContext(AbstractController.java:700)
 org.jboss.dependency.plugins.AbstractController.uninstallContext(AbstractController.java:630)
 org.jboss.dependency.plugins.AbstractController.uninstallContext(AbstractController.java:678)
 org.jboss.dependency.plugins.AbstractController.uninstallContext(AbstractController.java:630)
 org.jboss.dependency.plugins.AbstractController.uninstallContext(AbstractController.java:678)
 org.jboss.dependency.plugins.AbstractController.uninstallContext(AbstractController.java:630)
 org.jboss.dependency.plugins.AbstractController.uninstallContext(AbstractController.java:678)
 org.jboss.dependency.plugins.AbstractController.uninstallContext(AbstractController.java:630)
 org.jboss.dependency.plugins.AbstractController.uninstallContext(AbstractController.java:678)
 org.jboss.dependency.plugins.AbstractController.uninstallContext(AbstractController.java:630)
 org.jboss.dependency.plugins.AbstractController.uninstallContext(AbstractController.java:678)
 org.jboss.dependency.plugins.AbstractController.change(AbstractController.java:340)
 org.jboss.dependency.plugins.AbstractController.change(AbstractController.java:205)
 org.jboss.system.ServiceController.doChange(ServiceController.java:656)
 org.jboss.system.ServiceController.stop(ServiceController.java:481)
 org.jboss.system.deployers.ServiceDeployer.stop(ServiceDeployer.java:149)
 org.jboss.system.deployers.ServiceDeployer.undeploy(ServiceDeployer.java:129)
 org.jboss.system.deployers.ServiceDeployer.undeploy(ServiceDeployer.java:46)
 org.jboss.deployers.plugins.deployers.helpers.AbstractSimpleRealDeployer.undeploy(AbstractSimpleRealDeployer.java:64)
 org.jboss.deployers.plugins.deployer.AbstractSimpleDeployer.prepareUndeploy(AbstractSimpleDeployer.java:42)
 org.jboss.deployers.plugins.deployer.DeployerWrapper.prepareUndeploy(DeployerWrapper.java:130)
 org.jboss.deployers.plugins.deployment.MainDeployerImpl.prepareUndeploy(MainDeployerImpl.java:517)
 org.jboss.deployers.plugins.deployment.MainDeployerImpl.prepareUndeploy(MainDeployerImpl.java:526)
 org.jboss.deployers.plugins.deployment.MainDeployerImpl.process(MainDeployerImpl.java:448)
 org.jboss.deployers.plugins.deployment.MainDeployerImpl.process(MainDeployerImpl.java:406)
 org.jboss.deployers.plugins.deployment.MainDeployerImpl.shutdown(MainDeployerImpl.java:634)
 AOPContainerProxy$0.shutdown(AOPContainerProxy$0.java)
 org.jboss.system.server.profileservice.ServerImpl$ShutdownHook.shutdownDeployments(ServerImpl.java:767)
 org.jboss.system.server.profileservice.ServerImpl$ShutdownHook.shutdown(ServerImpl.java:749)
 org.jboss.system.server.profileservice.ServerImpl$ShutdownHook.run(ServerImpl.java:716)

How is the unbind getting hung up?

[timfox]

Scott-

What version of JBM are you running? The line numbers don't correspond to 1.2.GA code.

Also, do you have the full kill -3 stack dump, showing the other thread that has the lock.

[ovidiu.feodorov]

The latest version I tested the head with (and not all of integration tests were passing yet), was 1.2.0.CR1.

Let me upgrade to 1.2.0.GA and take it from there.

[starksm64]

I thought we had updated to the GA but its still only 1.2.0.CR1. I created JBAS-4185 as a broad issue for validating the jms tests and 1.2.0.GA integration into trunk and assigned it to Ovidiu for now.

[ovidiu.feodorov]

OK

This is our corresponding issue http://jira.jboss.org/jira/browse/JBMESSAGING-553

I've linked them.

[timfox]

There were a couple of deadlock issues at shutdown that I fixed between CR1 and GA.

I suspect it is one of these. If the problem doesn't occur after upgrading to 1.2 then we should close the issue.