Found a BUG in EAR hot deployment
I believe I found a bug in JBoss hot undeploy algorithm. Here is my setup:
- Jboss 3.2.3 running on Linux
- a single .ear archive
- this ear includes: .war archive, .sar archive, and .jar archive:
.sar
- Jboss service with mbean name "my.domain:id=xyz"
.war
- a servlet
- WEB-INF/jboss-web.xml with my.domain:id=xyz
.jar
- a statless ejb
- META-INF/jboss.xml with <enterprise-beans>.. ... my.domain:id=xyz.. .. </enterprise-beans>
As you can see, I want both the servlet and the ejb to explicitly depend on the 'my.domain:id=xyz' service coming from the .sar archive.
One the ejb gets created, the following two services get started:
'jboss.j2ee:jndiName=my.domain:id=xyz,service=EJB' with the mbean class StatelessSessionContainer.java
'jboss.j2ee:module=my.domain:id=xyz,service=EjbModule' with the mbean class EjbModule.java
After hot -undeploy (touch on the .ear file) the service 'jboss.j2ee:jndiName=my.domain:id=xyz,service=EJB' gets destroyed, but its mbean is not unregistered from the mbean registry.
So, during hot deploy happening right after that the service 'jboss.j2ee:jndiName=my.domain:id=xyz,service=EJB' failes to get initialized, throwing an exception ("the service already exists")
This bug was so annoying that I went of to debug JBoss. Here is what I found (in chronological sequence):
- The MainDeployer.java starts undeploy process for the ear
- it iterates over the sorted list of sub deployments and calls stop and destroy on them
- the order of the sub deployments in this process is the following: war, jar, sar (determined by DeplymentSorter.java)
- when war sub deployment gets destroyed, it nukes its own class loader:
// Nuke my stuff, this includes the class loader
di.cleanup();
This is in MainDeployer.java line #557 in jboss 3.2.3 source
- the class loader generates jmx notification "destroy" for every class loaded from this ear (a single class loader for the entire ear)
- the service controller of the service 'my.domain:id=xyz' gets the message to destroy this service; it determines the dependencies of this service and calls destroy on them
- the the session 'jboss.j2ee:jndiName=my.domain:id=xyz,service=EJB' gets destroyed; it does it successfully, however its mbean is not unregistered; the state of the session becomes "DESTROYED"
- the second service related to the stateless session ejb - 'jboss.j2ee:module=my.domain:id=xyz,service=EjbModule' gets destroyed. Under normal conditions, it is this guy that needs to be destroyed first. He then unregisters the mbean of the StatelessSessionContainer.java service. However, because of the following code in EjbModule.java destroyService(), line #371, this does not happen:
Container con = (Container) iter.previous();
ObjectName jmxName = con.getJmxName();
int conState = con.getState();
boolean destroyContainer = conState == CREATED || conState == STOPPED || conState == FAILED;
The reason it doesn't happen is because the conState is DESTROYED.
So, most likely one (or more) of the following statements is true:
1. It's very questionable why the sub deployment of the main deployment is allowed to nuke its class loader. This should be done by the main deployment.
2. The effect of the .. in jboss.xml should be such that 'jboss.j2ee:module=my.domain:id=xyz,service=EjbModule' (the EjbModule.java) becomes the dependent service, and is always gets destroyed before jboss.j2ee:jndiName=my.domain:id=xyz,service=EJB' (This is least likely in my opinion)
3. connState == DESTROYED should be one of the several service states causing boolean destroyContainer to become true (This is most likely to be true)
Please let me know if there is another workaround for this problem.
Yuri
