Cleanup problem with Ejb3Registrar and proxy factories

[brian.stansberry]

This post is specifically about code in the proxy module, but I think it might point to a general issue with Ejb3McRegistrar. Cutting to the chase, it looks like Ejb3McRegistrar leaves no way to back out a failed bind() call.

I had a test with a misnamed deployment descriptor, which led to a failure in proxy factory registration:

2008-08-21 18:50:39,196 ERROR [org.jboss.kernel.plugins.dependency.AbstractKernelController] (RMI TCP Connection(18)-192.168.1.146) Error installing to Start: name=ProxyFactory/ClusteredStatefulRemote state=Create
java.lang.IllegalStateException: Partition Ejb3IsLocalTestPartition not found
 at org.jboss.ha.framework.server.HAPartitionLocator.getHAPartition(HAPartitionLocator.java:124)
 at org.jboss.ejb3.proxy.clustered.registry.ProxyClusteringRegistry.registerClusteredBean(ProxyClusteringRegistry.java:114)
 at org.jboss.ejb3.proxy.clustered.factory.session.stateful.StatefulSessionClusteredProxyFactory.start(StatefulSessionClusteredProxyFactory.java:111)
 at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
 .....
 at org.jboss.kernel.plugins.dependency.AbstractKernelController.install(AbstractKernelController.java:106)
 at org.jboss.ejb3.common.registrar.plugin.mc.Ejb3McRegistrar.install(Ejb3McRegistrar.java:309)
 at org.jboss.ejb3.common.registrar.plugin.mc.Ejb3McRegistrar.bind(Ejb3McRegistrar.java:163)
 at org.jboss.ejb3.proxy.jndiregistrar.JndiSessionRegistrarBase.registerProxyFactory(JndiSessionRegistrarBase.java:906)
 .....

That's fine; it was a bad deployment. But the problem is the undeploy didn't properly clean up:

2008-08-21 18:50:39,264 DEBUG [org.jboss.ejb3.stateful.StatefulContainer] (RMI TCP Connection(18)-192.168.1.146) Failed to cleanup after start() failure
org.jboss.ejb3.common.registrar.spi.NotBoundException: Requested value bound at name "ProxyFactory/ClusteredStatefulRemote" is not bound.
 at org.jboss.ejb3.common.registrar.plugin.mc.Ejb3McRegistrar.lookup(Ejb3McRegistrar.java:91)
 at org.jboss.ejb3.proxy.jndiregistrar.JndiSessionRegistrarBase.deregisterProxyFactory(JndiSessionRegistrarBase.java:934)
 at org.jboss.ejb3.proxy.jndiregistrar.JndiSessionRegistrarBase.unbindEjb(JndiSessionRegistrarBase.java:477)
 at org.jboss.ejb3.session.SessionSpecContainer.lockedStop(SessionSpecContainer.java:732)
 at org.jboss.ejb3.stateful.StatefulContainer.lockedStop(StatefulContainer.java:324)
 at org.jboss.ejb3.stateful.StatefulContainer.lockedStart(StatefulContainer.java:308)
 at org.jboss.ejb3.EJBContainer.start(EJBContainer.java:855)

The effect of this is the "ProxyFactory/ClusteredStatefulRemote" bean was left registered in the MC, but not in an installed state. When later tests would deploy the same jar, they would all fail with "java.lang.IllegalStateException: ProxyFactory/ClusteredStatelessRemote is already installed". AFAICT, the only way to clean this up is to reboot the server.

There's a couple levels of issues here:

1) JndiSessionRegistrarBase.deregisterProxyFactory is doing a Ejb3Registrar.lookup before doing an unbind. It does this so it can make some assertions. Problem is Ejb3McRegistrar implements lookup by calling KernelController.getInstalledContext(). That will return null in this case, because "ProxyFactory/ClusteredStatefulRemote" never made it to ControllerState.INSTALLED. As a result, lookup() throws an exception.

2) If Ejb3McRegistrar.lookup() somehow was changed to get past that problem and find the ControllerContext, I believe it would still fail, because it checks ControllerContext.getError() and throws an exception if it finds one.

3) Even if the JndiSessionRegistrarBase.deregisterProxyFactory's call to Ejb3Registrar.lookup() were eliminated and the method went directly to Ejb3Registrar.unbind(), it would still fail because the first thing Ejb3McRegistrar.unbind() does is call lookup(). :-)

Bottom line, it looks like Ejb3McRegistrar leaves no way to back out a failed bind() call.

[alrubinger]

OK, Ejb3Registrar is intended to be an MC Abstraction, so it looks like in hiding MC's notion of a ControllerState I've allowed an object to remain in limbo

https://jira.jboss.org/jira/browse/EJBTHREE-1472

S,
ALR

[alrubinger]

I've fixed the problem with the Ejb3Registrar, as verified by the following test:

/**
 * Tests that an object bound with state other than "INSTALLED" may still be
 * unbound from MC via the Ejb3Registrar
 *
 * EJBTHREE-1472
 */
 @Test
 public void testUnbindFromControllerStateOtherThanInstalled() throws Throwable
 {
 // Create a new key/value pair for binding into MC
 String name = "MyObject";
 Object value = new Object();

 // Construct BMDB, adding an unmet dependency
 BeanMetaDataBuilder bmdb = BeanMetaDataBuilder.createBuilder(name, value.getClass().getName());
 bmdb.addDependency("SomeDependencyThatDoesn'tExist");

 // Install into MC, though because of the unmet dependency will not reach "INSTALLED" state
 try
 {
 getBootstrap().getKernel().getController().install(bmdb.getBeanMetaData(), value);
 }
 catch (Throwable e)
 {
 throw new RuntimeException("Could not install at name \"" + name + "\" value " + value, e);
 }

 // Ensure that the install didn't completely succeed
 ControllerContext context = getBootstrap().getKernel().getController().getContext(name, null);
 TestCase.assertTrue("The test object should not be fully installed for this test", !context.getState().equals(
 ControllerState.INSTALLED));

 // Unbind
 Ejb3RegistrarLocator.locateRegistrar().unbind(name);

 // Check that we've unbound
 boolean isUnbound = false;
 try
 {
 Ejb3RegistrarLocator.locateRegistrar().lookup(name);
 }
 catch (NotBoundException nbe)
 {
 isUnbound = true;
 }
 TestCase.assertTrue("The test object should be unbound", isUnbound);

 }

I guess I'll leave the checks that discovered this error in place in EJB3 Proxy JndiSessionRegistrarBase.deregisterProxyFactory(); after all, they gave us a nice fail-fast, right? ;)

S,
ALR