JBAS-5578 ServiceMBeanSupport as MC bean

[brian.stansberry]

Discussion of http://jira.jboss.com/jira/browse/JBAS-5578

Kind of an odd issue as it relates to doing something strange -- deploying a ServiceMBeanSupport subclass via a beans.xml, and then annotating it as @JMX. The create/start/stop/destroyService methods don't get invoked.

This comes up because I tend to try to deploy things as MC beans, but for backwards compatibility reasons things like HASingletonController are still ServiceMBeanSupport subclasses. So I hit weird issues. I can work around them, so not critical. But want to bring it up in case it suggests some more fundamental problem.

Anyway, here's what I think is going on:

I think this has to do with ServiceControllerLifecycleCallback. The @JMX annotation means this class handles starting the service. Leads to this:

ServiceControllerLifecycleCallback.install --> ServiceController.start(ObjectName) --> ServiceController.doChange(controller, context, ControllerState.INSTALLED, "start")

does some stuff... and then calls into ServiceMBeanSupport.start() which has this:

 if (serviceName != null && isJBossInternalLifecycleExposed)
 server.invoke(ServiceController.OBJECT_NAME, "start", new Object[] { serviceName }, SERVICE_CONTROLLER_SIG);
 else
 jbossInternalStart();

The first branch of the if test is taken and that calls ServiceController.start(ObjectName) again (i.e. a recursive call). That call hits this and returns:

 // If we are already started (can happen in dependencies) just return
 if (ctx.state == ServiceContext.RUNNING || ctx.state == ServiceContext.FAILED)
 {
 log.debug("Ignoring start request for service: " + ctx.objectName + " at state " + ctx.getStateString());
 return;
 }

Effect of all this is the startService() method never gets invoked.


One possible solution to this is to add a way to configure the bean (a setter) such that isJBossInternalLifecycleExposed returns false. Avoid the recursive call.

[adrian.brock]

I'm not seeing this.

Try (which currently passes)

./build.sh one-test -Dtest=org.jboss.test.jmx.test.AnnotatedJMXUnitTestCase

and modify it to show what you are seeing.

NOTE. For a POJO, the ServiceController invocations don't invoke
create/start/stop/destroy anymore. That is done via the POJO invocations
see http://jira.jboss.com/jira/browse/JBAS-5355
Otherwise you would get duplicate invocations.

[adrian.brock]

Never mind, I think I figured out what the problem is.

The issue occurs when you register a ServiceMBeanSupport MBean using registerDirect=true on the JMX annotation.

The callstack then goes something like:
MC - > ServiceMBeanSupport.create() -> ServiceController.create()
ServiceController.create() then ignores the request because it thinks it is a call
on the JMX lifecycle, i.e. a duplicate

I fixed this problem by introducing POJO lifecycle events in ServiceMBeanSupport
so we can differentiate the two.

 @Create
 public void pojoCreate() throws Exception
 {
 jbossInternalCreate();
 }

This brings up two other issues. The first of which I fixed which is related
to the confusion when JMX and POJO lifecycles.

If you invoke a lifecycle operation on the JMX console (or any MBeanServer operation)
then it should be redirected to the MC for a POJO not the ServiceController.
The JMX part is just a wrapper.
Since we now have the seperate pojoXXX methods, this is easy to spot,
so ServiceMBeanSupport now does the redirection, e.g.

public class ServiceMBeanSupport
 extends JBossNotificationBroadcasterSupport
 implements ServiceMBean, MBeanRegistration,

// Implement this interface so we know the true name not the JMX wrapper
// when we are registered as a POJO
KernelControllerContextAware

 public void setKernelControllerContext(KernelControllerContext controllerContext) throws Exception
 {
 this.controllerContext = controllerContext;
 }

 public void unsetKernelControllerContext(KernelControllerContext controllerContext) throws Exception
 {
 this.controllerContext = null;
 }

// Redirect JMX invocations to the real MC context, e.g.

 public void create() throws Exception
 {
 if (controllerContext != null)
 pojoChange(ControllerState.CREATE);
 else if (serviceName != null && isJBossInternalLifecycleExposed)
 server.invoke(ServiceController.OBJECT_NAME, "create", new Object[] { serviceName }, SERVICE_CONTROLLER_SIG);
 else
 jbossInternalCreate();
 }

The second issue is the use of StandardMBean in the @JMX handling.
The problem is that this masks any implemention of MBeanRegistration
or NotificationEmitter.

This is probably not a real issue since you can always use
registerDirectly=true on the @JMX annotation
to expose those contracts.
You're pretty certain its going to be a real mbean if you implements them.

A more general fix would to use of an extension of StandardMBean
that delegates those interface implementations to underlying pojo
if it implements them.

[adrian.brock]

The second issue is the use of StandardMBean in the @JMX handling.
The problem is that this masks any implemention of MBeanRegistration
or NotificationEmitter.

This masking was why my original test worked.
With the MBeanRegistration callback we don't know the ObjectName
and so can't redirect lifecycle requests to the ServiceController.

[alesj]

// Implement this interface so we know the true name not the JMX wrapper
// when we are registered as a POJO
KernelControllerContextAware

btw: this breaks the AS5_trunk ;-)

Just did a clean 'svn update, build clean most':

_default:compile-classes:
 [mkdir] Created dir: C:\projects\jboss5\trunk\jbossmq\output\gen-src
 [javac] Compiling 200 source files to C:\projects\jboss5\trunk\jbossmq\output\classes
C:\projects\jboss5\trunk\jbossmq\src\main\org\jboss\mq\il\ServerILJMXService.java:48: cannot access org.jboss.kernel.spi.dependency.KernelControllerContextAware
file org\jboss\kernel\spi\dependency\KernelControllerContextAware.class not found
public abstract class ServerILJMXService extends ServiceMBeanSupport implements ServerILJMXServiceMBean

[adrian.brock]

I've fixed that one.
I'll do a clean build and see what other classpaths need fixing.

[alesj]

I'll do a clean build and see what other classpaths need fixing.

I've had to fix:
- jbossmq
- management
- console

I can commit that, or you will?

[adrian.brock]

I've already done it. ;-)

[brian.stansberry]

Thanks, Adrian, for this. :-) Sorry the original report was unclear.

[brian.stansberry]

The changes you made resolved the problem I was seeing -- failing org.jboss.test.cluster.defaultcfg.test.HASingletonElectionPolicyUnitTestCase. I also see the HASingletonController for the HASingletonDeploymentScanner bean now properly starts the singleton (although the scanner doesn't work correctly -- unrelated issue).

I saw a couple regressions from this:

1) HAPartitionStateTransferTestCase has some failures. This is because the test fixture overrides ServiceMBeanSupport.start() and caches a ref to any exception thrown. Your addition of @Start public void pojoStart() means an exception thrown in startService() no longer propagates through start(). I changed the fixture to override startService() instead of start() (which is more correct anyway) and the test works fine.

2) PartitionRestartUnitTestCase now fails. I don't see anything wrong with the way ServiceMBeanSupport is working in this test; its a UnifiedInvoker/Remoting issue. See http://www.jboss.com/index.html?module=bb&op=viewtopic&t=136822. It's not clear to me why this test started failing now, other than that perhaps ServiceMBeanSupport wasn't working properly before and was masking the remoting issue.