[Resloved] OpenSplice and the ESB

[pintodragon]

Software being used:

• Jboss 5.1 AS
• JBoss ESB 4.9
• Default configuration using JDK 1.6
• ESB Config Version 1.0.1
• OpenSpliceV5.3 Community Edition
• CentOS 5.5
• Eclipse 3.6

Idl processed using a standalone config:

• Idlpp –S –l java Test.idl

To fix a class loading issue I have placed dcpssaj.jar and dcpscj.jar in my jboss/server/default/lib directory.

On first deploy the OpenSplice gateway I have created works fine but when I try to redeploy it I get the following error message:

14:43:32,161ERROR [STDERR] org.omg.CORBA.BAD_PARAM vmcid: 0x0  minor code: 0  completed: No
14:43:32,162ERROR [STDERR]             atorg.opensplice.dds.dcps.Utilities.createException(Unknown Source)
14:43:32,162ERROR [STDERR]             atTest.TestItemDataReaderHelper.narrow(TestItemDataReaderHelper.java:15)
14:43:32,162ERROR [STDERR]             atexample.dds.gateway.OpenSpliceGateway.doInitialise(OpenSpliceGateway.java:82)
14:43:32,162ERROR [STDERR]             atorg.jboss.soa.esb.listeners.lifecycle.AbstractManagedLifecycle.initialise(AbstractManagedLifecycle.java:133)
14:43:32,162ERROR [STDERR]             atorg.jboss.soa.esb.listeners.lifecycle.ManagedLifecycleController.initialiseInstances(ManagedLifecycleController.java:109)
14:43:32,162ERROR [STDERR]             atorg.jboss.soa.esb.listeners.lifecycle.ManagedLifecycleController.start(ManagedLifecycleController.java:66)
14:43:32,162ERROR [STDERR]             atorg.jboss.soa.esb.listeners.deployers.mc.EsbDeployment.start(EsbDeployment.java:126)
14:43:32,162ERROR [STDERR]             atsun.reflect.NativeMethodAccessorImpl.invoke0(NativeMethod)
14:43:32,162ERROR [STDERR]             atsun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
14:43:32,162ERROR [STDERR]             atsun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
14:43:32,162ERROR [STDERR]             atjava.lang.reflect.Method.invoke(Method.java:597)
14:43:32,162ERROR [STDERR]             at org.jboss.reflect.plugins.introspection.ReflectionUtils.invoke(ReflectionUtils.java:59)
14:43:32,162ERROR [STDERR]             atorg.jboss.reflect.plugins.introspection.ReflectMethodInfoImpl.invoke(ReflectMethodInfoImpl.java:150)
14:43:32,162ERROR [STDERR]             atorg.jboss.joinpoint.plugins.BasicMethodJoinPoint.dispatch(BasicMethodJoinPoint.java:66)
14:43:32,163ERROR [STDERR]             atorg.jboss.kernel.plugins.dependency.KernelControllerContextAction$JoinpointDispatchWrapper.execute(KernelControllerContextAction.java:241)
14:43:32,163ERROR [STDERR]             atorg.jboss.kernel.plugins.dependency.ExecutionWrapper.execute(ExecutionWrapper.java:47)
14:43:32,163ERROR [STDERR]             atorg.jboss.kernel.plugins.dependency.KernelControllerContextAction.dispatchExecutionWrapper(KernelControllerContextAction.java:109)
14:43:32,163ERROR [STDERR]             atorg.jboss.kernel.plugins.dependency.KernelControllerContextAction.dispatchJoinPoint(KernelControllerContextAction.java:70)
14:43:32,163ERROR [STDERR]             atorg.jboss.kernel.plugins.dependency.LifecycleAction.installActionInternal(LifecycleAction.java:221)
14:43:32,163ERROR [STDERR]             atorg.jboss.kernel.plugins.dependency.InstallsAwareAction.installAction(InstallsAwareAction.java:54)
14:43:32,163ERROR [STDERR]             atorg.jboss.kernel.plugins.dependency.InstallsAwareAction.installAction(InstallsAwareAction.java:42)
14:43:32,163ERROR [STDERR]             atorg.jboss.dependency.plugins.action.SimpleControllerContextAction.simpleInstallAction(SimpleControllerContextAction.java:62)
14:43:32,163ERROR [STDERR]             atorg.jboss.dependency.plugins.action.AccessControllerContextAction.install(AccessControllerContextAction.java:71)
14:43:32,163ERROR [STDERR]             atorg.jboss.dependency.plugins.AbstractControllerContextActions.install(AbstractControllerContextActions.java:51)
14:43:32,163ERROR [STDERR]             atorg.jboss.dependency.plugins.AbstractControllerContext.install(AbstractControllerContext.java:348)
14:43:32,163ERROR [STDERR]             atorg.jboss.dependency.plugins.AbstractController.install(AbstractController.java:1631)
14:43:32,163ERROR [STDERR]             atorg.jboss.dependency.plugins.AbstractController.incrementState(AbstractController.java:934)
14:43:32,163ERROR [STDERR]             atorg.jboss.dependency.plugins.AbstractController.resolveContexts(AbstractController.java:1082)
14:43:32,163ERROR [STDERR]             atorg.jboss.dependency.plugins.AbstractController.resolveContexts(AbstractController.java:984)
14:43:32,163ERROR [STDERR]             atorg.jboss.dependency.plugins.AbstractController.install(AbstractController.java:774)
14:43:32,163ERROR [STDERR]             atorg.jboss.dependency.plugins.AbstractController.install(AbstractController.java:540)
14:43:32,163ERROR [STDERR]             atorg.jboss.deployers.vfs.deployer.kernel.BeanMetaDataDeployer.deploy(BeanMetaDataDeployer.java:121)
14:43:32,163ERROR [STDERR]             atorg.jboss.deployers.vfs.deployer.kernel.BeanMetaDataDeployer.deploy(BeanMetaDataDeployer.java:51)
14:43:32,164ERROR [STDERR]             atorg.jboss.deployers.spi.deployer.helpers.AbstractSimpleRealDeployer.internalDeploy(AbstractSimpleRealDeployer.java:62)
14:43:32,164ERROR [STDERR]             atorg.jboss.deployers.spi.deployer.helpers.AbstractRealDeployer.deploy(AbstractRealDeployer.java:50)
14:43:32,164ERROR [STDERR]             atorg.jboss.deployers.plugins.deployers.DeployerWrapper.deploy(DeployerWrapper.java:171)
14:43:32,164ERROR [STDERR]             atorg.jboss.deployers.plugins.deployers.DeployersImpl.doDeploy(DeployersImpl.java:1439)
14:43:32,164ERROR [STDERR]             atorg.jboss.deployers.plugins.deployers.DeployersImpl.doInstallParentFirst(DeployersImpl.java:1157)
14:43:32,164ERROR [STDERR]             atorg.jboss.deployers.plugins.deployers.DeployersImpl.doInstallParentFirst(DeployersImpl.java:1178)
14:43:32,164ERROR [STDERR]             atorg.jboss.deployers.plugins.deployers.DeployersImpl.install(DeployersImpl.java:1098)
14:43:32,164ERROR [STDERR]             atorg.jboss.dependency.plugins.AbstractControllerContext.install(AbstractControllerContext.java:348)
14:43:32,164ERROR [STDERR]             atorg.jboss.dependency.plugins.AbstractController.install(AbstractController.java:1631)
14:43:32,164ERROR [STDERR]             atorg.jboss.dependency.plugins.AbstractController.incrementState(AbstractController.java:934)
14:43:32,164ERROR [STDERR]             atorg.jboss.dependency.plugins.AbstractController.resolveContexts(AbstractController.java:1082)
14:43:32,164ERROR [STDERR]             atorg.jboss.dependency.plugins.AbstractController.resolveContexts(AbstractController.java:984)
14:43:32,164ERROR [STDERR]             atorg.jboss.dependency.plugins.AbstractController.change(AbstractController.java:822)
14:43:32,164ERROR [STDERR]             atorg.jboss.dependency.plugins.AbstractController.change(AbstractController.java:553)
14:43:32,164ERROR [STDERR]             atorg.jboss.deployers.plugins.deployers.DeployersImpl.process(DeployersImpl.java:781)
14:43:32,164ERROR [STDERR]             at org.jboss.deployers.plugins.main.MainDeployerImpl.process(MainDeployerImpl.java:702)
14:43:32,164ERROR [STDERR]             atorg.jboss.system.server.profileservice.repository.MainDeployerAdapter.process(MainDeployerAdapter.java:117)
14:43:32,164ERROR [STDERR]             atorg.jboss.system.server.profileservice.hotdeploy.HDScanner.scan(HDScanner.java:362)
14:43:32,165ERROR [STDERR]             atorg.jboss.system.server.profileservice.hotdeploy.HDScanner.run(HDScanner.java:255)
14:43:32,165ERROR [STDERR]             atjava.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441)
14:43:32,165ERROR [STDERR]             atjava.util.concurrent.FutureTask$Sync.innerRunAndReset(FutureTask.java:317)
14:43:32,165ERROR [STDERR]             atjava.util.concurrent.FutureTask.runAndReset(FutureTask.java:150)
14:43:32,165ERROR [STDERR]             atjava.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(ScheduledThreadPoolExecutor.java:98)
14:43:32,165ERROR [STDERR]             atjava.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.runPeriodic(ScheduledThreadPoolExecutor.java:180)
14:43:32,165ERROR [STDERR]             atjava.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:204)
14:43:32,165ERROR [STDERR]             atjava.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
14:43:32,165ERROR [STDERR]             atjava.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
14:43:32,165ERROR [STDERR]             atjava.lang.Thread.run(Thread.java:662)
14:43:32,167WARN  [AbstractThreadedManagedLifecycle]Unexpected error from doRun()
java.lang.NullPointerException
            atexample.dds.gateway.OpenSpliceGateway.doRun(OpenSpliceGateway.java:101)
            atorg.jboss.soa.esb.listeners.lifecycle.AbstractThreadedManagedLifecycle.run(AbstractThreadedManagedLifecycle.java:115)
            at java.lang.Thread.run(Thread.java:662)

I know that the cast exception thrown by CORBA is due to a DataReaderImpl not being an instanceof a DataReaderImpl (which doesn't make sense to me as I have seen that it is an instance of during debugging of the gateway). Does anyone have any ideas on what is happening? My only guess is that it has something to do with the class loader and/or something internal to OpenSplice.

I am including my test ESB as well as the source to the ESB and the idl used to generate the OpenSplice code.

[kcbabo]

You might want to check with the folks at OpenSplice.  This forum is for user discussion related directly to JBoss ESB.

[pintodragon]

I did make a post over at the OpenSplice forums as well. However since I am running into this issue when I deploy an ESB to the JBoss ESB and then attempt to redeploy it to the JBoss ESB and get the error, I am hoping that someone here might have an idea or a path to pursue in finding a solution for the issue at hand.

** Update **

So I have narrowed it down to the line testSubscriber.create_datareader(....) is returning a TestItemDataReaderImpl (according to the eclipse debugger) but when a test is done against instanceof it fails. The interesting thing is that the first time running this code it works fine and if I restart the jboss app server it deploys fine. It is only when I undeploy or redeploy the application (useful when making minor tweaks to both the jboss-esb.xml file and the gateway itself) that I get the CORBA.BAD_PARAM error.

I realize that this could be an issue with OpenSplice's api's and I have posted on their forums this same topic. The reason I am posting this here is because of how I am getting the error. There has to be something happening with the jboss app server that is causing the instantiation of the TestItemDataReaderImpl to not be an instanceof TestItemDataReaderImpl. Be it a classloading issue, incorrectly deploying the .jar files or just poor programming practice of my own code.

If anyone has any guidance or ideas of paths to go they would be greatly appreciated.

[tcunning]

It sounds like maybe it's using the ClassLoader of the previous deployment?      That should be something you can verify with a debugger.

[pintodragon]

It is a ClassLoader issue. I have a feeling that on Monday I will be posting a bug report to the OpenSplice community.

[pintodragon]

http://forums.opensplice.org/index.php?/topic/1097-create-participant-fails/

So it isn't an OpenSplice issue per say.

Understand that native code generally loads java classes by delegating back to the classloader attached to the java thread environment. In containers (like Tomcat or JBoss) this may be a different thread than the one in which you've originally loaded certain required resources. A general solution for passing a classloader between threads is below; you may have to find an approach specific to how Tomcat or JBoss manages threads.

The JNI related items are loaded by the first java thread in the environment that uses them. They are not released once the references to them in my code is through with them due to how JBoss handles those threads and the possibility of there being static references that still linger. A way I found around this is to "wrap" the DomainParticipant create all the way through the TestItemDataReader creation in a separate static factory and call it from my classes. Basically took everything out of doInitialise and placed it in a separate class. Then placed the static class in a jar that is located in the lib directory of the default server. Now the dataread is loaded and is always there and doesn't get recreated which was causing the issues with the corba bad param. Not a perfect solution but it does allow redeployment of the ESB without restarting the entire server.

Edit: Rather than using a static datareader I created a ThreadLocal<DataReader> and it worked just as well.