ejb3.0 problem, container not yet available??

[moksha2007]

Hi,

Brief situation:

A -> B (remote ejb lookup + invocation -> success)

B performs some ejbTimer logic, in the timeout I proceed

B -> A (remote ejb lookup + reflection/invocation -> fails due container not yet available)

I'm using Jboss 4.2.2.GA and pass the JNDI properties, as well as the interface (name) and method from A to B. From there I do some reflection to invoke a method. A and B are seperate Jboss instances on a single machine, using virtual ip's.

In a single JVM this works.

At first I discovered I had to set a RMISecurityManager at both machines, and so I did. Both sides have all permissions/grant. Based on the fact I succeed from A to B I get the feeling this is done ok, but perhaps I forgot something?

A snippet of the log/stacktrace:

2008-04-04 18:38:42,497 DEBUG [org.jboss.ejb.txtimer.TimerImpl] setTimerState: in_timeout
2008-04-04 18:38:42,531 WARN [org.jboss.ejb3.LocalProxy] Container jboss.j2ee:ear=phonemessage-1.0.0.ear,jar=phonemessage-engine-1.0.0.jar,name=CallHandlerServiceBean,service=EJB3,VMID=ff2143713a7260c3:-350bd643:1191a4ce5d2:-7ffd is not yet available
2008-04-04 18:38:42,531 DEBUG [com.thespeechcorporation.service.timer.ScheduleBean] Lookup result (using: 127.0.0.106:1099): CallHandlerServiceBean
2008-04-04 18:38:42,531 INFO [com.thespeechcorporation.service.timer.ScheduleBean] About to invoke a callback on: 'com.phonemessage.service.CallHandlerService'
2008-04-04 18:38:42,531 WARN [org.jboss.ejb3.LocalProxy] Container jboss.j2ee:ear=phonemessage-1.0.0.ear,jar=phonemessage-engine-1.0.0.jar,name=CallHandlerServiceBean,service=EJB3,VMID=ff2143713a7260c3:-350bd643:1191a4ce5d2:-7ffd is not yet available

and from there it breaks...

Any suggestions? If more information is required, let me know.

PS> I did a cross post, not sure where the post should go to:
http://www.jboss.com/index.html?module=bb&op=viewtopic&t=133106

Thanks in advance,
Mike Ahlers

[alrubinger]

Just a case of a cyclic dependency, looks like. Try out @IgnoreDependency?

In the future, provide the error stack trace so we can see what's up. ;)

S,
ALR

[moksha2007]

Oke, heres the stacktrace:

2008-04-04 18:38:42,531 WARN [org.jboss.ejb3.LocalProxy] Container jboss.j2ee:ear=phonemessage-1.0.0.ear,jar=phonemessage-engine-1.0.0.jar,name=CallHandlerServiceBean,service=EJB3,VMID=ff2143713a7260c3:-350bd643:1191a4ce5d2:-7ffd is not yet available
2008-04-04 18:38:42,531 ERROR [com.thespeechcorporation.service.timer.ScheduleBean] Unable to handle timer: null
java.lang.reflect.InvocationTargetException
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at com.thespeechcorporation.service.timer.ScheduleBean.handleTimeout(ScheduleBean.java:273)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.jboss.aop.joinpoint.MethodInvocation.invokeNext(MethodInvocation.java:112)
at org.jboss.ejb3.interceptor.InvocationContextImpl.proceed(InvocationContextImpl.java:166)
at org.jboss.ejb3.interceptor.EJB3InterceptorsInterceptor.invoke(EJB3InterceptorsInterceptor.java:63)
at org.jboss.aop.joinpoint.MethodInvocation.invokeNext(MethodInvocation.java:101)
at org.jboss.ejb3.entity.TransactionScopedEntityManagerInterceptor.invoke(TransactionScopedEntityManagerInterceptor.java:54)
at org.jboss.aop.joinpoint.MethodInvocation.invokeNext(MethodInvocation.java:101)
at org.jboss.ejb3.AllowedOperationsInterceptor.invoke(AllowedOperationsInterceptor.java:47)
at org.jboss.aop.joinpoint.MethodInvocation.invokeNext(MethodInvocation.java:101)
at org.jboss.aspects.tx.TxPolicy.invokeInOurTx(TxPolicy.java:79)
at org.jboss.aspects.tx.TxInterceptor$Required.invoke(TxInterceptor.java:191)
at org.jboss.aop.joinpoint.MethodInvocation.invokeNext(MethodInvocation.java:101)
at org.jboss.aspects.tx.TxPropagationInterceptor.invoke(TxPropagationInterceptor.java:95)
at org.jboss.aop.joinpoint.MethodInvocation.invokeNext(MethodInvocation.java:101)
at org.jboss.ejb3.stateless.StatelessInstanceInterceptor.invoke(StatelessInstanceInterceptor.java:62)
at org.jboss.aop.joinpoint.MethodInvocation.invokeNext(MethodInvocation.java:101)
at org.jboss.ejb3.security.Ejb3AuthenticationInterceptor.invoke(Ejb3AuthenticationInterceptor.java:108)
at org.jboss.aop.joinpoint.MethodInvocation.invokeNext(MethodInvocation.java:101)
at org.jboss.ejb3.ENCPropagationInterceptor.invoke(ENCPropagationInterceptor.java:46)
at org.jboss.aop.joinpoint.MethodInvocation.invokeNext(MethodInvocation.java:101)
at org.jboss.ejb3.asynchronous.AsynchronousInterceptor.invoke(AsynchronousInterceptor.java:106)
at org.jboss.aop.joinpoint.MethodInvocation.invokeNext(MethodInvocation.java:101)
at org.jboss.ejb3.stateless.StatelessContainer.callTimeout(StatelessContainer.java:175)
at org.jboss.ejb.txtimer.TimerImpl$TimerTaskImpl.run(TimerImpl.java:561)
at java.util.TimerThread.mainLoop(Timer.java:512)
at java.util.TimerThread.run(Timer.java:462)
Caused by: javax.ejb.EJBException: Invalid (i.e. remote) invocation of local interface (null container)
at org.jboss.ejb3.stateless.StatelessLocalProxy.invoke(StatelessLocalProxy.java:80)
at $Proxy88.deliver(Unknown Source)
... 34 more

[alrubinger]

Much more helpful.

Caused by: javax.ejb.EJBException: Invalid (i.e. remote) invocation of local interface (null container)
 at org.jboss.ejb3.stateless.StatelessLocalProxy.invoke(StatelessLocalProxy.java:80)
 at $Proxy88.deliver(Unknown Source)

Looks like someone's doing spookiness. ;)

Probably gotta post the code showing what you're doing here too, but the error at face value reads simple enough. Are you grabbing the local proxy via JNDI from a separate JVM?

"Local" and "Remote" pertain to JVMs - local is only inter-JVM calls. Anything else is "remote", regardless of the physical location of the runtime. Pass-by-value and network marshalling are required, and local passes references.

S,
ALR

[moksha2007]

Oke, going to put in some background:

Instance A:

From boot.log:
15:56:18,265 DEBUG [ServiceCreator] Created bean: jboss:service=Naming
15:56:18,265 DEBUG [ServiceConfigurator] CallByValue set to false in jboss:service=Naming
15:56:18,266 DEBUG [ServiceConfigurator] Port set to 1099 in jboss:service=Naming
15:56:18,267 DEBUG [ServiceConfigurator] BindAddress set to 127.0.0.106 in jboss:service=Naming
15:56:18,267 DEBUG [ServiceConfigurator] RmiPort set to 1098 in jboss:service=Naming
15:56:18,268 DEBUG [ServiceConfigurator] RmiBindAddress set to 127.0.0.106 in jboss:service=Naming

Instance B:

From boot.log:
15:57:29,698 DEBUG [ServiceCreator] Created bean: jboss:service=Naming
15:57:29,698 DEBUG [ServiceConfigurator] CallByValue set to false in jboss:service=Naming
15:57:29,699 DEBUG [ServiceConfigurator] Port set to 1099 in jboss:service=Naming
15:57:29,700 DEBUG [ServiceConfigurator] BindAddress set to 127.0.0.107 in jboss:service=Naming
15:57:29,701 DEBUG [ServiceConfigurator] RmiPort set to 1098 in jboss:service=Naming
15:57:29,701 DEBUG [ServiceConfigurator] RmiBindAddress set to 127.0.0.107 in jboss:service=Naming

I can telnet to both, so I guess those are running fine.

In instance A, I have a method that does this (basically setting up some context to be used for the callback/invocation as soon the timer goes off at B):

public void schedule(...) {
...
Properties props = new Properties();
props.put(Context.INITIAL_CONTEXT_FACTORY, "org.jnp.interfaces.NamingContextFactory");
props.put(Context.PROVIDER_URL, "127.0.0.106:1099");
...
ScheduleRequest request = new ScheduleRequest();
request.setApplicationContext(phoneMessage.getId());
request.setStartTime(startTime);
request.setEndTime(endTime);
request.setAverageDuration(new Long(duration).intValue());
request.setCallbackMethod("deliver");
request.setJndiContext(props);
request.setJndiReferenceLookup(CallHandlerService.class.getName());
request.setPreferredDeliveryDate(phoneMessage.getPreferredDeliveryDate());
request.setTimezone(phoneMessage.getTimeZone());
request.setTotalCalls(this.phoneMessageRepository.getNumberOfRecipients(phoneMessage.getId()));
this.getScheduler().schedule(request);
if (log.isInfoEnabled()) {
log.info("Done scheduling PhoneMesage with id: " + phoneMessage.getId());
}
}

As you can see, I am avoiding cyclic dependencies, as I try to keep the applications 'unaware' of each other, by passing primitives.
From the logs in A, I do see the scheduling is done. The lookup is done as follow:

protected Scheduler getScheduler() {
return (Scheduler) this.lookupService(Scheduler.class.getName());
}

protected Object lookupService(String serviceName) {
// Set security policy
if(System.getSecurityManager() == null) {
this.log.info("Using security policy file: " + System.getProperty("java.security.policy"));
System.setSecurityManager(new RMISecurityManager());
}
Properties props = new Properties();
props.put(Context.INITIAL_CONTEXT_FACTORY, "org.jnp.interfaces.NamingContextFactory");
props.put(Context.PROVIDER_URL, "jnp://127.0.0.107:1099");

try {
this.log.info("Lookup called in CallHandlerServiceBean...");
Context ctx = new InitialContext(props);
return ctx.lookup(serviceName);
} catch (NamingException e) {
log.fatal("Unable to lookup service interface: " + serviceName);
e.printStackTrace();
}
return null;
}

The callback method:

public void deliver(Serializable applicationContext, int numberOfCalls) {
this.log.info("Deliver called with serializable: " + applicationContext);
...
}

In instance B:

@Timeout
public void handleTimeout(Timer timer) {
// Fetch context from timer
if(timer.getInfo() instanceof ScheduleContext) {
try {
ScheduleContext scheduleContext = (ScheduleContext) timer.getInfo();
// Set security policy
if(System.getSecurityManager() == null) {
if(this.log.isDebugEnabled()) {
this.log.debug("Using security policy file: " + System.getProperty("java.security.policy"));
}
System.setSecurityManager(new RMISecurityManager());
}
// Obtain reference of remote interface
if(this.log.isDebugEnabled()) {
this.log.debug("Lookup called in ScheduleBean...");
}
InitialContext initialContext = new InitialContext(scheduleContext.getJndiContext());
Object remoteInterface = initialContext.lookup(scheduleContext.getJndiReferenceLookup());
if(this.log.isDebugEnabled()) {
this.log.debug("Lookup result (using: " + scheduleContext.getJndiContext().getProperty(Context.PROVIDER_URL) + "): " + remoteInterface);
}
// Obtain method
Method method = remoteInterface.getClass().getMethod(scheduleContext.getMethodName(), new Class[] {Serializable.class, int.class});
// Invoke call back
if(this.log.isInfoEnabled()) {
this.log.info("About to invoke a callback on: '" + scheduleContext.getJndiReferenceLookup() + "'");
}
method.invoke(remoteInterface, new Object[] {scheduleContext.getApplicationContext(), scheduleContext.getTotalCalls()});
if(this.log.isInfoEnabled()) {
this.log.info("Callback succeeded...");
}
} catch (NamingException e) {
this.log.error("Unable to handle timer: " + e.getMessage(), e);
} catch (SecurityException e) {
this.log.error("Unable to handle timer: " + e.getMessage(), e);
} catch (NoSuchMethodException e) {
this.log.error("Unable to handle timer: " + e.getMessage(), e);
} catch (IllegalArgumentException e) {
this.log.error("Unable to handle timer: " + e.getMessage(), e);
} catch (IllegalAccessException e) {
this.log.error("Unable to handle timer: " + e.getMessage(), e);
} catch (InvocationTargetException e) {
this.log.error("Unable to handle timer: " + e.getMessage(), e);
} catch (Exception e) {
this.log.error("Unable to handle timer: " + e.getMessage(), e);
}
} else {
if(this.log.isWarnEnabled()) {
this.log.warn("Unable to handle timer event, missing ScheduleContext...");
}
}

I agree the try catch is verbose and could be optimized, but hey...

Is this a classloading issue? Call by value? LocalProxy should be something remote? Hopefully you can grasp the picture of what I am trying todo here.

Thanks in advance,
Mike

[alrubinger]

"moksha2007" wrote:

request.setJndiReferenceLookup(CallHandlerService.class.getName());

Does this point to a Local Business interface? Check out JNDIView in the JMX Console.

If so, that would explain the error.

S,
ALR

[moksha2007]

From which instance you'd like to know the JNDI settings? But the code/annotation is configured for both:

@Stateless
@Local({CallHandlerService.class})
@Remote({CallHandlerService.class})
@Interceptors({NullReferenceCheck.class})
public class CallHandlerServiceBean implements CallHandlerService {

Some snippets from server.log in instance A:

2008-04-04 18:37:45,858 DEBUG [org.jboss.ejb3.Ejb3DescriptorHandler] adding class annotation org.jboss.annotation.ejb.LocalBinding to com.phonemessage.service.CallHandlerServiceBean org.jboss.annotation.ejb.LocalBindingImpl@372f2b32
2008-04-04 18:37:45,859 DEBUG [org.jboss.ejb3.Ejb3DescriptorHandler] adding class annotation org.jboss.annotation.ejb.RemoteBinding to com.phonemessage.service.CallHandlerServiceBean [RemoteBindingImpl:, jndi=com.phonemessage.service.CallHandlerService, stack=, bindUrl=, proxyFactory=interface org.jboss.ejb3.remoting.RemoteProxyFactory]
2008-04-04 18:37:45,864 DEBUG [org.jboss.ejb3.Ejb3DescriptorHandler] adding class annotation org.jboss.annotation.ejb.RemoteBinding to com.phonemessage.service.CallHandlerServiceBean [RemoteBindingImpl:, jndi=com.phonemessage.service.CallHandlerService, stack=, bindUrl=, proxyFactory=interface org.jboss.ejb3.remoting.RemoteProxyFactory]
2008-04-04 18:37:45,865 DEBUG [org.jboss.ejb3.Ejb3AnnotationHandler] found EJB3: ejbName=CallHandlerServiceBean, class=com.phonemessage.service.CallHandlerServiceBean, type=STATELESS

2008-04-04 18:37:47,483 DEBUG [org.jboss.system.ServiceController] starting service jboss.j2ee:ear=phonemessage-1.0.0.ear,jar=phonemessage-engine-1.0.0.jar,name=CallHandlerServiceBean,service=EJB3

2008-04-04 18:37:47,487 INFO [org.jboss.ejb3.EJBContainer] STARTED EJB: com.phonemessage.service.CallHandlerServiceBean ejbName: CallHandlerServiceBean
2008-04-04 18:37:47,506 DEBUG [org.jboss.ejb3.stateless.BaseStatelessProxyFactory] Binding proxy for CallHandlerServiceBean in JNDI at com.phonemessage.service.CallHandlerService
2008-04-04 18:37:47,515 DEBUG [org.jboss.ejb3.stateless.BaseStatelessProxyFactory] Binding proxy for CallHandlerServiceBean in JNDI at com.phonemessage.service.CallHandlerService

2008-04-04 18:37:47,520 DEBUG [org.jboss.ejb3.Ejb3Deployment] Bound ejb3 container jboss.j2ee:ear=phonemessage-1.0.0.ear,jar=phonemessage-engine-1.0.0.jar,name=CallHandlerServiceBean,service=EJB3
2008-04-04 18:37:47,520 INFO [org.jboss.ejb3.JmxKernelAbstraction] jboss.j2ee:ear=phonemessage-1.0.0.ear,jar=phonemessage-engine-1.0.0.jar,name=CallHandlerServiceBean,service=EJB3

The JNDI view, using JMX on instance A shows:

com.phonemessage.service.CallHandlerService (proxy: $Proxy95 implements interface com.phonemessage.service.CallHandlerService,interface org.jboss.ejb3.JBossProxy)

Does that answer the question?

I'd like to emphasize when deploying both ears in a single instance, the code works (given the JNDI is changed localhost:1099) for a local development machine. However, that is not the situation on the production machine.

[moksha2007]

Now to think of it,

Could it be that:

Adding some JNDI property to search for a remote proxy will solve it? If so, how and what do I do that?

or

I split the local and remote interface, so they come with unique names. That will 'force' the lookup in obtaining the right proxy.

Your thoughts?

[moksha2007]

Problem solved, by enforcing lookup of remote interface. I did this by changing the name of the interface to: CallHandlerServiceRemote.

Hopefully others are helped by this as well.

[alrubinger]

Yes, you arrived at the solution. Here you have:

@Local({CallHandlerService.class})
@Remote({CallHandlerService.class})

Which your version of JBoss has allowed, but in fact this is prohibited:

"EJB3 Core Specification 4.6.6" wrote:
The same business interface cannot be both a local and a remote business interface of the bean.

I can't find the JIRA for this at the moment, but it's been fixed and now cites an error on deployment.

S,
ALR

[moksha2007]

Thats ok, anyways I want to thank you for the support/effort!

[wolfc]

http://jira.jboss.org/jira/browse/EJBTHREE-751