java.lang.IllegalStateException: No EJB receiver available for handling

[mattforshaw]

Hi all,

I'm running into a few issues getting remote EJB invocation working between two JBoss 7.1.1 instances.

I have created two fresh installations of JBoss, and followed the instructions at https://docs.jboss.org/author/display/AS71/EJB+invocations+from+a+remote+server+instance yet I encounter the following error...

17:07:42,538 ERROR [org.apache.catalina.core.ContainerBase.[jboss.web].[default-host].[/example].[jsp]] (http-localhost-127.0.0.1-8080-1) Servlet.service() for servlet jsp threw exception: java.lang.IllegalStateException: No EJB receiver available for handling [appName:,modulename:service,distinctname:] combination for invocation context org.jboss.ejb.client.EJBClientInvocationContext@19c33aa

          at org.jboss.ejb.client.EJBClientContext.requireEJBReceiver(EJBClientContext.java:584) [jboss-ejb-client-1.0.5.Final.jar:1.0.5.Final]

          at org.jboss.ejb.client.ReceiverInterceptor.handleInvocation(ReceiverInterceptor.java:119) [jboss-ejb-client-1.0.5.Final.jar:1.0.5.Final]

          at org.jboss.ejb.client.EJBClientInvocationContext.sendRequest(EJBClientInvocationContext.java:181) [jboss-ejb-client-1.0.5.Final.jar:1.0.5.Final]

          at org.jboss.ejb.client.EJBInvocationHandler.doInvoke(EJBInvocationHandler.java:136) [jboss-ejb-client-1.0.5.Final.jar:1.0.5.Final]

          at org.jboss.ejb.client.EJBInvocationHandler.doInvoke(EJBInvocationHandler.java:121) [jboss-ejb-client-1.0.5.Final.jar:1.0.5.Final]

          at org.jboss.ejb.client.EJBInvocationHandler.invoke(EJBInvocationHandler.java:104) [jboss-ejb-client-1.0.5.Final.jar:1.0.5.Final]

          at $Proxy21.createUser(Unknown Source)          at org.apache.jsp.index_jsp._jspService(index_jsp.java:75)

          at org.apache.jasper.runtime.HttpJspBase.service(HttpJspBase.java:70) [jbossweb-7.0.13.Final.jar:]

          at javax.servlet.http.HttpServlet.service(HttpServlet.java:847) [jboss-servlet-api_3.0_spec-1.0.0.Final.jar:1.0.0.Final]

          at org.apache.jasper.servlet.JspServletWrapper.service(JspServletWrapper.java:369) [jbossweb-7.0.13.Final.jar:]

          at org.apache.jasper.servlet.JspServlet.serviceJspFile(JspServlet.java:326) [jbossweb-7.0.13.Final.jar:]

          at org.apache.jasper.servlet.JspServlet.service(JspServlet.java:253) [jbossweb-7.0.13.Final.jar:]

          at javax.servlet.http.HttpServlet.service(HttpServlet.java:847) [jboss-servlet-api_3.0_spec-1.0.0.Final.jar:1.0.0.Final]

          at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:329) [jbossweb-7.0.13.Final.jar:]

          at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:248) [jbossweb-7.0.13.Final.jar:]

          at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:275) [jbossweb-7.0.13.Final.jar:]

          at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:161) [jbossweb-7.0.13.Final.jar:]

          at org.jboss.as.jpa.interceptor.WebNonTxEmCloserValve.invoke(WebNonTxEmCloserValve.java:50) [jboss-as-jpa-7.1.1.Final.jar:7.1.1.Final]

          at org.jboss.as.web.security.SecurityContextAssociationValve.invoke(SecurityContextAssociationValve.java:153) [jboss-as-web-7.1.1.Final.jar:7.1.1.Final]

          at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:155) [jbossweb-7.0.13.Final.jar:]

          at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102) [jbossweb-7.0.13.Final.jar:]

          at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109) [jbossweb-7.0.13.Final.jar:]

          at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:368) [jbossweb-7.0.13.Final.jar:]

          at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:877) [jbossweb-7.0.13.Final.jar:]

          at org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:671) [jbossweb-7.0.13.Final.jar:]

          at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:930) [jbossweb-7.0.13.Final.jar:]

          at java.lang.Thread.run(Thread.java:680) [classes.jar:1.6.0_35]

Turning up the logging level does not give any further information related to this issue.

I am running both instances from standalone-full.xml, one with a port offset of 100. My arguments on startup are as follows.

sh bin/standalone.sh -server-config=standalone-full.xml -Djboss.node.name=one

sh bin/standalone.sh -server-config=standalone-full.xml -Djboss.socket.binding.port-offset=100 -Djboss.node.name=two

(Edit: To clarify, I have two distinct jboss directories.)

A number of threads suggest adding jndiProperties.put("jboss.naming.client.ejb.context", true); which I have tried to no avail.

I have also, while I don't think this should be necessary, tried to deploy within an ear to see if that helps, again to no avail.

In an attempt to simplify matters I tried to remove JBoss security for ejb calls by removing the security-realm in my standalone-full.xml file but this did not make any difference.

Interestingly, the ejb-remote quickstart runs successfully when running the client inside maven, yet I still encounter issues calling the remote methods from my JSP.

I have attached copies of my code (at the moment very simple test to get things off the ground). If there's anything else that would be helpful to know, don't hesitate to get in touch.

Any help in the matter would be much appreciated!

All the best,

Matt

Edit: I have tested on two machines (one Linux and the other OS X Mountain Lion), with the same result on both.

Message was edited by: Matt Forshaw

[wdfink]

You might have a look into this example https://github.com/wfink/jboss-as-quickstart/tree/ejb-multi-server

[paul.robinson]

Cheers,

FYI, I'm working with Matt (OP) to get this scenario working for a course we are delivering.

Do you require a specific version of AS7 to get this working? I ran the install-domain.cli script against a fresh jbossas-7.1.1.Final and I get the following output:

...
#94 /server-group=quickstart-ejb-multi-appWeb-server/jvm=default:add()
#95 /host=master/server-config=app-web:add(auto-start=true, group=quickstart-ejb-multi-appWeb-server, socket-binding-port-offset=300)
Failed to execute batch: JBAS010850: No handler for operation remove at address [
    ("host" => "master"),
    ("core-service" => "management"),
    ("security-realm" => "ApplicationRealm"),
    ("authentication" => "local")
]

Thanks,

Paul.

[mattforshaw]

Good morning all,

Apologies for the delay replying. I saw the same output from install-domain.cli and had assumed I had missed a config step. I've tried again from a second fresh installation and run up against the same problem.

Any other thoughts would be greatly appreciated.

All the best,

Matt

[paul.robinson]

Matt,

You can reproduce your problem by running the ejb-remote quickstart with the jboss-ejb-client.properties file missing. I suspect the problem is that jboss-ejb-client.properties is in the wrong location in your .war. However, according to Jaikiran, you have put it in the correct location (.war/WEB-INF/jboss-ejb-client.xml). See: https://community.jboss.org/message/717063#717063

Paul.

[paul.robinson]

Matt,

I've tried taking the ejb-remote quickstart and converting the client to a .war. I get the same problem as you do.

From looking at other people with the same issue, it seems like "No EJB receiver available for handling" is a general error message and doesn't convey the real root cause of the problem.

I also don't think it's a security issue as the remote client, driven by maven does work, which I don't believe it would if it was the security blocking the call.

Paul.

[jaikiran]

When you are doing server to server EJB invocations, the jboss-ejb-client.properties won't come into picture. You'll need a jboss-ejb-client.xml. The article that Matt pointed to has the details. If you are still running into issues with that, then please post the output of

jar -tf yourwar.war

and also the contents of the jboss-ejb-client.xml file.

[paul.robinson]

Hello Jaikiran, Thanks for helping us out with this.

I've gone through the guide that Matt posted a link to. I now think I have the client setup, but ptobably incorrectly as I'm still seeing the same error.

Here's my jboss-ejb-client.xml:

<jboss-ejb-client xmlns="urn:jboss:ejb-client:1.0">
    <client-context>
        <ejb-receivers>
            <remoting-ejb-receiver outbound-connection-ref="remote-ejb-connection"/>
        </ejb-receivers>
    </client-context>
</jboss-ejb-client>

and the contents of my war:

jar -tf target/example.war 
META-INF/
META-INF/MANIFEST.MF
WEB-INF/
WEB-INF/classes/
WEB-INF/lib/
index.jsp
WEB-INF/jboss-ejb-client.xml
WEB-INF/lib/service-1.0.jar
WEB-INF/web.xml
META-INF/maven/
META-INF/maven/uk.ac.ncl.cs.example/
META-INF/maven/uk.ac.ncl.cs.example/web/
META-INF/maven/uk.ac.ncl.cs.example/web/pom.xml
META-INF/maven/uk.ac.ncl.cs.example/web/pom.properties

One thing that confused me was, where to put the jboss-ejb-client.xml in the war. I've seen many people saying to put it in different places.

In my example, I'm binding the servers to 127.0.0.2 and 127.0.0.3. I've attached the standalone-full.xml configs for both servers.

Thanks again,

Paul.

[jaikiran]

Paul Robinson wrote:

One thing that confused me was, where to put the jboss-ejb-client.xml in the war. I've seen many people saying to put it in different places.

For top level .war deployments, the jboss-ejb-client.xml is supposed to be in the WEB-INF folder (just like you would place the web.xml file).

Paul Robinson wrote:

In my example, I'm binding the servers to 127.0.0.2 and 127.0.0.3.

If both those servers are running on the same machine then it's important to assign a unique jboss.node.name for both those instances. That article has a note about it. So you should be starting those servers with something like:

./standalone.sh -Djboss.node.name=foo

./standalone.sh -Djboss.node.name=bar

If this doesn't get you past the errors then let me know, I'll take a deeper look. Right now, I'm just trying to rule out the obvious issues.

[paul.robinson]

Jaikiran,

jaikiran pai wrote:

Paul Robinson wrote:

In my example, I'm binding the servers to 127.0.0.2 and 127.0.0.3.

If both those servers are running on the same machine then it's important to assign a unique jboss.node.name for both those instances. That article has a note about it. So you should be starting those servers with something like:

./standalone.sh -Djboss.node.name=foo

./standalone.sh -Djboss.node.name=bar

If this doesn't get you past the errors then let me know, I'll take a deeper look. Right now, I'm just trying to rule out the obvious issues.

This was the problem. I thought binding to different IP addresses would emulate, more closely, a distrbuted environment. Do you know why this was not the case?

[paul.robinson]

Also,

We intend to have a scenario with one 'client server' connecting to three 'service servers', where each 'service server' will host a different remote EJB. My understanding is that we can add multiple 'remoting-ejb-receiver' elements (and the corresponding config to the standalone-full.xml) to the 'jboss-ejb-client.xml' and the correct destination will be selected based on which has the required bean deployed?

Is this correct?

Thanks,

Paul.

[mattforshaw]

I would be interested to know what the expected bahaviour is when a bean is deployed to more than one server.

In my standalone-full.xml I have added an additional outbound-socket-binding for the second server. In my code I then create InitialContexts as follows...

final Hashtable jndiProperties = new Hashtable();

    jndiProperties.put(Context.URL_PKG_PREFIXES, "org.jboss.ejb.client.naming");

          jndiProperties.put(Context.PROVIDER_URL, "remote://localhost:4547");

          jndiProperties.put(Context.SECURITY_PRINCIPAL, "ejb");

          jndiProperties.put(Context.SECURITY_CREDENTIALS, "test");

    final Context context = new InitialContext(jndiProperties);

    final Hashtable jndiPropertiesB = new Hashtable();

    jndiPropertiesB.put(Context.URL_PKG_PREFIXES, "org.jboss.ejb.client.naming");

          jndiPropertiesB.put(Context.PROVIDER_URL, "remote://localhost:4647");

          jndiPropertiesB.put(Context.SECURITY_PRINCIPAL, "ejb");

          jndiPropertiesB.put(Context.SECURITY_CREDENTIALS, "test");

    final Context contextB = new InitialContext(jndiPropertiesB);

Identical EJBs are deployed on two JBoss instances (one with port offset 100, another with port offset 200).

I then lookup and invoke my beans as follows...

umLocal = (UserManager) localContext.lookup("ejb:/service/UserManagerBean!uk.ac.ncl.cs.itec.csc8104.modelsolution.beans.UserManager");

umLocalB = (UserManager) localContextB.lookup("ejb:/service/UserManagerBean!uk.ac.ncl.cs.itec.csc8104.modelsolution.beans.UserManager");

umLocal.createUser("Firstname","lastname");

umLocalB.createUser("First","Last");

I would have expected (and know in AS5 that this would be the case) that the correct server would be selected depending whether I made the call against umLocal or umLocalB.

Under AS7, I am seeing both calls directed towards a single server (it is not clear at present how one server is chosen over another - I'll keep digging).

Also, if I do not start up one of the two remote servers, no exception is thrown when trying to create the InitialContext and the calls are directed to the other server.

Is this expected behaviour or a bug? Is it possible to specify programmatically which server to direct calls to, and to disable any attempts to fail over onto another server if one is unavailable?

Thanks,

Matt

[wdfink]

Yes Paul,

this will be correct as the EJB is identified by it's name and you have different EJB's deployed to each 'service server'.

[wdfink]

I would have expected (and know in AS5 that this would be the case) that the correct server would be selected depending whether I made the call against umLocal or umLocalB.

No that is not true because under the hood the EJBContext is used and you can not use the remote-naming-project inside the server as there is no possiblity to change the EJBSelector in this way.

The only possibility ATM is to use the distinct name for the application or server and use ejb:/service/<distinct>/UserManagerBean..." where <distinct> is different.

The behavior not a bug because the EJB is identified by it's name. The lookup creates only a local proxy (stub) without connecting the destination server.

[jaikiran]

Paul Robinson wrote:

My understanding is that we can add multiple 'remoting-ejb-receiver' elements (and the corresponding config to the standalone-full.xml) to the 'jboss-ejb-client.xml' and the correct destination will be selected based on which has the required bean deployed?

Is this correct?

Yes, that's correct. The xsd for the jboss-ejb-client.xml is available in the JBOSS_HOME/docs/schema folder.