11 Replies Latest reply on Oct 18, 2012 9:56 AM by jaikiran

    IllegalStateException: No EJB receiver available for handling when server name contains hyphen

    jasipher

      I have an application that has deployed and worked fine on multiple environments, but will not work on a server that has a hyphen in the name, e.g., xxxxqa-1.mycompany.com. I've deployed the application (successfully) in WebSphere on the same server just to verify that it's a JBoss-specific issue and not something strange we're doing in the application.

       

      The application is deployed as a EAR containing several EJBs and a couple web applications. It fails when one of the servlets tries to access an EJB during the servlet's initialization.

       

      I'll post a stack trace later after I get a new test environment set up (I wasn't quick enough to copy the logs to my laptop before QA decided to rename their servers), but here's what I found while trying to debug it. Line numbers are from jboss-as-7.1.2.Final/jboss-ejb-client-1.0.10.Final.

       

      The "No EJB receiver available" message is triggered by the "return null" at line 539 in org.jboss.ejb.client.EjbClientContext. This snippet starts at line 536

       

          EJBReceiver getEJBReceiver(final EJBClientInvocationContext invocationContext, final String appName, final String moduleName, final String distinctName) {

              final Iterator<EJBReceiver> iterator = getEJBReceivers(appName, moduleName, distinctName).iterator();

              if (!iterator.hasNext()) {

                  return null;

              }

       

      The getEJBReceivers(appName, moduleName, distinctName) ends up in this method starting at line 469

          private Collection<EJBReceiver> getEJBReceivers(final String appName, final String moduleName, final String distinctName,

                                                          final boolean attemptReconnect) {

              final Collection<EJBReceiver> eligibleEJBReceivers = new HashSet<EJBReceiver>();

              synchronized (this.ejbReceiverAssociations) {

                  for (final Map.Entry<EJBReceiver, ReceiverAssociation> entry : this.ejbReceiverAssociations.entrySet()) {

                      if (entry.getValue().associated) {

                          final EJBReceiver ejbReceiver = entry.getKey();

                          if (ejbReceiver.acceptsModule(appName, moduleName, distinctName)) {

                              eligibleEJBReceivers.add(ejbReceiver);

                          }

                      }

                  }

              }


      The interesting thing (to me anyway) is that if I put a breakpoint on the for statement at line 473 in getEJBReceivers(), this.ejbReceiverAssociations will be empty. But, if I move the breakpoint up a couple lines (before the synchronized), this.ejbReceiverAssociations will not be empty and I won't get the "No EJB receiver..." message. That makes me think that there could be some multi-threading issue, but it doesn't make sense that it would show up only in this one environment with the hyphen in the host name.

       

      Any suggestions will be appreciated.

        • 1. Re: IllegalStateException: No EJB receiver available for handling when server name contains hyphen
          wdfink

          Do you can reproduce this? If you replace the "-" from the server name does it works.

          At the moment it looks not clear to me (whether it is the '-' or a mthread problem).

          Could you provide the client code/config-files and the server.log output of the bean-deployment?

          • 2. Re: IllegalStateException: No EJB receiver available for handling when server name contains hyphen
            jasipher

            I deployed this on Debian Linux with a hyphen in the host name and it worked fine, so I think that's probably not the issue. The environment where I'm having the problem is Windows Server 2008 R2.

             

            It is reproducible and the contents of server.log follows. I can't really provide client code - I'm pretty sure my employer would frown on that - but if you can tell me what you're looking for I'd be glad to track it down and sanitiize enough that you can see what's going on.

             

            Thanks for your help.

             

             

            [Edited by jaikiran pai: John, that was a very big log that you had pasted inline in your post, making it dificult to read this thread. I have instead cut that text and attached that log to this post]

             

            [Edited by John Sipher: Thank you Jaikran. I looked and looked for a way to attache the file and couldn't find it, so I went ahead and pasted it. I'm sure there is a button or menu option  right here in front of me somewhere, but I still can't see it. And of course as I scroll down looking for the 'post' button after writing that, I see the 'Attach Files' thing right there. Just.Plain.Humiliating]

            • 3. Re: IllegalStateException: No EJB receiver available for handling when server name contains hyphen
              jasipher

              The hyphen definitely is not the issue. Our QA team renamed one of the problem servers and removed the hyphen, but the problem still exists.

              • 4. Re: IllegalStateException: No EJB receiver available for handling when server name contains hyphen
                ksreen

                There are some Warnings  "[org.jboss.ejb.client.remoting.ChannelAssociation] (Remoting "scjs2-uiservice1" task-3) Unsupported message received with header 0xffffffff" in the server.log. It could be because of client jar you might be using would be old and remoting protocol has changed. You may want to check the version of JBoss AS  on windows and you use the latest jar that goes with the version.

                • 5. Re: IllegalStateException: No EJB receiver available for handling when server name contains hyphen
                  jaikiran

                  ksreen wrote:

                   

                  There are some Warnings  "[org.jboss.ejb.client.remoting.ChannelAssociation] (Remoting "scjs2-uiservice1" task-3) Unsupported message received with header 0xffffffff" in the server.log.

                  That WARN can be ignored. We actually removed it from WARN logging some time back.

                  • 6. Re: IllegalStateException: No EJB receiver available for handling when server name contains hyphen
                    jasipher

                    Ok, now I understand the issue. I added some logging to org.jboss.ejb.client.EJBReceiver.registerModule() and .acceptsModule() so that I could see when modules are being registered vs. when the web application is trying is trying to access them.

                     

                    13:08:57,097 DEBUG [org.jboss.ejb.client.EJBReceiver] (MSC service thread 1-2) accepts 'service'/'shared_ejb'/'' = false

                    13:09:02,373 DEBUG [org.jboss.ejb.client.EJBReceiver] (MSC service thread 1-2) accepts 'service'/'shared_ejb'/'' = false

                    13:09:03,325 DEBUG [org.jboss.ejb.client.EJBReceiver] (MSC service thread 1-2) registering 'service'/'shared_ejb'/''

                    13:09:03,325 DEBUG [org.jboss.ejb.client.EJBReceiver] (MSC service thread 1-2) registering 'service'/'shared_ejb'/''

                     

                    I can see that the web application is starting and trying to access the EJBs before service/shared_ejb is registered with the LocalEJBReceiver. Any suggestions for how I prevent that?

                    • 7. Re: IllegalStateException: No EJB receiver available for handling when server name contains hyphen
                      jasipher

                      I'm not 100% sure of this, but I believe the problem environments are the only single-core systems where we've tried deploying the application with JBoss AS7. Is there any chance that could be causing the problem? I'm trying to get access to another single-core system just to see whether that's consistent.

                       

                      I tried addiing a method to EJBClientContext that would let it wait for the EJB module to be registered with an EJB receiver. Very simplistic - just check whether the module is registered, and if not sleep for 1/2 second and try again.

                       

                          static public void waitForModule(final String appName, final String moduleName, final String distinctName) {

                                    while (getCurrent().getEJBReceiver(appName, moduleName, distinctName) == null) {

                                              logger.info(String.format("Waiting for EJB receiver for module '%1$s'/'%2$s'/'%3$s'", appName, moduleName, distinctName));

                                                    try {

                                                              Thread.sleep(500);

                                                    } catch (InterruptedException ignored) {

                                                    }

                                    }

                          }

                       

                      The method just blocks forever. It's being called from a servlet's init() method, so apparently something in the stack above that is preventing the EJB module from finishing its initialization.

                      • 8. Re: IllegalStateException: No EJB receiver available for handling when server name contains hyphen
                        jaikiran

                        I can see that the web application is starting and trying to access the EJBs before service/shared_ejb is registered with the LocalEJBReceiver. Any suggestions for how I prevent that?

                        So your client is a web application? So far I had been thinking that you were talking about a remote standalone client. Apparently not.

                         

                        If your web application is doing this lookup in some servlet context listener and the EJBs are in a separate app, then there's no guarantee that the lookup will always succeed. Like you are seeing now. You should set up some kind of dependency between those applications. One way to do that is instead of doing a lookup, you can inject the EJB using the @EJB annotation like:

                         

                        @EJB(lookup="ejb:app/module/distinctname/beanName!beanInterface")

                        private MyBeanInOtherDeployment foo;

                        • 9. Re: IllegalStateException: No EJB receiver available for handling when server name contains hyphen
                          jasipher

                          The web app and the EJBs are in the same enterprise app. The application is built on an internally developed framework based on EJB 2.0, and modifying it to use EJB 3.0 annotations is not a practical solution. And, since we support other J2EE serverrs, using annotations with your non-standard ejb: prefix instead of java: wouldn't work even if we could go that route.

                          • 10. Re: IllegalStateException: No EJB receiver available for handling when server name contains hyphen
                            jaikiran

                            Annotations was just one option. You can use the web.xml to add a ejb-ref to let the container know that the web application depends on that EJB. Without specifying some kind of dependency, the container has no idea that you are going to do a lookup of an EJB when your web app starts.

                            • 11. Re: IllegalStateException: No EJB receiver available for handling when server name contains hyphen
                              jaikiran

                              jaikiran pai wrote:

                               

                              Annotations was just one option. You can use the web.xml to add a ejb-ref to let the container know that the web application depends on that EJB. Without specifying some kind of dependency, the container has no idea that you are going to do a lookup of an EJB when your web app starts.

                              Or you can go with this approach of using the spec provided initialize-in-order element in the application.xml https://docs.jboss.org/author/display/AS71/Developer+Guide#DeveloperGuide-application.xml