13 Replies Latest reply on Aug 17, 2012 8:31 PM by ybxiang.china

    Help! Operation failed with status WAITING + No EJB receiver available for handling

    ybxiang.china

      Dear all,

       

          I met a strange case during EJB invocation.

       

      Prepare:

            1. My JBoss 7.1.1 Server is bound to IP 59.78.26.210.

      standalone.bat -b=59.78.26.210
      

       

            2. My client code:

       

      String jndiName = "ejb:nms-server-ear/nms-server-ejb//SecuredRemoteSession!" + ISecuredRemoteSession.class.getName();//ear:good
      Properties p = new Properties();
      p.put("remote.connectionprovider.create.options.org.xnio.Options.SSL_ENABLED", "true");
      p.put("remote.connections", "default");
      p.put("remote.connection.default.host", 59.78.26.210);
      p.put("remote.connection.default.port", "4447");
      p.put("remote.connection.default.username", username);
      p.put("remote.connection.default.password", password);
      p.put("remote.connection.default.connect.options.org.xnio.Options.SASL_POLICY_NOANONYMOUS", "false");
      p.put("remote.connection.default.connect.options.org.xnio.Options.SASL_DISALLOWED_MECHANISMS", "JBOSS-LOCAL-USER");
      p.put("remote.connection.default.connect.options.org.xnio.Options.SASL_POLICY_NOPLAINTEXT", "false");
      p.put("remote.connection.default.connect.options.org.xnio.Options.SSL_STARTTLS", "true"); 
      EJBClientConfiguration cc = new PropertiesBasedEJBClientConfiguration(p);
      ContextSelector<EJBClientContext> selector = new ConfigBasedEJBClientContextSelector(cc);
      EJBClientContext.setSelector(selector);
      Properties props = new Properties();
      props.put(Context.URL_PKG_PREFIXES, "org.jboss.ejb.client.naming");
      InitialContext context = new InitialContext(props);
      securedRemoteSessionProxy = (ISecuredRemoteSession)context.lookup(jndiName);   
      securedRemoteSessionProxy.doSomething();
      

       

       

      Test.1

           If my client is on the same server which EJB is deployed.

           Everything is OK.

       

       

       

      Test.2

       

      If my client is on another host which is 10 miles away from the jboss 7.1.1 server (actually, this case can be reproduced if the two hosts are in same local area network but with different IP section), 

      (a) If I run above client code, then I got bellow exception:

      java.lang.RuntimeException: Operation failed with status WAITING
          at org.jboss.ejb.client.remoting.IoFutureHelper.get(IoFutureHelper.java:93)
          at org.jboss.ejb.client.remoting.ConfigBasedEJBClientContextSelector.setupEJBReceivers(ConfigBasedEJBClientContextSelector.java:121)
          at org.jboss.ejb.client.remoting.ConfigBasedEJBClientContextSelector.<init>(ConfigBasedEJBClientContextSelector.java:78)
          at com.ybxiang.nms.gui.platform.connection.ServerLink.initSecuredRemoteSession(ServerLink.java:354)
          at com.ybxiang.nms.gui.platform.connection.ServerLink.login(ServerLink.java:187)
          at com.ybxiang.nms.gui.platform.connection.ServerLink.login(ServerLink.java:178)
          at com.ybxiang.nms.gui.platform.login.LoginWizard$2.run(LoginWizard.java:121)
          at org.eclipse.jface.operation.ModalContext$ModalContextThread.run(ModalContext.java:121)"
      
      !ENTRY com.ybxiang.nms.gui.platform 0 0 2012-08-12 17:09:33.921
      !MESSAGE (Timezone is CST.) ;188; org.jboss.ejb.client: "JBoss EJB Client version 1.0.5.Final"
      
      !ENTRY com.ybxiang.nms.gui.platform 0 0 2012-08-12 17:09:43.953
      !MESSAGE (Timezone is CST.) ;188; com.ybxiang.nms.gui.platform.connection.ServerLink: "java.lang.IllegalStateException: No EJB receiver available for handling [appName:nms-server-ear,modulename:nms-server-ejb,distinctname:] combination for invocation context org.jboss.ejb.client.EJBClientInvocationContext@1650fb1"
      
      

       

       

      (b) If I debug above client code lines one by one:

      • At first I run the client till "EJBClientContext.setSelector(selector);", I will get the first exeption "Operation failed with status WAITING...".
      • Now, wait 5 seconds,
      • run the remainled code lines.
      • The second exception "java.lang.IllegalStateException: No EJB receiver available for handling..." disappeared! Everything works well!

       

       

      I tryied (a) and (b) many times and always get the same result.

       

      I searched the jboss forum and other forums, nothing useful is found.

       

      Would you tell me why?

        • 1. Re: Help! Operation failed with status WAITING + No EJB receiver available for handling
          ybxiang.china

          Now, I have migrated my application to jboss-as-7.2.0.Alpha1( jbossas-jboss-as-7.1.2.Final-531-gc2df112.zip), the result is same.

           

          Again, if server and client are on the same host, everything will be ok.

          My standalone.xml is attached.


          Have you tested real remote ejb invocation?

          Is there anybody using jboss 7.1.1 as product environment?

          • 2. Re: Help! Operation failed with status WAITING + No EJB receiver available for handling
            ybxiang.china

            Although I like the new OSGi framework, I think I had better to return back to old jboss-5.1.0.GA.

             

            • 3. Re: Help! Operation failed with status WAITING + No EJB receiver available for handling
              ochaloup

              Hi,

              that's quite strange. I suppose that the real remote ejb invocation works fine. But it's true that there are still few issues about remoting in cluster - e.g. https://issues.jboss.org/browse/JBPAPP-9349

              Did you try not to use  SSL - does it work then? And in order to check a problem from mentioned jira - maybe silly - could you try not to use SASL_DISALLOWED_MECHANISMS attribute in properties?

              • 4. Re: Help! Operation failed with status WAITING + No EJB receiver available for handling
                ybxiang.china

                Test.1 ping and telnet my server with bellow command:

                C:\Documents and Settings\yingbinx>ping  59.78.26.210

                 

                Pinging 59.78.26.210 with 32 bytes of data:

                 

                Reply from 59.78.26.210: bytes=32 time=43ms TTL=111

                Reply from 59.78.26.210: bytes=32 time=40ms TTL=111

                Reply from 59.78.26.210: bytes=32 time=45ms TTL=111

                Reply from 59.78.26.210: bytes=32 time=41ms TTL=111

                 

                Ping statistics for 59.78.26.210:

                    Packets: Sent = 4, Received = 4, Lost = 0 (0% loss),

                Approximate round trip times in milli-seconds:

                    Minimum = 40ms, Maximum = 45ms, Average = 42ms

                 

                C:\Documents and Settings\yingbinx> telnet 59.78.26.210 4447

                About 3 seconds later, the result is:

                               telnet.server.JPG

                 

                I think the telnet works well.

                • 5. Re: Help! Operation failed with status WAITING + No EJB receiver available for handling
                  ybxiang.china

                  Test.2 Without SSL

                   

                  (a) change my ApplicationRealm in standalone.xml from


                  <security-realm name="ApplicationRealm">

                  <server-identities>

                  <ssl>

                  <keystore path="server.keystore" relative-to="jboss.server.config.dir" password="ybxiang_keystore_password"/>

                  </ssl>

                  </server-identities>

                  <authentication>

                  <jaas name="nms-jaas-security-domain"/>

                  </authentication>

                  </security-realm>

                   

                  to

                   


                  <security-realm name="ApplicationRealm">

                  <authentication>

                  <jaas name="nms-jaas-security-domain"/>

                  </authentication>

                  </security-realm>

                   

                  Now restart the jboss server.

                   

                  Bellow code line

                  ContextSelector<EJBClientContext> selector = new ConfigBasedEJBClientContextSelector(cc);

                  will cause an exception:

                  !MESSAGE (Timezone is CST.) ;5668; org.jboss.ejb.client.remoting.ConfigBasedEJBClientContextSelector: "Could not register a EJB receiver for connection to 59.78.26.210:4447:

                  java.lang.RuntimeException: Operation failed with status WAITING

                      at org.jboss.ejb.client.remoting.IoFutureHelper.get(IoFutureHelper.java:93)

                      at org.jboss.ejb.client.remoting.ConfigBasedEJBClientContextSelector.setupEJBReceivers(ConfigBasedEJBClientContextSelector.java:119)

                      at org.jboss.ejb.client.remoting.ConfigBasedEJBClientContextSelector.<init>(ConfigBasedEJBClientContextSelector.java:76)

                      at com.ybxiang.nms.gui.platform.connection.ServerLink.initSecuredRemoteSession(ServerLink.java:354)

                      at com.ybxiang.nms.gui.platform.connection.ServerLink.login(ServerLink.java:187)

                      at com.ybxiang.nms.gui.platform.connection.ServerLink.login(ServerLink.java:178)

                      at com.ybxiang.nms.gui.platform.login.LoginWizard$2.run(LoginWizard.java:121)

                      at org.eclipse.jface.operation.ModalContext$ModalContextThread.run(ModalContext.java:121)"

                   

                   

                  And later code line

                   

                  securedRemoteSessionProxy.doSomething();

                   

                  will cause another exception:

                  java.lang.IllegalStateException: EJBCLIENT000025: No EJB receiver available for handling [appName:nms-server-ear, moduleName:nms-server-ejb, distinctName:] combination for invocation context org.jboss.ejb.client.EJBClientInvocationContext@135efa8"

                   

                   

                   

                  Now, remove

                  p.put("remote.connection.default.connect.options.org.xnio.Options.SASL_DISALLOWED_MECHANISMS", "JBOSS-LOCAL-USER");

                  The result is same: two exceptions are thrown.

                  • 6. Re: Help! Operation failed with status WAITING + No EJB receiver available for handling
                    ybxiang.china

                    In all of above cases, after client is cloed, the server print log:

                    10:01:48,396 INFO  [org.jboss.as] (Controller Boot Thread) JBAS015874: JBoss AS 7.2.0.Alpha1-SNAPSHOT "Steropes" started in 9031ms - Started 393 of 477 services (82 services are passive or on-demand)

                    10:02:55,131 ERROR [org.jboss.remoting.remote.connection] (Remoting "asbams04" read-1) JBREM000200: Remote connection failed: java.io.IOException: Connection reset by peer

                    10:02:55,144 ERROR [org.jboss.remoting.remote.connection] (Remoting "asbams04" read-1) JBREM000200: Remote connection failed: java.io.IOException: Connection reset by peer

                     

                    It means that, the client connected to the server correctly, but EJB invocation just fails.

                    • 7. Re: Help! Operation failed with status WAITING + No EJB receiver available for handling
                      ybxiang.china

                      Now, Let's debug ConfigBasedEJBClientContextSelector.class.

                       

                      code lines in setupEJBReceivers() is executed one by one.

                      (a) Before "final Connection connection = IoFutureHelper.get(futureConnection, connectionConfiguration.getConnectionTimeout(), TimeUnit.MILLISECONDS);" is excecuted, Let's wait 10 seconds.

                           This time, "java.lang.RuntimeException: Operation failed with status WAITING" disappeared!

                       

                      (b) If "final Connection connection = IoFutureHelper.get(futureConnection, connectionConfiguration.getConnectionTimeout(), TimeUnit.MILLISECONDS);" is excecuted immediately,

                             "java.lang.RuntimeException: Operation failed with status WAITING" is thrown!

                       

                      ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

                      Is there a property to make above code line wait automatically???

                      • 8. Re: Help! Operation failed with status WAITING + No EJB receiver available for handling
                        ybxiang.china

                        ....

                        • 9. Re: Help! Operation failed with status WAITING + No EJB receiver available for handling
                          ybxiang.china

                          Hi Ondřej Chaloupka,

                                   Thank you for your reply.

                           

                                    I think there is something wrong with xnio during public network accessing.

                                    The ping tells us: the remote server's response is fast (42ms).

                           

                           

                                    This issue cost me too much time, but I have to find a solutioin because it just likes a mouse shit in my caldron.

                          • 10. Re: Help! Operation failed with status WAITING + No EJB receiver available for handling
                            ybxiang.china

                            Now, the jboss 7 server is running in local area network:

                            layout.JPG

                             

                            Debug Result is same:

                             

                            (a) If EJB client suspends before bellow code line and wait 10 seconds:

                            final Connection connection = IoFutureHelper.get(futureConnection, connectionConfiguration.getConnectionTimeout(), TimeUnit.MILLISECONDS);
                            

                            Then run the rest code lines.

                            Everything is ok.

                             

                            (b) If EJB client execute above code line without wait, "Operation failed with status WAITING" will be thrown.

                             

                             

                            Note, 135.251.246.160 and 135.251.27.26 are in same local area with different IP sections. And the network status is excellent:

                            C:\Documents and Settings\yingbinx>ping 135.251.246.160
                            
                            Pinging 135.251.246.160 with 32 bytes of data:
                            
                            Reply from 135.251.246.160: bytes=32 time=1ms TTL=57
                            Reply from 135.251.246.160: bytes=32 time=1ms TTL=57
                            Reply from 135.251.246.160: bytes=32 time=1ms TTL=57
                            Reply from 135.251.246.160: bytes=32 time=1ms TTL=57
                            
                            Ping statistics for 135.251.246.160:
                                Packets: Sent = 4, Received = 4, Lost = 0 (0% loss),
                            Approximate round trip times in milli-seconds:
                                Minimum = 1ms, Maximum = 1ms, Average = 1ms
                            
                            C:\Documents and Settings\yingbinx>
                            
                            • 11. Re: Help! Operation failed with status WAITING + No EJB receiver available for handling
                              ybxiang.china

                              I found the timeout setting by debugging:

                              p.put("remote.connection.default.connect.timeout", "15000");

                               

                              But why JBoss spend so much time to connect to my server?

                              1 of 1 people found this helpful
                              • 12. Re: Help! Operation failed with status WAITING + No EJB receiver available for handling
                                ochaloup

                                Hi,

                                I would suppose that the bottleneck is in some other place than in remoting. How exactly do you define nms-jaas-security-domain?

                                E.g. whether you would use database there or any other authentication resource then I would suggest to check the time needed to connect to that resource.

                                • 13. Re: Help! Operation failed with status WAITING + No EJB receiver available for handling
                                  ybxiang.china

                                  I used local DB to authenticate user. Namely, the AS 7 and DB are on the same host.

                                   

                                  I will debug XNIO in near future and report the result here.

                                   

                                  Thanks!