6 Replies Latest reply on Mar 29, 2014 7:56 AM by zhouyaguo

    Failover does not work properly in remote-naming when lookup remote ejb

    zhouyaguo

      Hi,

           I'm using JBoss AS 7.2.0.Final.

           I deployed myejb.jar in two standalone AS7 instance and i write a client testcase as following:

       

      public class TestRemoteNaming {
          public static void main(String[] args) {
              int count = 0;
              while (true) {
                  try {
                      Properties props = new Properties();
                      props.put(Context.INITIAL_CONTEXT_FACTORY, "org.jboss.naming.remote.client.InitialContextFactory");
                      props.put(Context.PROVIDER_URL, "remote://172.17.252.66:31003,remote://172.17.252.63:31003");
                      props.put("jboss.naming.client.ejb.context", true);
                      InitialContext ctx = new InitialContext(props);
                      Object ob = ctx.lookup("cuam-ear/param-core/ParaSyncService!com.unionpay.param.core.sync.ParaSyncService");
                  } catch (Exception e) {
                      e.printStackTrace();
                      count++;
                      System.out.println("count==" + count);
                  }
              }
          }
      }
      

       

       

           What i expect is no exception occur, I think remote-naming will do failover inside lookup method, but console prints :

      count==1
        at org.jboss.as.naming.ServiceBasedNamingStore.lookup(ServiceBasedNamingStore.java:103)
        at org.jboss.as.naming.NamingContext.lookup(NamingContext.java:197)
        at org.jboss.as.naming.NamingContext.lookup(NamingContext.java:174)
        at org.jboss.naming.remote.protocol.v1.Protocol$1.handleServerMessage(Protocol.java:127)
        at org.jboss.naming.remote.protocol.v1.RemoteNamingServerV1$MessageReciever$1.run(RemoteNamingServerV1.java:73)
        at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:897)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:919)
        at java.lang.Thread.run(Thread.java:736)
      javax.naming.NameNotFoundException: cuam-ear/param-core/ParaSyncService!com.unionpay.param.core.sync.ParaSyncService -- service jboss.naming.context.java.jboss.exported.cuam-ear.param-core."ParaSyncService!com.unionpay.param.core.sync.ParaSyncService"
        at org.jboss.as.naming.ServiceBasedNamingStore.lookup(ServiceBasedNamingStore.java:103)
        at org.jboss.as.naming.NamingContext.lookup(NamingContext.java:197)
        at org.jboss.as.naming.NamingContext.lookup(NamingContext.java:174)
        at org.jboss.naming.remote.protocol.v1.Protocol$1.handleServerMessage(Protocol.java:127)
        at org.jboss.naming.remote.protocol.v1.RemoteNamingServerV1$MessageReciever$1.run(RemoteNamingServerV1.java:73)
        at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:897)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:919)
        at java.lang.Thread.run(Thread.java:736)
      count==2
      .....
      javax.naming.NameNotFoundException: cuam-ear/param-core/ParaSyncService!com.unionpay.param.core.sync.ParaSyncService -- service jboss.naming.context.java.jboss.exported.cuam-ear.param-core."ParaSyncService!com.unionpay.param.core.sync.ParaSyncService"
        at org.jboss.as.naming.ServiceBasedNamingStore.lookup(ServiceBasedNamingStore.java:103)
        at org.jboss.as.naming.NamingContext.lookup(NamingContext.java:197)
        at org.jboss.as.naming.NamingContext.lookup(NamingContext.java:174)
        at org.jboss.naming.remote.protocol.v1.Protocol$1.handleServerMessage(Protocol.java:127)
        at org.jboss.naming.remote.protocol.v1.RemoteNamingServerV1$MessageReciever$1.run(RemoteNamingServerV1.java:73)
        at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:897)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:919)
        at java.lang.Thread.run(Thread.java:736)
      count==24
      javax.naming.NameNotFoundException: cuam-ear/param-core/ParaSyncService!com.unionpay.param.core.sync.ParaSyncService -- service jboss.naming.context.java.jboss.exported.cuam-ear.param-core."ParaSyncService!com.unionpay.param.core.sync.ParaSyncService"
        at org.jboss.as.naming.ServiceBasedNamingStore.lookup(ServiceBasedNamingStore.java:103)
        at org.jboss.as.naming.NamingContext.lookup(NamingContext.java:197)
        at org.jboss.as.naming.NamingContext.lookup(NamingContext.java:174)
        at org.jboss.naming.remote.protocol.v1.Protocol$1.handleServerMessage(Protocol.java:127)
        at org.jboss.naming.remote.protocol.v1.RemoteNamingServerV1$MessageReciever$1.run(RemoteNamingServerV1.java:73)
        at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:897)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:919)
        at java.lang.Thread.run(Thread.java:736)
      count==25
      javax.naming.NamingException: Unable to invoke lookup, status=WAITING
      count==26
        at org.jboss.naming.remote.protocol.v1.Protocol$1.execute(Protocol.java:98)
        at org.jboss.naming.remote.protocol.v1.RemoteNamingStoreV1.lookup(RemoteNamingStoreV1.java:78)
        at org.jboss.naming.remote.client.HaRemoteNamingStore$1.operation(HaRemoteNamingStore.java:227)
        at org.jboss.naming.remote.client.HaRemoteNamingStore.namingOperation(HaRemoteNamingStore.java:114)
        at org.jboss.naming.remote.client.HaRemoteNamingStore.lookup(HaRemoteNamingStore.java:223)
        at org.jboss.naming.remote.client.RemoteContext.lookup(RemoteContext.java:79)
        at org.jboss.naming.remote.client.RemoteContext.lookup(RemoteContext.java:83)
        at javax.naming.InitialContext.lookup(InitialContext.java:392)
        at com.zyg.test.TestRemoteNaming.main(TestRemoteNaming.java:30)
      

           Why i got exception so many times?

           Any advice will be appreciated!

       

      Regards.

        • 1. Re: Failover does not work properly in remote-naming when lookup remote ejb
          wdfink

          First you should not create the context every time,  second you need to close it if you finish with it.

          In that case I think it would work better.

           

          But the remote-naming approach is not recommended to use for ejb invocations, you should use the ejb-client, see here

          • 2. Re: Failover does not work properly in remote-naming when lookup remote ejb
            zhouyaguo

            Thanks!

            After change my code like following, i still got many exceptions.

            public class TestRemoteNaming1 {

             

                public static void main(String[] args) {

                    int count = 0;

                    try {

                        Properties props = new Properties();

                        props.put(Context.INITIAL_CONTEXT_FACTORY, "org.jboss.naming.remote.client.InitialContextFactory");

                        props.put(Context.PROVIDER_URL, "remote://172.17.252.66:31003,remote://172.17.252.63:31003");

                        props.put("jboss.naming.client.ejb.context", true);

                        InitialContext ctx = new InitialContext(props);

             

                       while (true) {

                            try {

                                Object ob = ctx.lookup("cuam-ear/param-core/ParaSyncService!com.unionpay.param.core.sync.ParaSyncService");

                                System.out.println(ob);

                            } catch (Exception e) {

                                e.printStackTrace();

                                count++;

                                System.out.println("count==" + count);

                            }

                        }

             

                    } catch (Exception e) {

                        e.printStackTrace();

                    }

                }

            }

            output:

            Proxy for remote EJB StatelessEJBLocator{appName='cuam-ear', moduleName='param-core', distinctName='', beanName='ParaSyncService', view='interface com.unionpay.param.core.sync.ParaSyncService'}

            Proxy for remote EJB StatelessEJBLocator{appName='cuam-ear', moduleName='param-core', distinctName='', beanName='ParaSyncService', view='interface com.unionpay.param.core.sync.ParaSyncService'}

            Proxy for remote EJB StatelessEJBLocator{appName='cuam-ear', moduleName='param-core', distinctName='', beanName='ParaSyncService', view='interface com.unionpay.param.core.sync.ParaSyncService'}

            javax.naming.NameNotFoundException: cuam-ear/param-core/ParaSyncService!com.unionpay.param.core.sync.ParaSyncService -- service jboss.naming.context.java.jboss.exported.cuam-ear.param-core."ParaSyncService!com.unionpay.param.core.sync.ParaSyncService"

            count==1

              at org.jboss.as.naming.ServiceBasedNamingStore.lookup(ServiceBasedNamingStore.java:103)

              at org.jboss.as.naming.NamingContext.lookup(NamingContext.java:197)

              at org.jboss.as.naming.NamingContext.lookup(NamingContext.java:174)

              at org.jboss.naming.remote.protocol.v1.Protocol$1.handleServerMessage(Protocol.java:127)

              at org.jboss.naming.remote.protocol.v1.RemoteNamingServerV1$MessageReciever$1.run(RemoteNamingServerV1.java:73)

              at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:897)

              at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:919)

              at java.lang.Thread.run(Thread.java:736)

             

             

            ........

             

             

            javax.naming.NameNotFoundException: cuam-ear/param-core/ParaSyncService!com.unionpay.param.core.sync.ParaSyncService -- service jboss.naming.context.java.jboss.exported.cuam-ear.param-core."ParaSyncService!com.unionpay.param.core.sync.ParaSyncService"

              at org.jboss.as.naming.ServiceBasedNamingStore.lookup(ServiceBasedNamingStore.java:103)

              at org.jboss.as.naming.NamingContext.lookup(NamingContext.java:197)

              at org.jboss.as.naming.NamingContext.lookup(NamingContext.java:174)

              at org.jboss.naming.remote.protocol.v1.Protocol$1.handleServerMessage(Protocol.java:127)

              at org.jboss.naming.remote.protocol.v1.RemoteNamingServerV1$MessageReciever$1.run(RemoteNamingServerV1.java:73)

              at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:897)

              at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:919)

              at java.lang.Thread.run(Thread.java:736)

            count==34

            javax.naming.NamingException: Unable to invoke lookup, status=WAITING

            count==35

              at org.jboss.naming.remote.protocol.v1.Protocol$1.execute(Protocol.java:98)

              at org.jboss.naming.remote.protocol.v1.RemoteNamingStoreV1.lookup(RemoteNamingStoreV1.java:78)

              at org.jboss.naming.remote.client.HaRemoteNamingStore$1.operation(HaRemoteNamingStore.java:227)

              at org.jboss.naming.remote.client.HaRemoteNamingStore.namingOperation(HaRemoteNamingStore.java:114)

              at org.jboss.naming.remote.client.HaRemoteNamingStore.lookup(HaRemoteNamingStore.java:223)

              at org.jboss.naming.remote.client.RemoteContext.lookup(RemoteContext.java:79)

              at org.jboss.naming.remote.client.RemoteContext.lookup(RemoteContext.java:83)

              at javax.naming.InitialContext.lookup(InitialContext.java:392)

              at com.zyg.test.TestRemoteNaming1.main(TestRemoteNaming1.java:23)

            Proxy for remote EJB StatelessEJBLocator{appName='cuam-ear', moduleName='param-core', distinctName='', beanName='ParaSyncService', view='interface com.unionpay.param.core.sync.ParaSyncService'}

            Proxy for remote EJB StatelessEJBLocator{appName='cuam-ear', moduleName='param-core', distinctName='', beanName='ParaSyncService', view='interface com.unionpay.param.core.sync.ParaSyncService'}

            Proxy for remote EJB StatelessEJBLocator{appName='cuam-ear', moduleName='param-core', distinctName='', beanName='ParaSyncService', view='interface com.unionpay.param.core.sync.ParaSyncService'}

            Proxy for remote EJB StatelessEJBLocator{appName='cuam-ear', moduleName='param-core', distinctName='', beanName='ParaSyncService', view='interface com.unionpay.param.core.sync.ParaSyncService'}

            Proxy for remote EJB StatelessEJBLocator{appName='cuam-ear', moduleName='param-core', distinctName='', beanName='ParaSyncService', view='interface com.unionpay.param.core.sync.ParaSyncService'}

            It looks like remote-naming does not set first instance to bad when first exception occur. Any ideas?

            • 3. Re: Failover does not work properly in remote-naming when lookup remote ejb
              wdfink

              You should reuse the proxy and did not lookup it every time.

              But as I mentioned before you should use the ejb-client approach.

              • 4. Re: Failover does not work properly in remote-naming when lookup remote ejb
                zhouyaguo

                Thank you very much!

                According to your advice, i change my code to : (use ejb-client approach and reuse the proxy)

                public class TestClient {

                 

                    public static void main(String[] args) {

                        final String EJB_ADD_JNDI_NAME = "ejb:/demo-remote-ejb-ha/AddService!com.unionpay.demo.as7.ejb.remote.ha.AddRemote";

                 

                        Context context;

                        int count = 0;

                        try {

                            Properties props = new Properties();

                            props.put("remote.connections", "default,another");

                            props.put("remote.connectionprovider.create.options.org.xnio.Options.SSL_ENABLED", "false");

                 

                            props.put("remote.connection.default.host", "172.17.250.124");

                            props.put("remote.connection.default.port", "11003");

                            props.put("remote.connection.default.connect.options.org.xnio.Options.SASL_POLICY_NOANONYMOUS", "false");

                            props.put("remote.connection.default.connect.options.org.xnio.Options.SASL_POLICY_NOPLAINTEXT", "false");

                 

                            props.put("remote.connection.another.host", "172.17.250.124");

                            props.put("remote.connection.another.port", "21003");

                            props.put("remote.connection.another.connect.options.org.xnio.Options.SASL_POLICY_NOANONYMOUS", "false");

                            props.put("remote.connection.another.connect.options.org.xnio.Options.SASL_POLICY_NOPLAINTEXT", "false");

                 

                            props.put(Context.URL_PKG_PREFIXES, "org.jboss.ejb.client.naming");

                            props.put("org.jboss.ejb.client.scoped.context", "true");

                 

                            context = new InitialContext(props);

                 

                            AddRemote service = (AddRemote) context.lookup(EJB_ADD_JNDI_NAME);

                 

                            while (true) {

                                Thread.currentThread().sleep(2000);

                 

                                try {

                                    System.out.println("remote call ejb result: " + service.add(1, 2));

                                } catch (Exception e) {

                                    e.printStackTrace();

                                    count++;

                                    System.out.println("count=" + count);

                                }

                            }

                 

                        } catch (Exception e) {

                            e.printStackTrace();

                        }

                    }

                }

                output is:

                Mar 28, 2014 5:21:56 PM org.xnio.Xnio <clinit>

                INFO: XNIO Version 3.0.7.GA

                Mar 28, 2014 5:21:56 PM org.xnio.nio.NioXnio <clinit>

                INFO: XNIO NIO Implementation Version 3.0.7.GA

                Mar 28, 2014 5:21:56 PM org.jboss.remoting3.EndpointImpl <clinit>

                INFO: JBoss Remoting version 3.2.14.GA

                Mar 28, 2014 5:21:57 PM org.jboss.ejb.client.remoting.VersionReceiver handleMessage

                INFO: EJBCLIENT000017: Received server version 1 and marshalling strategies [river]

                Mar 28, 2014 5:21:57 PM org.jboss.ejb.client.remoting.RemotingConnectionEJBReceiver associate

                INFO: EJBCLIENT000013: Successful version handshake completed for receiver context EJBReceiverContext{clientContext=org.jboss.ejb.client.EJBClientContext@554d7745, receiver=Remoting connection EJB receiver [connection=org.jboss.ejb.client.remoting.ConnectionPool$PooledConnection@50d17ec3,channel=jboss.ejb,nodename=zygpc]} on channel Channel ID e93429f3 (outbound) of Remoting connection 6b6d2702 to /172.17.250.124:11003

                Mar 28, 2014 5:21:57 PM org.jboss.ejb.client.EJBClient <clinit>

                INFO: JBoss EJB Client version 1.0.25.Final

                remote call ejb result: 3

                remote call ejb result: 3

                remote call ejb result: 3

                Mar 28, 2014 5:22:04 PM org.jboss.ejb.client.remoting.ChannelAssociation$ResponseReceiver handleEnd

                INFO: EJBCLIENT000016: Channel Channel ID e93429f3 (outbound) of Remoting connection 6b6d2702 to /172.17.250.124:11003 can no longer process messages

                java.lang.IllegalStateException: EJBCLIENT000025: No EJB receiver available for handling [appName:, moduleName:demo-remote-ejb-ha, distinctName:] combination for invocation context org.jboss.ejb.client.EJBClientInvocationContext@6602e323

                count=1

                  at org.jboss.ejb.client.EJBClientContext.requireEJBReceiver(EJBClientContext.java:735)

                  at org.jboss.ejb.client.ReceiverInterceptor.handleInvocation(ReceiverInterceptor.java:116)

                  at org.jboss.ejb.client.EJBClientInvocationContext.sendRequest(EJBClientInvocationContext.java:183)

                  at org.jboss.ejb.client.EJBInvocationHandler.sendRequestWithPossibleRetries(EJBInvocationHandler.java:253)

                  at org.jboss.ejb.client.EJBInvocationHandler.doInvoke(EJBInvocationHandler.java:198)

                  at org.jboss.ejb.client.EJBInvocationHandler.doInvoke(EJBInvocationHandler.java:181)

                  at org.jboss.ejb.client.EJBInvocationHandler.invoke(EJBInvocationHandler.java:144)

                  at com.sun.proxy.$Proxy0.add(Unknown Source)

                  at com.unionpay.demo.as7.ejb.remote.ha.client.TestClient.main(TestClient.java:49)

                java.lang.IllegalStateException: EJBCLIENT000025: No EJB receiver available for handling [appName:, moduleName:demo-remote-ejb-ha, distinctName:] combination for invocation context org.jboss.ejb.client.EJBClientInvocationContext@27b4c1d7

                  at org.jboss.ejb.client.EJBClientContext.requireEJBReceiver(EJBClientContext.java:735)

                count=2 at org.jboss.ejb.client.ReceiverInterceptor.handleInvocation(ReceiverInterceptor.java:116)

                  at org.jboss.ejb.client.EJBClientInvocationContext.sendRequest(EJBClientInvocationContext.java:183)

                  at org.jboss.ejb.client.EJBInvocationHandler.sendRequestWithPossibleRetries(EJBInvocationHandler.java:253)

                  at org.jboss.ejb.client.EJBInvocationHandler.doInvoke(EJBInvocationHandler.java:198)

                  at org.jboss.ejb.client.EJBInvocationHandler.doInvoke(EJBInvocationHandler.java:181)

                  at org.jboss.ejb.client.EJBInvocationHandler.invoke(EJBInvocationHandler.java:144)

                  at com.sun.proxy.$Proxy0.add(Unknown Source)

                  at com.unionpay.demo.as7.ejb.remote.ha.client.TestClient.main(TestClient.java:49)

                 

                 

                java.lang.IllegalStateException: EJBCLIENT000025: No EJB receiver available for handling [appName:, moduleName:demo-remote-ejb-ha, distinctName:] combination for invocation context org.jboss.ejb.client.EJBClientInvocationContext@83e96cf

                  at org.jboss.ejb.client.EJBClientContext.requireEJBReceiver(EJBClientContext.java:735)

                count=3

                  at org.jboss.ejb.client.ReceiverInterceptor.handleInvocation(ReceiverInterceptor.java:116)

                  at org.jboss.ejb.client.EJBClientInvocationContext.sendRequest(EJBClientInvocationContext.java:183)

                  at org.jboss.ejb.client.EJBInvocationHandler.sendRequestWithPossibleRetries(EJBInvocationHandler.java:253)

                  at org.jboss.ejb.client.EJBInvocationHandler.doInvoke(EJBInvocationHandler.java:198)

                  at org.jboss.ejb.client.EJBInvocationHandler.doInvoke(EJBInvocationHandler.java:181)

                  at org.jboss.ejb.client.EJBInvocationHandler.invoke(EJBInvocationHandler.java:144)

                  at com.sun.proxy.$Proxy0.add(Unknown Source)

                  at com.unionpay.demo.as7.ejb.remote.ha.client.TestClient.main(TestClient.java:49)

                Got no luck. no failover effect. ;(

                Thank you for your time!

                • 5. Re: Failover does not work properly in remote-naming when lookup remote ejb
                  wdfink

                  If you start only one server, did the test run correct with both servers?

                  What do you do if the first invocation fail? Maybe you add a sleep to be clear when a server goes down.

                  • 6. Re: Failover does not work properly in remote-naming when lookup remote ejb
                    zhouyaguo

                    Let me explain more clear.

                    I deployed an ejb jar in two as7 instances.Both of instances are running well,and then I started my client testcase.During the client while loop running,i stopped the first as7 instance, what I hope is that ejb-client can flag the 1st instance bad and invoke the 2nd instance automatically