1 2 Previous Next 18 Replies Latest reply on Oct 19, 2012 4:24 AM by jaikiran

    Calls against multiple servers from a single client

    thammoud

      Hello,

       

      We have a need to invoke remote EJB calls against multiple servers. The invoker is a scheduling engine and the targets are client specific instances. We configure an EJB Client Selector per remote host. Looking closely at EJBClientContext.setSelector(selector), this method is using a static variable called (SELECTOR) and not a thread local. i.e You can not use this from multiple threads simultaneously without turning the invoker into a single threaded process. Undersirable for the use case at hand. Is there any workaround for this? Thank you for all your help in this. We are using 7.1.1.

        • 1. Re: Calls against multiple servers from a single client
          wdfink

          You might implement your own Selector and implement ContextSelector<EJBClientContext>.

          Here you can store as many selectors as you want in a Map and select the specific one Thread specific.

           

          I'm working at an example for such use cases and try to implement a quickstart for this, see #78.

          • 2. Re: Calls against multiple servers from a single client
            jaikiran

            To be more clear, you don't have to keep changing the selector. Just set one selector and let it hand out the appropriate EJBClientContext depending on the specific client criteria that you are talking about.

            • 3. Re: Calls against multiple servers from a single client
              thammoud

              Jaikiran,

               

              I can not see how handing out the appropriate EJBClientContext solves this. I believe that EJBClientContext is the cause of this problem to the static usage in SELECTOR.

               

              This is what we are currently doing using the JBOSS ejb client API to get around the multiple thread trashing SELECTOR in EJBClientContext.

               

              For each server target (Unique by credentials and host name/Port), we are creating a new ConfigBasedEJBClientContextSelector which in turn creates an EJBClientContext. When we invoke an ejb method, some invoker code in JBOSS will call requireCurrent on the EJBClientContext which will trash any SELECTOR currently set if being invoked from a separate thread. Our code is wrapping the proxy returned the InitialContext.lookup to globally lock before preceeding with an ejb call. This naturally will cause the server to become a client (Really, a very expensive server) to become single threaded.

               

              static class RemoteInvocationHandler implements InvocationHandler {

                      static  Object                            globalLock = new Object();

                      private ContextSelector<EJBClientContext> selector;

                      private Object                            delegate;

               

                      public RemoteInvocationHandler(ContextSelector<EJBClientContext> selector, Object delegate) {

                          this.selector = selector;

                          this.delegate = delegate;

                      }

               

                      public Object invoke(Object proxy, Method method, Object[] args) throws Throwable {

                          synchronized(globalLock) {  // EVIL Code

                              EJBClientContext.setSelector(selector);

                              Object result = method.invoke(delegate, args);

                              return result;

                          }

                      }

              }

               

              Thank you for your prompt reply.

              • 4. Re: Calls against multiple servers from a single client
                jaikiran

                You don't have to do that. Use just 1 single selector and set it once during the lifetime of that application and let that selector then do the necessary EJBClientContext creation and handing out. I can give you an pseudocode example on how to do this, but what criteria do you use to identify a client with a destination server? I mean, I need more details about this part:

                 

                Our code is wrapping the proxy returned the InitialContext.lookup to globally lock before preceeding with an ejb call

                • 5. Re: Calls against multiple servers from a single client
                  thammoud

                  We are using hostname/username/ to create unique ConfigBasedEJBClientContextSelector. I will appreciate the pseudo code

                  • 6. Re: Calls against multiple servers from a single client
                    wdfink

                    Do you use different host/user for each thread?

                    I would use a ThreadLocal approach instead of synchronize, it might be a bottleneck.

                    • 7. Re: Calls against multiple servers from a single client
                      thammoud

                      Hello Wolf,

                       

                      There is no (nor there should be any) affinity between a thread and an EJB proxy. The user can execute against an EJB proxy from ANY thread and not just the one that created it. It is at invocation time, the proper EJB context (Stored in the proxy code above) needs to be setup so that the EJBInvoker code from JBOSS sees the "right" context for THAT thread. Today, EJBContext has SELECTOR as a static so all threads will trample on each other when the code calls requireCurrent from multiple threads. i.e ThreadLocal should be in EJBContext and it should be set during the actual invocation process.

                       

                      I hope I answered your question. Thank you for all your help in this.

                      • 8. Re: Calls against multiple servers from a single client
                        thammoud

                        If you look at the bolded code in EJBInvocationHandler:

                         

                        Object doInvoke(final T proxy, final Method method, final Object[] args) throws Throwable {

                                final MethodHandler handler = clientSideMethods.get(new MethodKey(method));

                                if (handler != null && handler.canHandleInvocation(this, proxy, method, args)) {

                                    return handler.invoke(this, proxy, method, args);

                                }

                                final EJBClientContext context = EJBClientContext.requireCurrent();

                                return doInvoke(this, async, proxy, method, args, context);

                            }

                         

                        EJBClientContext is using a non-thread local static. I don't know how to make that thread safe with the above suggestions. Looking forward to hearing from you.

                        • 9. Re: Calls against multiple servers from a single client
                          wdfink

                          Hi Tarek,

                          I suppose you have knowledge with former AS versions.

                          Throw the knowlege about ejb-clients away, it's complete different.

                           

                          Since AS7 the Proxy (stub) does not contain any information about the server, it is only a implementation of the Interface that is created by reflection during runtime lookup().

                          The Proxy knows only the name "ejb:app/module/distinct/bean!view".

                          If you execute a method the ejb-client library decide how to go, here is the knowlege what server exists set by the initial creation or the jboss-ejb-client.properties.

                          The selector handle the phys. connections to the servers and I'm not sure what happen in your case setting the selectors for each call, regardless that such sync-block might be a concurrency bottleneck.

                           

                          Le me finish my example I mentioned above and it might will give you an idea.

                          • 10. Re: Calls against multiple servers from a single client
                            thammoud

                            Hi Wolf,

                             

                            The sync call (Labeld Evil) is used in the example above of what a multithreaded client will have to do and avoid the selector trashing and function properly albeit serially. It is not used for to demonstrate "best practices". Out multi-server test cases broke immediately in AS7 thus the hack to show the extent of this problem. I believe that the EJBClientContext with a static selector inhibits proper implementation of a multithreaded client in AS7.

                             

                            Tarek

                            • 11. Re: Calls against multiple servers from a single client
                              wdfink

                              Tarek Hammoud wrote:

                               

                              I believe that the EJBClientContext with a static selector inhibits proper implementation of a multithreaded client in AS7.

                              That will not be true, it is possible to have multithread clients with AS7, but as I mentionend it is completly different in implmentation and use.

                               

                              So what is your exact requirement. Do you

                              - deploy the same applcation at different servers with different credentials and need to decide which one is called?

                              - deploy different EJBs spread on servers and want to call it? Simple, add all servers to the ejb-client.properties and call the different names

                               

                               

                              Wolf

                              • 12. Re: Calls against multiple servers from a single client
                                thammoud

                                Hi Wolf,

                                 

                                 

                                Deploy the same applcation at different servers with different credentials and need to decide which one is called?

                                 

                                That is one of the use cases.

                                 

                                That will not be true, it is possible to have multithread clients with AS7, but as I mentionend it is completly different in implmentation and use.

                                 

                                This why I opened this ticket. While the framework clearly allows for multiple targets and there is nothing to inhibit a multithreaded client, I am pointing out a race condition in the implementation with respect to static in EJBContext. Again, looking at the EJBInvocationHandler, you can clearly see that it is indirectly manipulating a static member by calling the requireCurrent(). Two threads in the EJBInvocationHandler doInvoke code can end up pointing to the wrong SELECTOR.

                                • 13. Re: Calls against multiple servers from a single client
                                  rodakr

                                  Terek is right

                                   

                                  Try Simple example.

                                   

                                  One Stand alone Client starting 2 Threads. Each Thread calls an EJB on different JBoss 7x Server. Once Static EJBClientContext.setSelector is called seconde times, it brakes one of both Threads...

                                  no EJBRecever avaible exception... 25 is the result. I don't see any sinfull solution as long as global static  EJBClientContext.setSelector  is needed!!!

                                   

                                  This happens with both:  jboss-remote-naming and jboss-ejb-client with AS7.1.2

                                  • 14. Re: Calls against multiple servers from a single client
                                    wdfink

                                    set the selector dynamical is not recommended.

                                    You can add both server to the same client context and, if the application is the same. use the distinct approach to separate the applications.

                                    Or you might implement a selector that is able to handle the invocations in a thread save way.

                                    1 2 Previous Next