4 Replies Latest reply on Aug 24, 2009 12:52 PM by brian.stansberry

    Custom InvokerInterceptor

    bdinnerv

      Hi All,

      After much reading up, it appears that I need to write a custom InvokerInterceptor to achieve clustering of Session beans in JBoss (I am running on 4.2.3) and I have a couple of questions before I venture down this road (although it is not a lot of code when looking at it)

      I have an application that runs entirely in the container, I no external client, the system is implemented in Sessions beans, Entity Beans, JMX MBeans and MDB Beans (it also has a minimal web client built on tapestry 5 for mainly management and configuration tasks), all in EJB3, and is a fairly highly stressed system which has become cpu bound, hence we are looking at gaining a performance increase by clustering.

      As the application is fully contained within the container, we have the scenario where clients and session beans run in the same JVM (MDB's and JMX beans act as clients as well as session beans dependent on other session beans) so I predict a large performance increase by being able to load balance our session bean calls between multiple servers, however the limitation of the "optimization" where by the InvokerInterceptor routes calls to local instances only is a big limiting factor and does not allow us to distribute the load, hence looking at implementing a custom InvokerInterceptor.

      1. When overriding hasLocalTarget and simply returning false, will the round robin process still send some requests to the "local" session bean instance as well as the "remote" instances in the cluster, or will it send ALL requests to the remote instances / nodes for processing and ignore the local instance all together?

      2. If the answer to 1 is "Yes, it will send all requests to remote nodes" under the scenario of simply returning false, is the InvokerInterceptor "statefull". What i mean by this is would something like the following work:

      public class CustomInvokerInterceptor extends InvokerInterceptor {
      
       private boolean _local = false;
      
       public boolean hasLocalTarget() {
       if (_local) {
       _local = false;
       return true;
       } else {
       _local = true;
       return false;
       }
       }
      }
      


      such that between calls to hasLocalTarget, the state of _local will be maintained, or will the be a fresh instantiation between invocations and thus always return false? If this is not the case and all calls end up going remote, we would essentially have an under utilised node on the cluster that simply acts as a client to the other nodes (I hate wasted resources :) )

      3. Based on the fact that there are a number of threads regarding this issue over some time now, is it still an issue? IE, in 4.2.3 or even 5, do we still need to implement a custom InvokerInterceptor to "de-optimize" JBoss so we can optimize out application, or has it now been made configurable such that we can change a config setting to tell the container that we want a certain behavior or some other way to achieve RoundRobin load balancing? I understand the fact that it is more resource intensive to route a call to a remote node when it could be serviced locally, but that rule doesn't cater very well for the scenario where the local node is 100% CPU stressed and an application has become CPU bound.

      Cheers,

      Ben

        • 1. Re: Custom InvokerInterceptor
          bdinnerv

          Hi All,

          Further to my last post, can anyone provide some insight into which parameters need to be changed when utilising a custom InvokerInterceptor?

          The easy way would be to take the shotgun approach and replace all references to org.jboss.invocation.InvocationInterceptor in standardjboss.xml but I am guessing that this is a bit over the top and that only certain references to it need to be changed to get SLSB clustering to work.

          Looking at the config, I would guess that only the clustered-entity-unified-invoker section in the config needs to be changed, but I am not 100% sure as there are quite a few other references to InvokerInterceptor throughout the file.

          Any input from people who have done this config would be appreciated.

          Cheers,

          Ben

          • 2. Re: Custom InvokerInterceptor
            bdinnerv

            Well, after a day of testing and changes, I cant for the life of me get my session beans to cluster.

            I have implemented a custom InvokerInterceptor, placed my jar in server/all/lib, changed all references to org.jboss.invocation.InvokerInterceptor with my custom one, even tried removing all references to the MarshellingInterceptor (not the full class name I know, but it slips my mind at the moment) but the interceptor does not seem to get called. I have some sysout printing in my implementation of hasLocalTarget and also in the default no args constructor and am not getting any messages logged nor am I getting round robin load distribution for my session beans. Is there somewhere else other that standardboss.xml that I need to make the change to get the my custom Invoker used?

            I am using JBoss 4.2.3 with the all configuration on jdk 1.5.16 / linux and EJB3, the session beans are annotated as follows:

            @Stateless
            @Clustered(loadBalancePolicy = RoundRobin.class)
            @Remote(ICampaignService.class)


            And my client uses the following:

            try {
            Properties env = new Properties();
            env.setProperty(Context.INITIAL_CONTEXT_FACTORY, "org.jnp.interfaces.NamingContextFactory");
            env.setProperty(Context.URL_PKG_PREFIXES,"jboss.naming:org.jnp.interfaces");
            env.setProperty("jnp.partitionName", "DefaultPartition");
            env.setProperty(Context.PROVIDER_URL, "localhost:1100");

            _initialContext = new InitialContext(env);

            } catch (NamingException e) {


            fooo bar . . .

            _campaignService = (ICampaignService) lookup("teleblast/CampaignServiceBean/remote");

            
            Cheers,
            
            Ben


            • 3. Re: Custom InvokerInterceptor
              chrismeadows

              I'm bumping this as I would be very interested in the response from JBoss themselves. The original questions raised by Ben are highly relevant to the last two JBoss based projects I've worked on, and I'm now hitting it on a 3rd.

              Would it be possible for JBoss to give a statement on what is the recommended approach to load balancing calls when client and session bean are both inside the container, and that container is clustered?

              I found a response to this thread on techienuggets http://www.techienuggets.com/Comments?tx=52347 as follows:


              think you'll want to modify the EJB stack defined in server/all/deploy/ejb3-interceptors-aop.xml.
              You can add your custom interceptor or simply remove/comment-out the "IsLocal" interceptor.

              For example, if you want to make the change for clustered stateless session beans the applicable section could look like this:
              <stack name="ClusteredStatelessSessionClientInterceptors">
              <!-- interceptor-ref name="org.jboss.ejb3.remoting.ClusteredIsLocalInterceptor"/ -->
              <interceptor-ref name="org.jboss.aspects.security.SecurityClientInterceptor"/>
              <interceptor-ref name="org.jboss.aspects.tx.ClientTxPropagationInterceptor"/>
              <interceptor-ref name="org.jboss.aspects.remoting.ClusterChooserInterceptor"/>
              <interceptor-ref name="org.jboss.aspects.remoting.InvokeRemoteInterceptor"/>
              </stack>

              Notice the commented out ClusteredIsLocalInterceptor.

              I've been trying to do the exact same thing as you (i.e. cluster ejbs called from within the same JBoss 4.2.3 server). And I just figured this out today. I think it does what I want, but haven't fully "exercised" it in any way. Let me know if this helps.

              -Randy


              Could JBoss verify the above as a valid or even recommended approach?

              Regards,

              Chris


              • 4. Re: Custom InvokerInterceptor
                brian.stansberry

                Does your application initiate transactions on the client side?

                If so, removing the ClusteredIsLocalInterceptor can lead to problems described at https://jira.jboss.org/jira/browse/EJBTHREE-1005. TBH I believe replacing a ClusteredIsLocalInterceptor with something custom would face the same problem.