2 Replies Latest reply on May 2, 2010 1:56 PM by dbschofield

    How to tune InVMAcceptor?

    dbschofield

      I have a test application running in JBoss 5.1 with HornetQ 2.0 installed.  I have been doing some performance testing and I am noticing that the InVMAcceptor is much slower than I would anticipate.  I have not done any tuning and the application in respects to JMS is 100% contained in one JBoss instance.   I am hoping someone has some insight on how to speed things up.

       

      Basically I have 100 concurrent HTTP requests.  At any giving time 25 of those requests are sitting in java.lang.Thread.setPriority0 during org.hornetq.jms.client.HornetQConnection.close().  All 25 threads have the same stack trace shown below.

       

      "http-localhost%2F10.160.20.36-8080-28" daemon prio=10 tid=0x09715400 nid=0x343d runnable [0x85a91000]
         java.lang.Thread.State: RUNNABLE
              at java.lang.Thread.setPriority0(Native Method)
              at java.lang.Thread.setPriority(Thread.java:1021)
              at java.lang.Thread.init(Thread.java:356)
              at java.lang.Thread.<init>(Thread.java:377)
              at org.hornetq.core.remoting.impl.invm.InVMAcceptor$Listener$1.<init>(InVMAcceptor.java:238)
              at org.hornetq.core.remoting.impl.invm.InVMAcceptor$Listener.connectionDestroyed(InVMAcceptor.java:235)
              at org.hornetq.core.remoting.impl.invm.InVMConnection.close(InVMConnection.java:91)
              - locked <0xda74a368> (a org.hornetq.core.remoting.impl.invm.InVMConnection)
              at org.hornetq.core.remoting.impl.invm.InVMAcceptor.disconnect(InVMAcceptor.java:205)
              at org.hornetq.core.remoting.impl.invm.InVMConnector$Listener.connectionDestroyed(InVMConnector.java:192)
              at org.hornetq.core.remoting.impl.invm.InVMConnection.close(InVMConnection.java:91)
              - locked <0xda74a138> (a org.hornetq.core.remoting.impl.invm.InVMConnection)
              at org.hornetq.core.remoting.impl.RemotingConnectionImpl.internalClose(RemotingConnectionImpl.java:480)
              at org.hornetq.core.remoting.impl.RemotingConnectionImpl.destroy(RemotingConnectionImpl.java:274)
              at org.hornetq.core.client.impl.FailoverManagerImpl.checkCloseConnection(FailoverManagerImpl.java:877)
              at org.hornetq.core.client.impl.FailoverManagerImpl.removeSession(FailoverManagerImpl.java:497)
              - locked <0xda748a60> (a java.lang.Object)
              - locked <0xda748a58> (a java.lang.Object)
              at org.hornetq.core.client.impl.ClientSessionImpl.doCleanup(ClientSessionImpl.java:1614)
              at org.hornetq.core.client.impl.ClientSessionImpl.close(ClientSessionImpl.java:811)
              at org.hornetq.core.client.impl.DelegatingSession.close(DelegatingSession.java:144)
              at org.hornetq.jms.client.HornetQConnection.close(HornetQConnection.java:294)
              - locked <0xda746c20> (a org.hornetq.jms.client.HornetQConnection)
              at org.apache.geronimo.samples.daytrader.ejb3.TradeSLSBBean.publishQuotePriceChange(TradeSLSBBean.java:632)
              at sun.reflect.GeneratedMethodAccessor319.invoke(Unknown Source)
              at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
              at java.lang.reflect.Method.invoke(Method.java:597)
              at org.jboss.aop.joinpoint.MethodInvocation.invokeTarget(MethodInvocation.java:122)
              at org.jboss.aop.joinpoint.MethodInvocation.invokeNext(MethodInvocation.java:111)
              at org.jboss.ejb3.EJBContainerInvocationWrapper.invokeNext(EJBContainerInvocationWrapper.java:69)
              at org.jboss.ejb3.interceptors.aop.InterceptorSequencer.invoke(InterceptorSequencer.java:76)
              at org.jboss.ejb3.interceptors.aop.InterceptorSequencer.aroundInvoke(InterceptorSequencer.java:62)
              at sun.reflect.GeneratedMethodAccessor272.invoke(Unknown Source)
              at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
              at java.lang.reflect.Method.invoke(Method.java:597)
              at org.jboss.aop.advice.PerJoinpointAdvice.invoke(PerJoinpointAdvice.java:174)
              at org.jboss.aop.joinpoint.MethodInvocation.invokeNext(MethodInvocation.java:102)
              at org.jboss.ejb3.interceptors.aop.InvocationContextInterceptor.fillMethod(InvocationContextInterceptor.java:72)
              at org.jboss.aop.advice.org.jboss.ejb3.interceptors.aop.InvocationContextInterceptor_z_fillMethod_169091.invoke(InvocationContextInterceptor_z_fillMethod_169091.java)
              at org.jboss.aop.joinpoint.MethodInvocation.invokeNext(MethodInvocation.java:102)
              at org.jboss.ejb3.interceptors.aop.InvocationContextInterceptor.setup(InvocationContextInterceptor.java:88)
              at org.jboss.aop.advice.org.jboss.ejb3.interceptors.aop.InvocationContextInterceptor_z_setup_169091.invoke(InvocationContextInterceptor_z_setup_169091.java)

      at org.jboss.aop.joinpoint.MethodInvocation.invokeNext(MethodInvocation.java:102)
              at org.jboss.ejb3.connectionmanager.CachedConnectionInterceptor.invoke(CachedConnectionInterceptor.java:62)
              at org.jboss.aop.joinpoint.MethodInvocation.invokeNext(MethodInvocation.java:102)
              at org.jboss.ejb3.entity.TransactionScopedEntityManagerInterceptor.invoke(TransactionScopedEntityManagerInterceptor.java:56)
              at org.jboss.aop.joinpoint.MethodInvocation.invokeNext(MethodInvocation.java:102)
              at org.jboss.ejb3.AllowedOperationsInterceptor.invoke(AllowedOperationsInterceptor.java:47)
              at org.jboss.aop.joinpoint.MethodInvocation.invokeNext(MethodInvocation.java:102)
              at org.jboss.ejb3.tx.NullInterceptor.invoke(NullInterceptor.java:42)
              at org.jboss.aop.joinpoint.MethodInvocation.invokeNext(MethodInvocation.java:102)
              at org.jboss.ejb3.stateless.StatelessInstanceInterceptor.invoke(StatelessInstanceInterceptor.java:68)
              at org.jboss.aop.joinpoint.MethodInvocation.invokeNext(MethodInvocation.java:102)
              at org.jboss.aspects.tx.TxPolicy.invokeInOurTx(TxPolicy.java:79)
              at org.jboss.aspects.tx.TxInterceptor$RequiresNew.invoke(TxInterceptor.java:252)
              at org.jboss.aop.joinpoint.MethodInvocation.invokeNext(MethodInvocation.java:102)
              at org.jboss.aspects.tx.TxPropagationInterceptor.invoke(TxPropagationInterceptor.java:76)
              at org.jboss.aop.joinpoint.MethodInvocation.invokeNext(MethodInvocation.java:102)
              at org.jboss.ejb3.tx.NullInterceptor.invoke(NullInterceptor.java:42)
              at org.jboss.aop.joinpoint.MethodInvocation.invokeNext(MethodInvocation.java:102)
              at org.jboss.ejb3.security.RoleBasedAuthorizationInterceptorv2.invoke(RoleBasedAuthorizationInterceptorv2.java:201)
              at org.jboss.aop.joinpoint.MethodInvocation.invokeNext(MethodInvocation.java:102)
              at org.jboss.ejb3.security.Ejb3AuthenticationInterceptorv2.invoke(Ejb3AuthenticationInterceptorv2.java:186)
              at org.jboss.aop.joinpoint.MethodInvocation.invokeNext(MethodInvocation.java:102)
              at org.jboss.ejb3.ENCPropagationInterceptor.invoke(ENCPropagationInterceptor.java:41)
              at org.jboss.aop.joinpoint.MethodInvocation.invokeNext(MethodInvocation.java:102)
              at org.jboss.ejb3.BlockContainerShutdownInterceptor.invoke(BlockContainerShutdownInterceptor.java:67)
              at org.jboss.aop.joinpoint.MethodInvocation.invokeNext(MethodInvocation.java:102)
              at org.jboss.aspects.currentinvocation.CurrentInvocationInterceptor.invoke(CurrentInvocationInterceptor.java:67)
              at org.jboss.aop.joinpoint.MethodInvocation.invokeNext(MethodInvocation.java:102)
              at org.jboss.ejb3.interceptor.EJB3TCCLInterceptor.invoke(EJB3TCCLInterceptor.java:86)
              at org.jboss.aop.joinpoint.MethodInvocation.invokeNext(MethodInvocation.java:102)
              at org.jboss.ejb3.stateless.StatelessContainer.dynamicInvoke(StatelessContainer.java:438)
              at org.jboss.ejb3.remoting.IsLocalInterceptor.invokeLocal(IsLocalInterceptor.java:85)
              at org.jboss.ejb3.remoting.IsLocalInterceptor.invoke(IsLocalInterceptor.java:72)
              at org.jboss.aop.joinpoint.MethodInvocation.invokeNext(MethodInvocation.java:102)
              at org.jboss.aspects.remoting.PojiProxy.invoke(PojiProxy.java:62)
              at $Proxy193.invoke(Unknown Source)
              at org.jboss.ejb3.proxy.impl.handler.session.SessionProxyInvocationHandlerBase.invoke(SessionProxyInvocationHandlerBase.java:188)
              at $Proxy182.publishQuotePriceChange(Unknown Source)
              at org.apache.geronimo.samples.daytrader.ejb3.TradeSLSBBean.updateQuotePriceVolume(TradeSLSBBean.java:446)
              at sun.reflect.GeneratedMethodAccessor312.invoke(Unknown Source)
              at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
              at java.lang.reflect.Method.invoke(Method.java:597)
              at org.jboss.aop.joinpoint.MethodInvocation.invokeTarget(MethodInvocation.java:122)
              at org.jboss.aop.joinpoint.MethodInvocation.invokeNext(MethodInvocation.java:111)
              at org.jboss.ejb3.EJBContainerInvocationWrapper.invokeNext(EJBContainerInvocationWrapper.java:69)
              at org.jboss.ejb3.interceptors.aop.InterceptorSequencer.invoke(InterceptorSequencer.java:76)
              at org.jboss.ejb3.interceptors.aop.InterceptorSequencer.aroundInvoke(InterceptorSequencer.java:62)
              at sun.reflect.GeneratedMethodAccessor272.invoke(Unknown Source)
              at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
              at java.lang.reflect.Method.invoke(Method.java:597)
              at org.jboss.aop.advice.PerJoinpointAdvice.invoke(PerJoinpointAdvice.java:174)
              at org.jboss.aop.joinpoint.MethodInvocation.invokeNext(MethodInvocation.java:102)
              at org.jboss.ejb3.interceptors.aop.InvocationContextInterceptor.fillMethod(InvocationContextInterceptor.java:72)
              at org.jboss.aop.advice.org.jboss.ejb3.interceptors.aop.InvocationContextInterceptor_z_fillMethod_169091.invoke(InvocationContextInterceptor_z_fillMethod_169091.java)

      at org.jboss.aop.joinpoint.MethodInvocation.invokeNext(MethodInvocation.java:102)
              at org.jboss.ejb3.interceptors.aop.InvocationContextInterceptor.setup(InvocationContextInterceptor.java:88)
              at org.jboss.aop.advice.org.jboss.ejb3.interceptors.aop.InvocationContextInterceptor_z_setup_169091.invoke(InvocationContextInterceptor_z_setup_169091.java)
              at org.jboss.aop.joinpoint.MethodInvocation.invokeNext(MethodInvocation.java:102)
              at org.jboss.ejb3.connectionmanager.CachedConnectionInterceptor.invoke(CachedConnectionInterceptor.java:62)
              at org.jboss.aop.joinpoint.MethodInvocation.invokeNext(MethodInvocation.java:102)
              at org.jboss.ejb3.entity.TransactionScopedEntityManagerInterceptor.invoke(TransactionScopedEntityManagerInterceptor.java:56)
              at org.jboss.aop.joinpoint.MethodInvocation.invokeNext(MethodInvocation.java:102)
              at org.jboss.ejb3.AllowedOperationsInterceptor.invoke(AllowedOperationsInterceptor.java:47)
              at org.jboss.aop.joinpoint.MethodInvocation.invokeNext(MethodInvocation.java:102)
              at org.jboss.ejb3.tx.NullInterceptor.invoke(NullInterceptor.java:42)
              at org.jboss.aop.joinpoint.MethodInvocation.invokeNext(MethodInvocation.java:102)
              at org.jboss.ejb3.stateless.StatelessInstanceInterceptor.invoke(StatelessInstanceInterceptor.java:68)
              at org.jboss.aop.joinpoint.MethodInvocation.invokeNext(MethodInvocation.java:102)
              at org.jboss.aspects.tx.TxPolicy.invokeInOurTx(TxPolicy.java:79)
              at org.jboss.aspects.tx.TxInterceptor$Required.invoke(TxInterceptor.java:190)
              at org.jboss.aop.joinpoint.MethodInvocation.invokeNext(MethodInvocation.java:102)
              at org.jboss.aspects.tx.TxPropagationInterceptor.invoke(TxPropagationInterceptor.java:76)
              at org.jboss.aop.joinpoint.MethodInvocation.invokeNext(MethodInvocation.java:102)
              at org.jboss.ejb3.tx.NullInterceptor.invoke(NullInterceptor.java:42)
              at org.jboss.aop.joinpoint.MethodInvocation.invokeNext(MethodInvocation.java:102)
              at org.jboss.ejb3.security.RoleBasedAuthorizationInterceptorv2.invoke(RoleBasedAuthorizationInterceptorv2.java:201)
              at org.jboss.aop.joinpoint.MethodInvocation.invokeNext(MethodInvocation.java:102)
              at org.jboss.ejb3.security.Ejb3AuthenticationInterceptorv2.invoke(Ejb3AuthenticationInterceptorv2.java:186)
              at org.jboss.aop.joinpoint.MethodInvocation.invokeNext(MethodInvocation.java:102)
              at org.jboss.ejb3.ENCPropagationInterceptor.invoke(ENCPropagationInterceptor.java:41)
              at org.jboss.aop.joinpoint.MethodInvocation.invokeNext(MethodInvocation.java:102)
              at org.jboss.ejb3.BlockContainerShutdownInterceptor.invoke(BlockContainerShutdownInterceptor.java:67)
              at org.jboss.aop.joinpoint.MethodInvocation.invokeNext(MethodInvocation.java:102)
              at org.jboss.aspects.currentinvocation.CurrentInvocationInterceptor.invoke(CurrentInvocationInterceptor.java:67)
              at org.jboss.aop.joinpoint.MethodInvocation.invokeNext(MethodInvocation.java:102)
              at org.jboss.ejb3.interceptor.EJB3TCCLInterceptor.invoke(EJB3TCCLInterceptor.java:86)
              at org.jboss.aop.joinpoint.MethodInvocation.invokeNext(MethodInvocation.java:102)
              at org.jboss.ejb3.stateless.StatelessContainer.dynamicInvoke(StatelessContainer.java:438)
              at org.jboss.ejb3.remoting.IsLocalInterceptor.invokeLocal(IsLocalInterceptor.java:85)
              at org.jboss.ejb3.remoting.IsLocalInterceptor.invoke(IsLocalInterceptor.java:72)
              at org.jboss.aop.joinpoint.MethodInvocation.invokeNext(MethodInvocation.java:102)
              at org.jboss.aspects.remoting.PojiProxy.invoke(PojiProxy.java:62)
              at $Proxy193.invoke(Unknown Source)
              at org.jboss.ejb3.proxy.impl.handler.session.SessionProxyInvocationHandlerBase.invoke(SessionProxyInvocationHandlerBase.java:188)
              at $Proxy192.updateQuotePriceVolume(Unknown Source)
              at org.apache.geronimo.samples.daytrader.TradeAction.updateQuotePriceVolume(TradeAction.java:392)
              at org.apache.geronimo.samples.daytrader.TradeAction.sell(TradeAction.java:241)
              at org.apache.geronimo.samples.daytrader.web.TradeServletAction.doSell(TradeServletAction.java:690)
              at org.apache.geronimo.samples.daytrader.web.TradeAppServlet.performTask(TradeAppServlet.java:163)
              at org.apache.geronimo.samples.daytrader.web.TradeAppServlet.doGet(TradeAppServlet.java:78)
              at javax.servlet.http.HttpServlet.service(HttpServlet.java:617)
              at javax.servlet.http.HttpServlet.service(HttpServlet.java:717)

              at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:290)
              at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
              at org.apache.geronimo.samples.daytrader.web.OrdersAlertFilter.doFilter(OrdersAlertFilter.java:91)
              at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
              at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
              at org.jboss.web.tomcat.filters.ReplyHeaderFilter.doFilter(ReplyHeaderFilter.java:96)
              at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
              at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
              at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:235)
              at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:191)
              at org.jboss.web.tomcat.security.SecurityAssociationValve.invoke(SecurityAssociationValve.java:190)
              at org.jboss.web.tomcat.security.JaccContextValve.invoke(JaccContextValve.java:92)
              at org.jboss.web.tomcat.security.SecurityContextEstablishmentValve.process(SecurityContextEstablishmentValve.java:126)
              at org.jboss.web.tomcat.security.SecurityContextEstablishmentValve.invoke(SecurityContextEstablishmentValve.java:70)
              at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127)
              at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102)
              at org.jboss.web.tomcat.service.jca.CachedConnectionValve.invoke(CachedConnectionValve.java:158)
              at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109)
              at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:330)
              at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:829)
              at org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:598)
              at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:447)
              at java.lang.Thread.run(Thread.java:619)

        • 1. Re: How to tune InVMAcceptor?
          timfox

          I'm not sure what you're trying to achieve here.

           

          At first glance looks like you're opening and closing connections rapidly (which would be an anti-pattern)

           

          Can you post a small self contained test program that demonstrates the issue?

          1 of 1 people found this helpful
          • 2. Re: How to tune InVMAcceptor?
            dbschofield

            Tim you are correct the code is opening and closing connections rapidly.  This got me thinking as to why the connection pooling of the JCA adaptor was not optimizing the get and close connection calls.  Sure enough the code was connecting directly to hornetq and not using the ConnectionFactory defined in the hornetq resource adaptor.  Mapped the resource-ref for the connectionFactory to java:JmsXA in jboss.xml and hornetq is now screaming as expected.  Thanks.