1 2 Previous Next 17 Replies Latest reply on Mar 23, 2005 6:10 PM by ben.wang Go to original post
      • 15. Re: TimeoutExceptions from TreeCache.callRemoteMethods()
        monocongo

        When I add

        [attribute name="DeadlockDetection"]true[/attribute]

        to the configuration of the TreeCacheAop MBean I get the following error when starting JBoss:

        10:18:42,795 INFO [ServiceConfigurator] Problem configuring service jboss.cache:service=TreeCacheAop
        org.jboss.deployment.DeploymentException: No Attribute found with name: DeadlockDetection
         at org.jboss.system.ServiceConfigurator.configure(ServiceConfigurator.java:263)
         at org.jboss.system.ServiceConfigurator.internalInstall(ServiceConfigurator.java:164)
         at org.jboss.system.ServiceConfigurator.install(ServiceConfigurator.java:118)
         at org.jboss.system.ServiceController.install(ServiceController.java:200)
         at sun.reflect.GeneratedMethodAccessor43.invoke(Unknown Source)
         at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
         at java.lang.reflect.Method.invoke(Method.java:585)
        
        etc....
        


        It looks like it should work, according to the TreeCache API, so why the DeploymentException ?


        --James

        • 16. Re: TimeoutExceptions from TreeCache.callRemoteMethods()
          monocongo

          I have modified my codes so that all transactions are being started and commited within the UserActivityManager MBean which is using the TreeCacheAop service to cache all user activity. Any EJB methods which access this UserActivityManager are marked with a NotSupported trans-attribute in the ejb-jar.xml in order to get around any potential nested transaction problems. My hope was that this would solve the deadlock problems that I was experiencing, but alas no such luck. I have no problems with the replication, but the deadlocks are still occurring with some frequency. Fortunately my application can handle these without failing, and no data seems to be corrupted in the TreeCacheAop. In other words after the TimeoutException is caught my application just hums right along as if nothing happened. Unfortunately it does cause an error to appear to users, and the user then needs to reinvoke the command which was unsuccessful, and this is something I'd really like to avoid, so I need to minimize or eliminate these deadlocks.

          Is there anything in the below stack trace which points to anything out of the ordinary with my configuration which might be causing this to happen ? Below the stack trace I list the jboss-service.xml which is being included in the SAR. Thanks for any insight.


          --James


          12:05:54,012 INFO [STDOUT] org.jboss.cache.lock.TimeoutException: rsp=sender=10.10.10.23:39435, retval=null, received=false, suspected=false
          12:05:54,013 INFO [STDOUT] at org.jboss.cache.TreeCache.callRemoteMethods(TreeCache.java:2169)
          12:05:54,014 INFO [STDOUT] at org.jboss.cache.TreeCache.callRemoteMethods(TreeCache.java:2191)
          12:05:54,014 INFO [STDOUT] at org.jboss.cache.interceptors.ReplicationInterceptor.handleReplicatedMethod(ReplicationInterceptor.java:117)
          12:05:54,014 INFO [STDOUT] at org.jboss.cache.interceptors.ReplicationInterceptor.invoke(ReplicationInterceptor.java:82)
          12:05:54,014 INFO [STDOUT] at org.jboss.cache.TreeCache.invokeMethod(TreeCache.java:3085)
          12:05:54,014 INFO [STDOUT] at org.jboss.cache.TreeCache.put(TreeCache.java:1728)
          12:05:54,014 INFO [STDOUT] at org.jboss.cache.aop.TreeCacheAop.getObject(TreeCacheAop.java:642)
          12:05:54,014 INFO [STDOUT] at org.jboss.cache.aop.CachedMapInterceptor.get(CachedMapInterceptor.java:67)
          12:05:54,014 INFO [STDOUT] at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
          12:05:54,014 INFO [STDOUT] at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
          12:05:54,015 INFO [STDOUT] at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
          12:05:54,015 INFO [STDOUT] at java.lang.reflect.Method.invoke(Method.java:585)
          12:05:54,015 INFO [STDOUT] at org.jboss.cache.aop.CollectionInterceptorUtil.invoke(CollectionInterceptorUtil.java:101)
          12:05:54,015 INFO [STDOUT] at org.jboss.cache.aop.CachedMapInterceptor.invoke(CachedMapInterceptor.java:47)
          12:05:54,015 INFO [STDOUT] at org.jboss.aop.joinpoint.MethodInvocation.invokeNext(MethodInvocation.java:66)
          12:05:54,015 INFO [STDOUT] at AOPClassProxy$1.get(AOPClassProxy$1.java)
          12:05:54,015 INFO [STDOUT] at com.harborsideplus.grover.bean.UserActivity.getLoginTime(UserActivity.java:306)
          12:05:54,015 INFO [STDOUT] at com.harborsideplus.grover.mbean.UserActivityManager.displayAllUserActivity(UserActivityManager.java:625)
          12:05:54,015 INFO [STDOUT] at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
          12:05:54,016 INFO [STDOUT] at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
          12:05:54,016 INFO [STDOUT] at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
          12:05:54,016 INFO [STDOUT] at java.lang.reflect.Method.invoke(Method.java:585)
          12:05:54,016 INFO [STDOUT] at org.jboss.mx.interceptor.ReflectedDispatcher.invoke(ReflectedDispatcher.java:144)
          12:05:54,016 INFO [STDOUT] at org.jboss.mx.server.Invocation.dispatch(Invocation.java:80)
          12:05:54,016 INFO [STDOUT] at org.jboss.mx.server.Invocation.invoke(Invocation.java:72)
          12:05:54,016 INFO [STDOUT] at org.jboss.mx.server.AbstractMBeanInvoker.invoke(AbstractMBeanInvoker.java:249)
          12:05:54,016 INFO [STDOUT] at org.jboss.mx.server.MBeanServerImpl.invoke(MBeanServerImpl.java:642)
          12:05:54,016 INFO [STDOUT] at org.jboss.jmx.adaptor.control.Server.invokeOpByName(Server.java:236)
          12:05:54,017 INFO [STDOUT] at org.jboss.jmx.adaptor.control.Server.invokeOp(Server.java:202)
          12:05:54,017 INFO [STDOUT] at org.jboss.jmx.adaptor.html.HtmlAdaptorServlet.invokeOp(HtmlAdaptorServlet.java:241)
          12:05:54,017 INFO [STDOUT] at org.jboss.jmx.adaptor.html.HtmlAdaptorServlet.processRequest(HtmlAdaptorServlet.java:79)
          12:05:54,017 INFO [STDOUT] at org.jboss.jmx.adaptor.html.HtmlAdaptorServlet.doPost(HtmlAdaptorServlet.java:61)
          12:05:54,017 INFO [STDOUT] at javax.servlet.http.HttpServlet.service(HttpServlet.java:717)
          12:05:54,017 INFO [STDOUT] at javax.servlet.http.HttpServlet.service(HttpServlet.java:810)
          12:05:54,017 INFO [STDOUT] at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:237)
          12:05:54,017 INFO [STDOUT] at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:157)
          12:05:54,017 INFO [STDOUT] at org.jboss.web.tomcat.filters.ReplyHeaderFilter.doFilter(ReplyHeaderFilter.java:75)
          12:05:54,018 INFO [STDOUT] at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:186)
          12:05:54,018 INFO [STDOUT] at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:157)
          12:05:54,018 INFO [STDOUT] at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:214)
          12:05:54,018 INFO [STDOUT] at org.apache.catalina.core.StandardValveContext.invokeNext(StandardValveContext.java:104)
          12:05:54,018 INFO [STDOUT] at org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.java:520)
          12:05:54,018 INFO [STDOUT] at org.apache.catalina.core.StandardContextValve.invokeInternal(StandardContextValve.java:198)
          12:05:54,019 INFO [STDOUT] at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:152)
          12:05:54,019 INFO [STDOUT] at org.apache.catalina.core.StandardValveContext.invokeNext(StandardValveContext.java:104)
          12:05:54,019 INFO [STDOUT] at org.jboss.web.tomcat.security.CustomPrincipalValve.invoke(CustomPrincipalValve.java:66)
          12:05:54,019 INFO [STDOUT] at org.apache.catalina.core.StandardValveContext.invokeNext(StandardValveContext.java:102)
          12:05:54,019 INFO [STDOUT] at org.jboss.web.tomcat.security.SecurityAssociationValve.invoke(SecurityAssociationValve.java:153)
          12:05:54,019 INFO [STDOUT] at org.apache.catalina.core.StandardValveContext.invokeNext(StandardValveContext.java:102)
          12:05:54,019 INFO [STDOUT] at org.jboss.web.tomcat.security.JaccContextValve.invoke(JaccContextValve.java:54)
          12:05:54,019 INFO [STDOUT] at org.apache.catalina.core.StandardValveContext.invokeNext(StandardValveContext.java:102)
          12:05:54,019 INFO [STDOUT] at org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.java:520)
          12:05:54,020 INFO [STDOUT] at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:137)
          12:05:54,020 INFO [STDOUT] at org.apache.catalina.core.StandardValveContext.invokeNext(StandardValveContext.java:104)
          12:05:54,020 INFO [STDOUT] at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:118)
          12:05:54,020 INFO [STDOUT] at org.apache.catalina.core.StandardValveContext.invokeNext(StandardValveContext.java:102)
          12:05:54,020 INFO [STDOUT] at org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.java:520)
          12:05:54,020 INFO [STDOUT] at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109)
          12:05:54,020 INFO [STDOUT] at org.apache.catalina.core.StandardValveContext.invokeNext(StandardValveContext.java:104)
          12:05:54,020 INFO [STDOUT] at org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.java:520)
          12:05:54,021 INFO [STDOUT] at org.apache.catalina.core.ContainerBase.invoke(ContainerBase.java:929)
          12:05:54,021 INFO [STDOUT] at org.apache.coyote.tomcat5.CoyoteAdapter.service(CoyoteAdapter.java:160)
          12:05:54,021 INFO [STDOUT] at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:799)
          12:05:54,021 INFO [STDOUT] at org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.processConnection(Http11Protocol.java:705)
          12:05:54,021 INFO [STDOUT] at org.apache.tomcat.util.net.TcpWorkerThread.runIt(PoolTcpEndpoint.java:577)
          12:05:54,021 INFO [STDOUT] at org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run(ThreadPool.java:683)
          12:05:54,021 INFO [STDOUT] at java.lang.Thread.run(Thread.java:595)
          12:05:54,022 WARN [NestedThrowable] Duplicate throwable nesting of same base type: class org.jboss.util.NestedRuntimeException is assignable from: class org.jboss.util.NestedRuntimeException
          12:05:54,022 WARN [UserActivityManager] Error listing the UserActivity objects stored in the TreeCache
          java.lang.reflect.InvocationTargetException
           at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
           at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
           at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
           at java.lang.reflect.Method.invoke(Method.java:585)
           at org.jboss.cache.aop.CollectionInterceptorUtil.invoke(CollectionInterceptorUtil.java:101)
           at org.jboss.cache.aop.CachedMapInterceptor.invoke(CachedMapInterceptor.java:47)
           at org.jboss.aop.joinpoint.MethodInvocation.invokeNext(MethodInvocation.java:66)
           at AOPClassProxy$1.get(AOPClassProxy$1.java)
           at com.harborsideplus.grover.bean.UserActivity.getLoginTime(UserActivity.java:306)
           at com.harborsideplus.grover.mbean.UserActivityManager.displayAllUserActivity(UserActivityManager.java:625)
           at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
           at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
           at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
           at java.lang.reflect.Method.invoke(Method.java:585)
           at org.jboss.mx.interceptor.ReflectedDispatcher.invoke(ReflectedDispatcher.java:144)
           at org.jboss.mx.server.Invocation.dispatch(Invocation.java:80)
           at org.jboss.mx.server.Invocation.invoke(Invocation.java:72)
           at org.jboss.mx.server.AbstractMBeanInvoker.invoke(AbstractMBeanInvoker.java:249)
           at org.jboss.mx.server.MBeanServerImpl.invoke(MBeanServerImpl.java:642)
           at org.jboss.jmx.adaptor.control.Server.invokeOpByName(Server.java:236)
           at org.jboss.jmx.adaptor.control.Server.invokeOp(Server.java:202)
           at org.jboss.jmx.adaptor.html.HtmlAdaptorServlet.invokeOp(HtmlAdaptorServlet.java:241)
           at org.jboss.jmx.adaptor.html.HtmlAdaptorServlet.processRequest(HtmlAdaptorServlet.java:79)
           at org.jboss.jmx.adaptor.html.HtmlAdaptorServlet.doPost(HtmlAdaptorServlet.java:61)
           at javax.servlet.http.HttpServlet.service(HttpServlet.java:717)
           at javax.servlet.http.HttpServlet.service(HttpServlet.java:810)
           at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:237)
           at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:157)
           at org.jboss.web.tomcat.filters.ReplyHeaderFilter.doFilter(ReplyHeaderFilter.java:75)
           at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:186)
           at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:157)
           at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:214)
           at org.apache.catalina.core.StandardValveContext.invokeNext(StandardValveContext.java:104)
           at org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.java:520)
           at org.apache.catalina.core.StandardContextValve.invokeInternal(StandardContextValve.java:198)
           at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:152)
           at org.apache.catalina.core.StandardValveContext.invokeNext(StandardValveContext.java:104)
           at org.jboss.web.tomcat.security.CustomPrincipalValve.invoke(CustomPrincipalValve.java:66)
           at org.apache.catalina.core.StandardValveContext.invokeNext(StandardValveContext.java:102)
           at org.jboss.web.tomcat.security.SecurityAssociationValve.invoke(SecurityAssociationValve.java:153)
           at org.apache.catalina.core.StandardValveContext.invokeNext(StandardValveContext.java:102)
           at org.jboss.web.tomcat.security.JaccContextValve.invoke(JaccContextValve.java:54)
           at org.apache.catalina.core.StandardValveContext.invokeNext(StandardValveContext.java:102)
           at org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.java:520)
           at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:137)
           at org.apache.catalina.core.StandardValveContext.invokeNext(StandardValveContext.java:104)
           at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:118)
           at org.apache.catalina.core.StandardValveContext.invokeNext(StandardValveContext.java:102)
           at org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.java:520)
           at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109)
           at org.apache.catalina.core.StandardValveContext.invokeNext(StandardValveContext.java:104)
           at org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.java:520)
           at org.apache.catalina.core.ContainerBase.invoke(ContainerBase.java:929)
           at org.apache.coyote.tomcat5.CoyoteAdapter.service(CoyoteAdapter.java:160)
           at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:799)
           at org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.processConnection(Http11Protocol.java:705)
           at org.apache.tomcat.util.net.TcpWorkerThread.runIt(PoolTcpEndpoint.java:577)
           at org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run(ThreadPool.java:683)
           at java.lang.Thread.run(Thread.java:595)
          Caused by: org.jboss.util.NestedRuntimeException: TreeCacheAop.getObject(): ; - nested throwable: (org.jboss.cache.lock.TimeoutException: rsp=sender=10.10.10.23:39435, retval=null, received=false, suspected=false); - nested throwable: (org.jboss.util.NestedRuntimeException: TreeCacheAop.getObject(): ; - nested throwable: (org.jboss.cache.lock.TimeoutException: rsp=sender=10.10.10.23:39435, retval=null, received=false, suspected=false))
           at org.jboss.cache.aop.CachedMapInterceptor.get(CachedMapInterceptor.java:69)
           ... 59 more
          Caused by: org.jboss.util.NestedRuntimeException: TreeCacheAop.getObject(): ; - nested throwable: (org.jboss.cache.lock.TimeoutException: rsp=sender=10.10.10.23:39435, retval=null, received=false, suspected=false)
           at org.jboss.cache.aop.TreeCacheAop.getObject(TreeCacheAop.java:659)
           at org.jboss.cache.aop.CachedMapInterceptor.get(CachedMapInterceptor.java:67)
           ... 59 more
          Caused by: org.jboss.cache.lock.TimeoutException: rsp=sender=10.10.10.23:39435, retval=null, received=false, suspected=false
           at org.jboss.cache.TreeCache.callRemoteMethods(TreeCache.java:2169)
           at org.jboss.cache.TreeCache.callRemoteMethods(TreeCache.java:2191)
           at org.jboss.cache.interceptors.ReplicationInterceptor.handleReplicatedMethod(ReplicationInterceptor.java:117)
           at org.jboss.cache.interceptors.ReplicationInterceptor.invoke(ReplicationInterceptor.java:82)
           at org.jboss.cache.TreeCache.invokeMethod(TreeCache.java:3085)
           at org.jboss.cache.TreeCache.put(TreeCache.java:1728)
           at org.jboss.cache.aop.TreeCacheAop.getObject(TreeCacheAop.java:642)
           ... 60 more
          12:05:54,080 ERROR [Engine] StandardWrapperValve[HtmlAdaptor]: Servlet.service() for servlet HtmlAdaptor threw exception
          javax.management.MBeanException
           at org.jboss.mx.interceptor.ReflectedDispatcher.handleInvocationExceptions(ReflectedDispatcher.java:169)
           at org.jboss.mx.interceptor.ReflectedDispatcher.invoke(ReflectedDispatcher.java:152)
           at org.jboss.mx.server.Invocation.dispatch(Invocation.java:80)
           at org.jboss.mx.server.Invocation.invoke(Invocation.java:72)
           at org.jboss.mx.server.AbstractMBeanInvoker.invoke(AbstractMBeanInvoker.java:249)
           at org.jboss.mx.server.MBeanServerImpl.invoke(MBeanServerImpl.java:642)
           at org.jboss.jmx.adaptor.control.Server.invokeOpByName(Server.java:236)
           at org.jboss.jmx.adaptor.control.Server.invokeOp(Server.java:202)
           at org.jboss.jmx.adaptor.html.HtmlAdaptorServlet.invokeOp(HtmlAdaptorServlet.java:241)
           at org.jboss.jmx.adaptor.html.HtmlAdaptorServlet.processRequest(HtmlAdaptorServlet.java:79)
           at org.jboss.jmx.adaptor.html.HtmlAdaptorServlet.doPost(HtmlAdaptorServlet.java:61)
           at javax.servlet.http.HttpServlet.service(HttpServlet.java:717)
           at javax.servlet.http.HttpServlet.service(HttpServlet.java:810)
           at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:237)
           at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:157)
           at org.jboss.web.tomcat.filters.ReplyHeaderFilter.doFilter(ReplyHeaderFilter.java:75)
           at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:186)
           at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:157)
           at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:214)
           at org.apache.catalina.core.StandardValveContext.invokeNext(StandardValveContext.java:104)
           at org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.java:520)
           at org.apache.catalina.core.StandardContextValve.invokeInternal(StandardContextValve.java:198)
           at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:152)
           at org.apache.catalina.core.StandardValveContext.invokeNext(StandardValveContext.java:104)
           at org.jboss.web.tomcat.security.CustomPrincipalValve.invoke(CustomPrincipalValve.java:66)
           at org.apache.catalina.core.StandardValveContext.invokeNext(StandardValveContext.java:102)
           at org.jboss.web.tomcat.security.SecurityAssociationValve.invoke(SecurityAssociationValve.java:153)
           at org.apache.catalina.core.StandardValveContext.invokeNext(StandardValveContext.java:102)
           at org.jboss.web.tomcat.security.JaccContextValve.invoke(JaccContextValve.java:54)
           at org.apache.catalina.core.StandardValveContext.invokeNext(StandardValveContext.java:102)
           at org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.java:520)
           at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:137)
           at org.apache.catalina.core.StandardValveContext.invokeNext(StandardValveContext.java:104)
           at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:118)
           at org.apache.catalina.core.StandardValveContext.invokeNext(StandardValveContext.java:102)
           at org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.java:520)
           at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109)
           at org.apache.catalina.core.StandardValveContext.invokeNext(StandardValveContext.java:104)
           at org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.java:520)
           at org.apache.catalina.core.ContainerBase.invoke(ContainerBase.java:929)
           at org.apache.coyote.tomcat5.CoyoteAdapter.service(CoyoteAdapter.java:160)
           at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:799)
           at org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.processConnection(Http11Protocol.java:705)
           at org.apache.tomcat.util.net.TcpWorkerThread.runIt(PoolTcpEndpoint.java:577)
           at org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run(ThreadPool.java:683)
           at java.lang.Thread.run(Thread.java:595)
          Caused by: javax.management.MBeanException: Error listing the UserActivity objects stored in the cache
           at com.harborsideplus.grover.mbean.UserActivityManager.displayAllUserActivity(UserActivityManager.java:702)
           at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
           at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
           at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
           at java.lang.reflect.Method.invoke(Method.java:585)
           at org.jboss.mx.interceptor.ReflectedDispatcher.invoke(ReflectedDispatcher.java:144)
           ... 44 more
          Caused by: java.lang.reflect.InvocationTargetException
           at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
           at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
           at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
           at java.lang.reflect.Method.invoke(Method.java:585)
           at org.jboss.cache.aop.CollectionInterceptorUtil.invoke(CollectionInterceptorUtil.java:101)
           at org.jboss.cache.aop.CachedMapInterceptor.invoke(CachedMapInterceptor.java:47)
           at org.jboss.aop.joinpoint.MethodInvocation.invokeNext(MethodInvocation.java:66)
           at AOPClassProxy$1.get(AOPClassProxy$1.java)
           at com.harborsideplus.grover.bean.UserActivity.getLoginTime(UserActivity.java:306)
           at com.harborsideplus.grover.mbean.UserActivityManager.displayAllUserActivity(UserActivityManager.java:625)
           ... 49 more
          Caused by: org.jboss.util.NestedRuntimeException: TreeCacheAop.getObject(): ; - nested throwable: (org.jboss.cache.lock.TimeoutException: rsp=sender=10.10.10.23:39435, retval=null, received=false, suspected=false); - nested throwable: (org.jboss.util.NestedRuntimeException: TreeCacheAop.getObject(): ; - nested throwable: (org.jboss.cache.lock.TimeoutException: rsp=sender=10.10.10.23:39435, retval=null, received=false, suspected=false))
           at org.jboss.cache.aop.CachedMapInterceptor.get(CachedMapInterceptor.java:69)
           ... 59 more
          Caused by: org.jboss.util.NestedRuntimeException: TreeCacheAop.getObject(): ; - nested throwable: (org.jboss.cache.lock.TimeoutException: rsp=sender=10.10.10.23:39435, retval=null, received=false, suspected=false)
           at org.jboss.cache.aop.TreeCacheAop.getObject(TreeCacheAop.java:659)
           at org.jboss.cache.aop.CachedMapInterceptor.get(CachedMapInterceptor.java:67)
           ... 59 more
          Caused by: org.jboss.cache.lock.TimeoutException: rsp=sender=10.10.10.23:39435, retval=null, received=false, suspected=false
           at org.jboss.cache.TreeCache.callRemoteMethods(TreeCache.java:2169)
           at org.jboss.cache.TreeCache.callRemoteMethods(TreeCache.java:2191)
           at org.jboss.cache.interceptors.ReplicationInterceptor.handleReplicatedMethod(ReplicationInterceptor.java:117)
           at org.jboss.cache.interceptors.ReplicationInterceptor.invoke(ReplicationInterceptor.java:82)
           at org.jboss.cache.TreeCache.invokeMethod(TreeCache.java:3085)
           at org.jboss.cache.TreeCache.put(TreeCache.java:1728)
           at org.jboss.cache.aop.TreeCacheAop.getObject(TreeCacheAop.java:642)
           ... 60 more



          The jboss-service.xml:

          <?xml version="1.0" encoding="UTF-8"?>
          
          <server>
          
           <mbean code="org.jboss.cache.aop.TreeCacheAop"
           name="jboss.cache:service=TreeCacheAop">
           <depends>jboss:service=Naming</depends>
           <depends>jboss:service=TransactionManager</depends>
           <attribute name="TransactionManagerLookupClass">org.jboss.cache.JBossTransactionManagerLookup</attribute>
           <!-- Isolation Levels: NONE, READ_UNCOMMITTED, READ_COMMITTED, REPEATABLE_READ, and SERIALIZABLE -->
           <attribute name="IsolationLevel">REPEATABLE_READ</attribute>
           <attribute name="CacheMode">REPL_SYNC</attribute>
           <attribute name="UseReplQueue">false</attribute>
           <attribute name="ReplQueueInterval">0</attribute>
           <attribute name="ReplQueueMaxElements">0</attribute>
           <attribute name="ClusterName">TreeCache-Cluster</attribute>
           <attribute name="ClusterConfig">
           <config>
           <UDP mcast_addr="228.1.2.3"
           mcast_port="45566"
           ip_ttl="64"
           ip_mcast="true"
           mcast_send_buf_size="150000"
           mcast_recv_buf_size="80000"
           ucast_send_buf_size="150000"
           ucast_recv_buf_size="80000"
           loopback="false"/>
           <PING timeout="2000"
           num_initial_members="3"
           up_thread="false"
           down_thread="false"/>
           <MERGE2 min_interval="10000"
           max_interval="20000"/>
           <FD_SOCK/>
           <VERIFY_SUSPECT timeout="1500"
           up_thread="false"
           down_thread="false"/>
           <pbcast.NAKACK gc_lag="50"
           retransmit_timeout="600,1200,2400,4800"
           up_thread="false"
           down_thread="false"/>
           <pbcast.STABLE desired_avg_gossip="20000"
           up_thread="false"
           down_thread="false"/>
           <UNICAST timeout="600,1200,2400"
           window_size="100"
           min_threshold="10"
           down_thread="false"/>
           <FRAG frag_size="8192"
           down_thread="false"
           up_thread="false"/>
           <pbcast.GMS join_timeout="5000"
           join_retry_timeout="2000"
           shun="true"
           print_local_addr="true"/>
           <pbcast.STATE_TRANSFER up_thread="true"
           down_thread="true"/>
           </config>
           </attribute>
           <attribute name="FetchStateOnStartup">true</attribute>
           <attribute name="InitialStateRetrievalTimeout">15000</attribute>
           <attribute name="SyncReplTimeout">10000</attribute>
           <attribute name="LockAcquisitionTimeout">30000</attribute>
           <attribute name="EvictionPolicyClass"></attribute>
           <!-- <attribute name="DeadlockDetection">true</attribute> -->
           </mbean>
          
          
           <mbean code="com.harborsideplus.grover.mbean.UserActivityManager"
           name="grover.management:service=UserActivityManager">
           <depends>jboss.cache:service=TreeCacheAop</depends>
           </mbean>
          
           <mbean code="com.harborsideplus.grover.mbean.UserErrorManager"
           name="grover.management:service=UserErrorManager">
           <depends>jboss.cache:service=TreeCacheAop</depends>
           </mbean>
          
           <mbean code="com.harborsideplus.grover.mbean.UserMessagesManager"
           name="grover.management:service=UserMessagesManager">
           <depends>jboss.cache:service=TreeCacheAop</depends>
           </mbean>
          
          </server>


          • 17. Re: TimeoutExceptions from TreeCache.callRemoteMethods()

            James,

            I'd really like to help you resolve this. But in order for me to troubleshoot, I'd need preferably a JUnit test case that I can re-produce the problem. Do you think that's possible? Or other alternative?

            This is TimeoutException meaning that the node that you access is used by other guys now.

            -Ben

            1 2 Previous Next