1 Reply Latest reply on Oct 2, 2010 3:41 PM by bfluri

    Intermittetly NPE during parseParameters in Tomcat's Request

    bfluri
    bfluri Jun 15, 2010 2:07 AM

      We have intermittet NPE during parseParemeters in org.apache.catalina.connector.Request. The more users are online, the more this NPE happens. After a JBoss restart, the NPEs disappear for a while. Within 24 hours we receive between one and over 400 of those NPE. It does not matter which service is called. Any service request can end in this NPE.

       

      Stacktrace:

      java.lang.NullPointerException
              at org.apache.catalina.connector.Request.parseParameters(Request.java:2517)
              at org.apache.catalina.connector.Request.getParameterNames(Request.java:1102)
              at org.apache.catalina.connector.Request.getParameterMap(Request.java:1082)
              at org.apache.catalina.connector.RequestFacade.getParameterMap(RequestFacade.java:414)
              at javax.servlet.ServletRequestWrapper.getParameterMap(ServletRequestWrapper.java:166)
              at org.jboss.seam.mock.MockExternalContext.getRequestParameterValuesMap(MockExternalContext.java:307)
              at org.jboss.seam.faces.Parameters.getRequestParameters(Parameters.java:61)
              at org.jboss.seam.Component.injectParameters(Component.java:1586)
              at org.jboss.seam.Component.inject(Component.java:1556)
              at org.jboss.seam.core.BijectionInterceptor.aroundInvoke(BijectionInterceptor.java:61)
              at org.jboss.seam.intercept.SeamInvocationContext.proceed(SeamInvocationContext.java:68)
              at org.jboss.seam.transaction.TransactionInterceptor$1.work(TransactionInterceptor.java:97)
              at org.jboss.seam.util.Work.workInTransaction(Work.java:61)
              at org.jboss.seam.transaction.TransactionInterceptor.aroundInvoke(TransactionInterceptor.java:91)
              at org.jboss.seam.intercept.SeamInvocationContext.proceed(SeamInvocationContext.java:68)
              at org.jboss.seam.core.MethodContextInterceptor.aroundInvoke(MethodContextInterceptor.java:44)
              at org.jboss.seam.intercept.SeamInvocationContext.proceed(SeamInvocationContext.java:68)
              at org.jboss.seam.security.SecurityInterceptor.aroundInvoke(SecurityInterceptor.java:163)
              at org.jboss.seam.intercept.SeamInvocationContext.proceed(SeamInvocationContext.java:68)
              at ExceptionInterceptor.aroundInvoke(ExceptionInterceptor.java:51)
              at sun.reflect.GeneratedMethodAccessor289.invoke(Unknown Source)
              at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
              at java.lang.reflect.Method.invoke(Method.java:597)
              at org.jboss.seam.util.Reflections.invoke(Reflections.java:22)
              at org.jboss.seam.intercept.Interceptor.aroundInvoke(Interceptor.java:187)
              at org.jboss.seam.intercept.SeamInvocationContext.proceed(SeamInvocationContext.java:72)
              at org.jboss.seam.intercept.RootInterceptor.invoke(RootInterceptor.java:107)
              at org.jboss.seam.intercept.JavaBeanInterceptor.interceptInvocation(JavaBeanInterceptor.java:185)
              at org.jboss.seam.intercept.JavaBeanInterceptor.invoke(JavaBeanInterceptor.java:103)
              at TaskService_$$_javassist_seam_7.getNumberOfUpdatedTasks(TaskService_$$_javassist_seam_7.java)
              at sun.reflect.GeneratedMethodAccessor319.invoke(Unknown Source)
              at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
              at java.lang.reflect.Method.invoke(Method.java:597)
              at org.jboss.seam.remoting.gwt.GWTToSeamAdapter.callWebRemoteMethod(GWTToSeamAdapter.java:100)
              at org.jboss.seam.remoting.gwt.GWTService.RPC_invokeAndEncodeResponse(GWTService.java:550)
              at org.jboss.seam.remoting.gwt.GWTService.processCall(GWTService.java:206)
              at org.jboss.seam.remoting.gwt.GWTService$1.process(GWTService.java:120)
              at org.jboss.seam.servlet.ContextualHttpServletRequest.run(ContextualHttpServletRequest.java:53)
              at org.jboss.seam.remoting.gwt.GWTService.getResource(GWTService.java:105)
              at org.jboss.seam.servlet.SeamResourceServlet.service(SeamResourceServlet.java:80)
              at javax.servlet.http.HttpServlet.service(HttpServlet.java:717)
              at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:290)
              at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
              at org.jboss.seam.servlet.SeamFilter$FilterChainImpl.doFilter(SeamFilter.java:83)
              at org.jboss.seam.web.LoggingFilter.doFilter(LoggingFilter.java:60)
              at org.jboss.seam.servlet.SeamFilter$FilterChainImpl.doFilter(SeamFilter.java:69)
              at org.jboss.seam.web.ExceptionFilter.doFilter(ExceptionFilter.java:64)
              at org.jboss.seam.servlet.SeamFilter$FilterChainImpl.doFilter(SeamFilter.java:69)
              at org.jboss.seam.web.RedirectFilter.doFilter(RedirectFilter.java:45)
              at org.jboss.seam.servlet.SeamFilter$FilterChainImpl.doFilter(SeamFilter.java:69)
              at org.jboss.seam.web.IdentityFilter.doFilter(IdentityFilter.java:40)
              at org.jboss.seam.servlet.SeamFilter$FilterChainImpl.doFilter(SeamFilter.java:69)
              at org.jboss.seam.servlet.SeamFilter$FilterChainImpl.doFilter(SeamFilter.java:73)
              at org.jboss.seam.servlet.SeamFilter.doFilter(SeamFilter.java:158)
              at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
              at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
              at org.jboss.web.tomcat.filters.ReplyHeaderFilter.doFilter(ReplyHeaderFilter.java:96)
              at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
              at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
              at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:235)
              at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:191)
              at org.jboss.web.tomcat.security.SecurityAssociationValve.invoke(SecurityAssociationValve.java:190)
              at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:433)
              at org.jboss.web.tomcat.security.JaccContextValve.invoke(JaccContextValve.java:92)
              at org.jboss.web.tomcat.security.SecurityContextEstablishmentValve.process(SecurityContextEstablishmentValve.java:126)
              at org.jboss.web.tomcat.security.SecurityContextEstablishmentValve.invoke(SecurityContextEstablishmentValve.java:70)
              at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127)
              at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102)
              at org.jboss.web.tomcat.service.jca.CachedConnectionValve.invoke(CachedConnectionValve.java:158)
              at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109)
              at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:330)
              at org.apache.coyote.ajp.AjpProcessor.process(AjpProcessor.java:436)
              at org.apache.coyote.ajp.AjpProtocol$AjpConnectionHandler.process(AjpProtocol.java:384)
              at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:447)
              at java.lang.Thread.run(Thread.java:619)

       

      We use JBoss AS 5.1.0.GA and Seam 2.2.0.GA. The JBoss receives the request from Apache 2 via mod_jk. The provided line number (Request.java:2517) indicates that the method of the request is null although the logs of, firebug (client), Apache, and mod_jk show that the method is POST.

       

      Currently, we neither have an idea what may be the root-cause of the NPE nor how we could solve this issue. We are speculating whether the problem has something to do with:

      • Garbage collection (JBoss is started with -Dsun.rmi.dgc.client.gcInterval=3600000 -Dsun.rmi.dgc.server.gcInterval=3600000)
      • Request recycling in Tomcat
      • Filter chain recycling in Tomcat
      • mod_jk balancing

       

      What can we do to find the cause of this problem? Is there any possible fix to this problem?

       

      Any help or suggestion is highly appreciated.

       

      Thanks!

        • 1. Re: Intermittetly NPE during parseParameters in Tomcat's Request
          bfluri
          bfluri Oct 2, 2010 3:41 PM (in response to bfluri)

          In the mean time we found the reason for the NPEs:

           

          Since we use GWT on the client-side, all client-server communication is done via GWT-RPC asynchronously. Very rarely, a logout call outruns another still processing RPC. The logout call invalidates the session, so the other RPC cannot complete normally which leads to an Exception during ServletLifecycle.endRequest(request); inside the ContextualHttpServletRequest. This Exception is handled by Seam's ExceptionFilter. Unfortunately, the ExceptionFilter can also not complete normally because of the invalidated Session leading to the following error:

           

          ERROR    [Seam Resource Servlet].error: Servlet.service() for servlet Seam Resource Servlet threw exception
java.lang.IllegalStateException: Cannot create a session after the response has been committed
        at org.apache.catalina.connector.Request.doGetSession(Request.java:2338)
        at org.apache.catalina.connector.Request.getSession(Request.java:2094)
        at org.apache.catalina.connector.RequestFacade.getSession(RequestFacade.java:833)
        at javax.servlet.http.HttpServletRequestWrapper.getSession(HttpServletRequestWrapper.java:216)
        at org.jboss.seam.mock.MockExternalContext.getSessionMap(MockExternalContext.java:357)
        at org.jboss.seam.contexts.FacesLifecycle.beginExceptionRecovery(FacesLifecycle.java:86)
        at org.jboss.seam.web.ExceptionFilter.endWebRequestAfterException(ExceptionFilter.java:96)
        at org.jboss.seam.web.ExceptionFilter.doFilter(ExceptionFilter.java:70)
        at org.jboss.seam.servlet.SeamFilter$FilterChainImpl.doFilter(SeamFilter.java:69)
        at org.jboss.seam.web.RedirectFilter.doFilter(RedirectFilter.java:45)
        at org.jboss.seam.servlet.SeamFilter$FilterChainImpl.doFilter(SeamFilter.java:69)
        at org.jboss.seam.web.IdentityFilter.doFilter(IdentityFilter.java:40)
        at org.jboss.seam.servlet.SeamFilter$FilterChainImpl.doFilter(SeamFilter.java:69)
        at org.jboss.seam.servlet.SeamFilter$FilterChainImpl.doFilter(SeamFilter.java:73)
        at org.jboss.seam.servlet.SeamFilter.doFilter(SeamFilter.java:158)
        at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
        at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
        at org.jboss.web.tomcat.filters.ReplyHeaderFilter.doFilter(ReplyHeaderFilter.java:96)
        at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
        at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
        at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:235)
        at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:191)
        at org.jboss.web.tomcat.security.SecurityAssociationValve.invoke(SecurityAssociationValve.java:190)
        at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:433)
        at org.jboss.web.tomcat.security.JaccContextValve.invoke(JaccContextValve.java:92)
        at org.jboss.web.tomcat.security.SecurityContextEstablishmentValve.process(SecurityContextEstablishmentValve.java:126)
        at org.jboss.web.tomcat.security.SecurityContextEstablishmentValve.invoke(SecurityContextEstablishmentValve.java:70)
        at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127)
        at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102)
        at org.jboss.web.tomcat.service.jca.CachedConnectionValve.invoke(CachedConnectionValve.java:158)
        at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109)
        at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:330)
        at org.apache.coyote.ajp.AjpProcessor.process(AjpProcessor.java:436)
        at org.apache.coyote.ajp.AjpProtocol$AjpConnectionHandler.process(AjpProtocol.java:384)
        at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:447)
        at java.lang.Thread.run(Thread.java:619)

           

          The MockExternalContext that is created in ExceptionFilter, "somehow" stays in the application context and is "sometimes" used to process requests. It even survives redeployments of our applications, so that we have to restart JBoss to get rid of the NPEs.

            • Actions