ERROR -- ApplicationDispatcher[] Servlet.service() for servl

[namitajain]

Hi,
I am running JBoss 3.2.5 on Linux with 1GHZ RAM and 180 GB hard-drive and have around 15 websites hosted on it (which are not hit so hard though).

Once in 4-5 days I get the following stack trace of erros in my Log and the particular Servlet (called PartQueryServlet of my applications) stops responding, as a result of which some part of all the websites is down, but the JBoss server keeps running. When I restart JBoss everything comes back up.

Please let me know what may cause this kind of error stack and what may I do to avoid it.
I assume that trying to allocate more JVM may not resolve the problem completely. I have also tuned my code n believe that there are no memory leaks in the code.

Looking for urgent response..
Thanks much..
N.J


--------ERROR STACKTRACE-------
07 Aug 2005 22:09:07 -- ERROR -- An exception or error occurred in the container during the request processing java.lang.OutOfMemoryError
07 Aug 2005 22:44:41 -- ERROR -- StandardWrapperValve[PartQuery]: Servlet.service() for servlet PartQuery threw exception java.lang.OutOfMemoryError
07 Aug 2005 22:44:51 -- ERROR -- ApplicationDispatcher[] Servlet.service() for servlet jsp threw exception java.lang.OutOfMemoryError
07 Aug 2005 22:44:51 -- ERROR -- org.apache.catalina.core.StandardHostValve@1a313e8: Exception Processing ErrorPage[exceptionType=javax.servlet.ServletException, location=/errPage.jsp]
javax.servlet.ServletException
at org.apache.jasper.servlet.JspServlet.service(JspServlet.java:244)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:810)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:237)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:157)
at org.apache.catalina.core.ApplicationDispatcher.invoke(ApplicationDispatcher.java:703)
at org.apache.catalina.core.ApplicationDispatcher.processRequest(ApplicationDispatcher.java:465)
at org.apache.catalina.core.ApplicationDispatcher.doForward(ApplicationDispatcher.java:398)
at org.apache.catalina.core.ApplicationDispatcher.forward(ApplicationDispatcher.java:312)
at org.apache.catalina.core.StandardHostValve.custom(StandardHostValve.java:396)
at org.apache.catalina.core.StandardHostValve.throwable(StandardHostValve.java:225)
at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:145)
at org.apache.catalina.core.StandardValveContext.invokeNext(StandardValveContext.java:104)
at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:117)
at org.apache.catalina.core.StandardValveContext.invokeNext(StandardValveContext.java:102)
at org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.java:520)
at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109)
at org.apache.catalina.core.StandardValveContext.invokeNext(StandardValveContext.java:104)
at org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.java:520)
at org.apache.catalina.core.ContainerBase.invoke(ContainerBase.java:929)
at org.apache.coyote.tomcat5.CoyoteAdapter.service(CoyoteAdapter.java:160)
at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:799)
at org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.processConnection(Http11Protocol.java:705)
at org.apache.tomcat.util.net.TcpWorkerThread.runIt(PoolTcpEndpoint.java:577)
at org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run(ThreadPool.java:683)
at java.lang.Thread.run(Thread.java:534)
07 Aug 2005 22:44:51 -- ERROR -- ----- Root Cause -----
java.lang.OutOfMemoryError

[namitajain]

Can anyone please HELP me out here...

Thanks again..
N.J

[peterj]

It does sound like the Java heap is slowly filling up with objects. Note that these objects can still be referenced; some code somewhere is probably hanging on to objects that it could release but has not done so. This type of memory leak is very hard to find. Anyway, you are correct in assuming that increasing the heap size will not fix the problem, only delay it. My recommendation is to set up a cron job to reboot the server once a day (in the middle of the night, perhaps).

[namitajain]

Thanks for the reply PeterJ. I had the same idea but if the server is hit very hard one fine day and this OutOfMemory error occurs, say sometime, in the evening then the websites will still be down till midnight (until the cron runs).

Not sure how to handle that.. :(
N.J

[peterj]

There is that, but my advice came from your statement that it takes 4 to 5 days for the heap to fill up, so based on current usage patterns rebooting once a day should avoid the problem. If usage dramatically increases, you will need to either reboot more often or run a second server either on another machine or on a larger machine (dual processor or more). The later depends on if the deployed applications/sites can handle running on multiple servers. Then you can alternate their reboot times so that one is always up.

As a side note, one can configure Microsoft's IIS to reboot an ASP.NET server at a certain interval. I'm not sure if they did this because they were afraid that people would write ASP.NET apps that slowly used up the heap, or if ASP.NET itself slowly eats up the heap.