8 Replies Latest reply on Sep 29, 2011 7:07 AM by srinu_ammina

    jboss server instance termination.

    srinu_ammina

      Hi,

       

      We are using the jboss 4.0.3 server for running the web application. We are getting this below error while updating the database after some time and jboss server instance is terminating.

       

      Could you please let us know is this the problem is at the Database server or at the application server with the below attached log statements that are last run before terminating the jboss server instance.

       

      Please help us in indetifying the issue w.r.t to logs.

       

      2011-09-23 17:06:22,588 INFO  [STDOUT] java.lang.NullPointerException

      2011-09-23 17:06:22,589 INFO  [STDOUT]  at org.apache.catalina.loader.WebappClassLoader.findResourceInternal(WebappClassLoader.java:1766)

      2011-09-23 17:06:22,589 INFO  [STDOUT]  at org.apache.catalina.loader.WebappClassLoader.findClassInternal(WebappClassLoader.java:1570)

      2011-09-23 17:06:22,589 INFO  [STDOUT]  at org.apache.catalina.loader.WebappClassLoader.findClass(WebappClassLoader.java:850)

      2011-09-23 17:06:22,589 INFO  [STDOUT]  at org.apache.catalina.loader.WebappClassLoader.loadClass(WebappClassLoader.java:1299)

      2011-09-23 17:06:22,589 INFO  [STDOUT]  at org.apache.catalina.loader.WebappClassLoader.loadClass(WebappClassLoader.java:1181)

      2011-09-23 17:06:22,589 INFO  [STDOUT]  at java.lang.ClassLoader.loadClassInternal(ClassLoader.java:319)

      2011-09-23 17:06:22,589 INFO  [STDOUT]  at java.lang.Class.forName0(Native Method)

      2011-09-23 17:06:22,589 INFO  [STDOUT]  at java.lang.Class.forName(Class.java:164)

      2011-09-23 17:06:22,589 INFO  [STDOUT]  at org.hibernate.util.ReflectHelper.classForName(ReflectHelper.java:108)

      2011-09-23 17:06:22,589 INFO  [STDOUT]  at org.hibernate.impl.SessionFactoryImpl.getImportedClassName(SessionFactoryImpl.java:745)

      2011-09-23 17:06:22,589 INFO  [STDOUT]  at org.hibernate.hql.QuerySplitter.getImportedClass(QuerySplitter.java:130)

      2011-09-23 17:06:22,589 INFO  [STDOUT]  at org.hibernate.hql.QuerySplitter.concreteQueries(QuerySplitter.java:96)

      2011-09-23 17:06:22,589 INFO  [STDOUT]  at org.hibernate.impl.SessionFactoryImpl.getQuery(SessionFactoryImpl.java:422)

      2011-09-23 17:06:22,589 INFO  [STDOUT]  at org.hibernate.impl.SessionImpl.getQueries(SessionImpl.java:884)

      2011-09-23 17:06:22,589 INFO  [STDOUT]  at org.hibernate.impl.SessionImpl.executeUpdate(SessionImpl.java:865)

      2011-09-23 17:06:22,589 INFO  [STDOUT]  at org.hibernate.impl.QueryImpl.executeUpdate(QueryImpl.java:89)

      2011-09-23 17:06:22,589 INFO  [STDOUT]  at com.tcs.telecom.ppm.util.DBAccess.updatetaridslatableApprv(DBAccess.java:39778)

        • 1. Re: jboss server instance termination.
          jfclere

          Weird... Can there be something like too many files open errors at the same time?

          • 2. Re: jboss server instance termination.
            srinu_ammina

            Hi,

             

            Not exactly too many files open error but it is related to

             

            2011-08-31 16:07:30,420 WARN  [org.apache.catalina.loader.WebappClassLoader] Failed to open JAR
            java.util.zip.ZipException: error in opening zip file

            But here we are not trying to open any zip file.

            Please find the below entire thread of log for the same. Please guide us what would be the reason and how to solve the issue, thank you for responding.

             

             

            -----------------------------------------------------------------

             

            2011-08-31 16:07:30,420 WARN  [org.apache.catalina.loader.WebappClassLoader] Failed to open JAR
            java.util.zip.ZipException: error in opening zip file
            at java.util.zip.ZipFile.open(Native Method)
            at java.util.zip.ZipFile.<init>(ZipFile.java:203)
            at java.util.jar.JarFile.<init>(JarFile.java:132)
            at java.util.jar.JarFile.<init>(JarFile.java:97)
            at org.apache.catalina.loader.WebappClassLoader.openJARs(WebappClassLoader.java:1544)
            at org.apache.catalina.loader.WebappClassLoader.findResourceInternal(WebappClassLoader.java:1763)
            at org.apache.catalina.loader.WebappClassLoader.findClassInternal(WebappClassLoader.java:1570)
            at org.apache.catalina.loader.WebappClassLoader.findClass(WebappClassLoader.java:850)
            at org.apache.catalina.loader.WebappClassLoader.loadClass(WebappClassLoader.java:1299)
            at org.apache.catalina.loader.WebappClassLoader.loadClass(WebappClassLoader.java:1181)
            at org.hibernate.util.ReflectHelper.classForName(ReflectHelper.java:102)
            at org.hibernate.impl.SessionFactoryImpl.getImportedClassName(SessionFactoryImpl.java:745)
            at org.hibernate.hql.QuerySplitter.getImportedClass(QuerySplitter.java:130)
            at org.hibernate.hql.QuerySplitter.concreteQueries(QuerySplitter.java:96)
            at org.hibernate.impl.SessionFactoryImpl.getQuery(SessionFactoryImpl.java:422)
            at org.hibernate.impl.SessionImpl.getQueries(SessionImpl.java:884)
            at org.hibernate.impl.SessionImpl.executeUpdate(SessionImpl.java:865)
            at org.hibernate.impl.QueryImpl.executeUpdate(QueryImpl.java:89)
            at com.tcs.telecom.ppm.util.DBAccess.updatePPMTableTwoFields(DBAccess.java:35702)
            at com.tcs.telecom.ppm.actions.ExcelAction.getFeasible(ExcelAction.java:2152)
            at sun.reflect.GeneratedMethodAccessor3435.invoke(Unknown Source)
            at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
            at java.lang.reflect.Method.invoke(Method.java:585)
            at org.apache.struts.actions.DispatchAction.dispatchMethod(DispatchAction.java:274)
            at org.apache.struts.actions.LookupDispatchAction.execute(LookupDispatchAction.java:160)
            at org.apache.struts.action.RequestProcessor.processActionPerform(RequestProcessor.java:419)
            at org.apache.struts.action.RequestProcessor.process(RequestProcessor.java:224)
            at org.apache.struts.action.ActionServlet.process(ActionServlet.java:1194)
            at org.apache.struts.action.ActionServlet.doPost(ActionServlet.java:432)
            at javax.servlet.http.HttpServlet.service(HttpServlet.java:717)
            at javax.servlet.http.HttpServlet.service(HttpServlet.java:810)
            at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:252)
            at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:173)
            at org.apache.catalina.core.ApplicationDispatcher.invoke(ApplicationDispatcher.java:672)
            at org.apache.catalina.core.ApplicationDispatcher.processRequest(ApplicationDispatcher.java:463)
            at org.apache.catalina.core.ApplicationDispatcher.doForward(ApplicationDispatcher.java:398)
            at org.apache.catalina.core.ApplicationDispatcher.forward(ApplicationDispatcher.java:301)
            at com.tcs.telecom.ppm.filter.LoginFilter.doFilter(LoginFilter.java:83)
            at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:202)
            at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:173)
            at org.jboss.web.tomcat.filters.ReplyHeaderFilter.doFilter(ReplyHeaderFilter.java:81)
            at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:202)
            at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:173)
            at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:213)
            at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:178)
            at org.jboss.web.tomcat.security.CustomPrincipalValve.invoke(CustomPrincipalValve.java:39)
            at org.jboss.web.tomcat.security.SecurityAssociationValve.invoke(SecurityAssociationValve.java:159)
            at org.jboss.web.tomcat.security.JaccContextValve.invoke(JaccContextValve.java:59)
            at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:126)
            at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:105)
            at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:107)
            at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:148)
            at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:856)
            at org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.processConnection(Http11Protocol.java:744)
            at org.apache.tomcat.util.net.PoolTcpEndpoint.processSocket(PoolTcpEndpoint.java:527)
            at org.apache.tomcat.util.net.MasterSlaveWorkerThread.run(MasterSlaveWorkerThread.java:112)
            at java.lang.Thread.run(Thread.java:595)
            2011-08-31 16:07:30,424 WARN  [org.apache.catalina.loader.WebappClassLoader] Failed to open JAR
            java.util.zip.ZipException: error in opening zip file
            at java.util.zip.ZipFile.open(Native Method)
            at java.util.zip.ZipFile.<init>(ZipFile.java:203)
            at java.util.jar.JarFile.<init>(JarFile.java:132)
            at java.util.jar.JarFile.<init>(JarFile.java:97)
            at org.apache.catalina.loader.WebappClassLoader.openJARs(WebappClassLoader.java:1544)
            at org.apache.catalina.loader.WebappClassLoader.findResourceInternal(WebappClassLoader.java:1763)
            at org.apache.catalina.loader.WebappClassLoader.findClassInternal(WebappClassLoader.java:1570)
            at org.apache.catalina.loader.WebappClassLoader.findClass(WebappClassLoader.java:850)
            at org.apache.catalina.loader.WebappClassLoader.loadClass(WebappClassLoader.java:1299)
            at org.apache.catalina.loader.WebappClassLoader.loadClass(WebappClassLoader.java:1181)
            at org.hibernate.util.ReflectHelper.classForName(ReflectHelper.java:102)
            at org.hibernate.impl.SessionFactoryImpl.getImportedClassName(SessionFactoryImpl.java:745)
            at org.hibernate.hql.QuerySplitter.getImportedClass(QuerySplitter.java:130)
            at org.hibernate.hql.QuerySplitter.concreteQueries(QuerySplitter.java:96)
            at org.hibernate.impl.SessionFactoryImpl.getQuery(SessionFactoryImpl.java:422)
            at org.hibernate.impl.SessionImpl.getQueries(SessionImpl.java:884)
            at org.hibernate.impl.SessionImpl.executeUpdate(SessionImpl.java:865)
            at org.hibernate.impl.QueryImpl.executeUpdate(QueryImpl.java:89)
            at com.tcs.telecom.ppm.util.DBAccess.updatePPMTableTwoFields(DBAccess.java:35702)
            at com.tcs.telecom.ppm.actions.ExcelAction.getFeasible(ExcelAction.java:2152)
            at sun.reflect.GeneratedMethodAccessor3435.invoke(Unknown Source)
            at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
            at java.lang.reflect.Method.invoke(Method.java:585)
            at org.apache.struts.actions.DispatchAction.dispatchMethod(DispatchAction.java:274)
            at org.apache.struts.actions.LookupDispatchAction.execute(LookupDispatchAction.java:160)
            at org.apache.struts.action.RequestProcessor.processActionPerform(RequestProcessor.java:419)
            at org.apache.struts.action.RequestProcessor.process(RequestProcessor.java:224)
            at org.apache.struts.action.ActionServlet.process(ActionServlet.java:1194)
            at org.apache.struts.action.ActionServlet.doPost(ActionServlet.java:432)
            at javax.servlet.http.HttpServlet.service(HttpServlet.java:717)
            at javax.servlet.http.HttpServlet.service(HttpServlet.java:810)
            at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:252)
            at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:173)
            at org.apache.catalina.core.ApplicationDispatcher.invoke(ApplicationDispatcher.java:672)
            at org.apache.catalina.core.ApplicationDispatcher.processRequest(ApplicationDispatcher.java:463)
            at org.apache.catalina.core.ApplicationDispatcher.doForward(ApplicationDispatcher.java:398)
            at org.apache.catalina.core.ApplicationDispatcher.forward(ApplicationDispatcher.java:301)
            at com.tcs.telecom.ppm.filter.LoginFilter.doFilter(LoginFilter.java:83)
            at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:202)
            at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:173)
            at org.jboss.web.tomcat.filters.ReplyHeaderFilter.doFilter(ReplyHeaderFilter.java:81)
            at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:202)
            at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:173)
            at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:213)
            at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:178)
            at org.jboss.web.tomcat.security.CustomPrincipalValve.invoke(CustomPrincipalValve.java:39)
            at org.jboss.web.tomcat.security.SecurityAssociationValve.invoke(SecurityAssociationValve.java:159)
            at org.jboss.web.tomcat.security.JaccContextValve.invoke(JaccContextValve.java:59)
            at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:126)
            at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:105)
            at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:107)
            at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:148)
            at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:856)
            at org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.processConnection(Http11Protocol.java:744)
            at org.apache.tomcat.util.net.PoolTcpEndpoint.processSocket(PoolTcpEndpoint.java:527)
            at org.apache.tomcat.util.net.MasterSlaveWorkerThread.run(MasterSlaveWorkerThread.java:112)
            at java.lang.Thread.run(Thread.java:595)
            2011-08-31 16:07:30,429 WARN  [org.apache.catalina.loader.WebappClassLoader] Failed to open JAR
            java.util.zip.ZipException: error in opening zip file
            at java.util.zip.ZipFile.open(Native Method)
            at java.util.zip.ZipFile.<init>(ZipFile.java:203)
            at java.util.jar.JarFile.<init>(JarFile.java:132)
            at java.util.jar.JarFile.<init>(JarFile.java:97)
            at org.apache.catalina.loader.WebappClassLoader.openJARs(WebappClassLoader.java:1544)
            at org.apache.catalina.loader.WebappClassLoader.findResourceInternal(WebappClassLoader.java:1763)
            at org.apache.catalina.loader.WebappClassLoader.findClassInternal(WebappClassLoader.java:1570)
            at org.apache.catalina.loader.WebappClassLoader.findClass(WebappClassLoader.java:850)
            at org.apache.catalina.loader.WebappClassLoader.loadClass(WebappClassLoader.java:1299)
            at org.apache.catalina.loader.WebappClassLoader.loadClass(WebappClassLoader.java:1181)
            at org.hibernate.util.ReflectHelper.classForName(ReflectHelper.java:102)
            at org.hibernate.impl.SessionFactoryImpl.getImportedClassName(SessionFactoryImpl.java:745)
            at org.hibernate.hql.QuerySplitter.getImportedClass(QuerySplitter.java:130)
            at org.hibernate.hql.QuerySplitter.concreteQueries(QuerySplitter.java:96)
            at org.hibernate.impl.SessionFactoryImpl.getQuery(SessionFactoryImpl.java:422)
            at org.hibernate.impl.SessionImpl.getQueries(SessionImpl.java:884)
            at org.hibernate.impl.SessionImpl.executeUpdate(SessionImpl.java:865)
            at org.hibernate.impl.QueryImpl.executeUpdate(QueryImpl.java:89)
            at com.tcs.telecom.ppm.util.DBAccess.updatePPMTableTwoFields(DBAccess.java:35702)
            at com.tcs.telecom.ppm.actions.ExcelAction.getFeasible(ExcelAction.java:2152)
            at sun.reflect.GeneratedMethodAccessor3435.invoke(Unknown Source)
            at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
            at java.lang.reflect.Method.invoke(Method.java:585)
            at org.apache.struts.actions.DispatchAction.dispatchMethod(DispatchAction.java:274)
            at org.apache.struts.actions.LookupDispatchAction.execute(LookupDispatchAction.java:160)
            at org.apache.struts.action.RequestProcessor.processActionPerform(RequestProcessor.java:419)
            at org.apache.struts.action.RequestProcessor.process(RequestProcessor.java:224)
            at org.apache.struts.action.ActionServlet.process(ActionServlet.java:1194)
            at org.apache.struts.action.ActionServlet.doPost(ActionServlet.java:432)
            at javax.servlet.http.HttpServlet.service(HttpServlet.java:717)
            at javax.servlet.http.HttpServlet.service(HttpServlet.java:810)
            at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:252)
            at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:173)
            at org.apache.catalina.core.ApplicationDispatcher.invoke(ApplicationDispatcher.java:672)
            at org.apache.catalina.core.ApplicationDispatcher.processRequest(ApplicationDispatcher.java:463)
            at org.apache.catalina.core.ApplicationDispatcher.doForward(ApplicationDispatcher.java:398)
            at org.apache.catalina.core.ApplicationDispatcher.forward(ApplicationDispatcher.java:301)
            at com.tcs.telecom.ppm.filter.LoginFilter.doFilter(LoginFilter.java:83)
            at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:202)
            at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:173)
            at org.jboss.web.tomcat.filters.ReplyHeaderFilter.doFilter(ReplyHeaderFilter.java:81)
            at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:202)
            at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:173)
            at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:213)
            at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:178)
            at org.jboss.web.tomcat.security.CustomPrincipalValve.invoke(CustomPrincipalValve.java:39)
            at org.jboss.web.tomcat.security.SecurityAssociationValve.invoke(SecurityAssociationValve.java:159)
            at org.jboss.web.tomcat.security.JaccContextValve.invoke(JaccContextValve.java:59)
            at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:126)
            at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:105)
            at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:107)
            at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:148)
            at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:856)
            at org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.processConnection(Http11Protocol.java:744)
            at org.apache.tomcat.util.net.PoolTcpEndpoint.processSocket(PoolTcpEndpoint.java:527)
            at org.apache.tomcat.util.net.MasterSlaveWorkerThread.run(MasterSlaveWorkerThread.java:112)
            at java.lang.Thread.run(Thread.java:595)
            2011-08-31 16:07:30,430 WARN  [org.apache.catalina.loader.WebappClassLoader] Failed to open JAR

             

            -----------------------------------------------------------------

            • 3. Re: jboss server instance termination.
              jfclere

              "Not exactly too many files open error but it is related to"

              How can you be so sure of that?

              If the "Failed to open JAR" started we you "updated the database" then what is "updating the database" exactly?

              • 4. Re: jboss server instance termination.
                srinu_ammina

                Sorry, in all most all the incidents if we check the last logged logs it is displaying "Failed to open JAR" messages followed by the database updates.

                But not sure why this issue is happening. Please guide us how to investigate and dig further on this issue.

                • 5. Re: jboss server instance termination.
                  jfclere

                  what is "updating the database" exactly?

                  • 6. Re: jboss server instance termination.
                    srinu_ammina

                    We are updating two fields in the database and the exact code for the updating code is:

                     

                     

                     

                    public static void

                     

                    updatePPMTableTwoFields(String tableName, String fieldName1,

                    String fieldValue1, String fieldName2, String fieldValue2,String fieldName3, String fieldValue3)

                    throws Exception{

                    Session session=

                    null;

                     

                     

                    try

                    {

                     

                    HibernateTemplate hibernateTemplate = (HibernateTemplate) PPMSpringInitializer.

                    appContext.getBean(PPMConfigurationConstants.PPM_HIBERNATE_TEMPLATE);

                     

                    session= hibernateTemplate.getSessionFactory().openSession();

                     

                     

                     

                    int rows = session.createQuery("update " + tableName + " set "+ fieldName1 + "='"+ fieldValue1+"' , "+ fieldName2 + "='"

                    + fieldValue2+"' where "+ fieldName3+ "='" + fieldValue3+ "'").executeUpdate();

                     

                    System.out.println(rows + " rows updated"

                    );

                    catch

                    (Exception e) {

                    System.out.println("Exception in Updating PPM Tables$$$$$$$$$"

                    +e);

                     

                    e.printStackTrace();

                     

                    throw e;

                     

                    }

                     

                       finally

                     

                    {

                    session.close();

                    }

                    }

                     

                     

                    After this update statements, instance is teminating. So if we observe the logs in all the cases, when the instance is terminated it is logging the message with "Failed to open jar" followed by the update statements as above. 

                     

                     

                     

                     

                     

                     

                     

                     

                     

                     

                     

                     

                     

                     

                     

                     

                     

                     

                     

                    • 7. Re: jboss server instance termination.
                      jfclere

                      So you change a jar of a running application, correct?

                      • 8. Re: jboss server instance termination.
                        srinu_ammina

                        No we are not touching and changing any of the jars related to the running application in the jboss server.

                        Another information is, it is not happening all the times in the same database update method and we found there are some cases where the database updates even passed in the same method call.

                         

                        But after some iterations, suddenly server instance is terminating.