6 Replies Latest reply on Aug 3, 2006 4:14 PM by scottlance

    JBoss automatic shutdown during boot

    scottlance
    scottlance Aug 2, 2006 5:30 PM

      We are currently setting up JBoss 4.0.3SP1 for deployment. We are using RedHat 3.4.6-2. We are trying to setup JBoss so that it will automatically start during boot time, we've modified the jboss_init_redhat.sh and placed it in /etc/rc.d/init.d and did chkconfig and the whole deal.

      When we reboot the machine, the service will only come up 50-75% of the time, the remainder it seems to come up and then automatically shutdown (see log below). This seems to happen about 30 seconds after JBoss has finsihed starting up. The next message is:

      [org.jboss.deployment.scanner.AbstractDeploymentScanner$ScannerThread] Notified to shutdown

      Has anyone seen this issue before? I've scoured the net and the forums looking for a solution and have yet to find one.

      2006-08-02 11:56:11,249 DEBUG [org.jboss.deployment.MainDeployer] End deployment start on package: jboss-service.xml
      2006-08-02 11:56:11,249 DEBUG [org.jboss.deployment.MainDeployer] Deployed package: file:/usr/jboss/jboss-4.0.3SP1/server/default/conf/jboss-service.xml
      2006-08-02 11:56:11,252 DEBUG [org.jboss.web.tomcat.tc5.Tomcat5] Saw org.jboss.system.server.started notification, starting connectors
      2006-08-02 11:56:11,293 INFO [org.apache.coyote.http11.Http11Protocol] Starting Coyote HTTP/1.1 on http-0.0.0.0-80
      2006-08-02 11:56:11,904 INFO [org.apache.jk.common.ChannelSocket] JK: ajp13 listening on /0.0.0.0:8009
      2006-08-02 11:56:11,922 INFO [org.apache.jk.server.JkMain] Jk running ID=0 time=0/48 config=null
      2006-08-02 11:56:11,937 INFO [org.jboss.system.server.Server] JBoss (MX MicroKernel) [4.0.3SP1 (build: CVSTag=JBoss_4_0_3_SP1 date=200510231054)] Started in 53s:631ms
      2006-08-02 11:56:46,605 DEBUG
      [org.jboss.deployment.scanner.AbstractDeploymentScanner$ScannerThread] Notified to shutdown
      2006-08-02 11:56:46,606 INFO [org.jboss.system.server.Server] Runtime shutdown hook called, forceHalt: true
      2006-08-02 11:56:46,607 INFO [org.jboss.system.server.Server] JBoss SHUTDOWN: Undeploying all packages
      2006-08-02 11:56:46,608 DEBUG [org.jboss.deployment.MainDeployer] Undeploying file:/usr/jboss/jboss-4.0.3SP1/server/default/deploy/jmx-console.war/
      2006-08-02 11:56:46,609 DEBUG [org.jboss.system.ServiceController] stopping service: jboss.web.deployment:war=jmx-console.war,id=-801867523
      2006-08-02 11:56:46,609 DEBUG [org.jboss.system.ServiceController] stopping dependent services for: jboss.web.deployment:war=jmx-console.war,id=-801867523 dependent services are: []
      2006-08-02 11:56:46,610 DEBUG [org.jboss.web.WebModule] Stopping jboss.web.deployment:war=jmx-console.war,id=-801867523
      2006-08-02 11:56:46,610 INFO [org.jboss.web.tomcat.tc5.TomcatDeployer] undeploy, ctxPath=/jmx-console, warUrl=.../deploy/jmx-console.war/
      2006-08-02 11:56:46,636 DEBUG [org.jboss.web.WebModule] Stopped jboss.web.deployment:war=jmx-console.war,id=-801867523
      2006-08-02 11:56:46,637 DEBUG [org.jboss.webservice.ServiceDeployer] handleNotification: org.jboss.deployment.SubDeployer.stop,jmx-console.war
      2006-08-02 11:56:46,638 DEBUG [org.jboss.system.ServiceController] destroying service: jboss.web.deployment:war=jmx-console.war,id=-801867523
      2006-08-02 11:56:46,638 DEBUG [org.jboss.system.ServiceController] destroying dependent services for: jboss.web.deployment:war=jmx-console.war,id=-801867523 dependent services are: []
      2006-08-02 11:56:46,639 DEBUG [org.jboss.web.WebModule] Destroying jboss.web.deployment:war=jmx-console.war,id=-801867523
      2006-08-02 11:56:46,639 DEBUG [org.jboss.web.WebModule] Destroyed jboss.web.deployment:war=jmx-console.war,id=-801867523
      2006-08-02 11:56:46,640 DEBUG [org.jboss.system.ServiceController] removing service: jboss.web.deployment:war=jmx-console.war,id=-801867523
      2006-08-02 11:56:46,640 DEBUG [org.jboss.system.ServiceController] removing already unregistered jboss.web.deployment:war=jmx-console.war,id=-801867523 from server
      2006-08-02 11:56:46,640 DEBUG [org.jboss.webservice.ServiceDeployer] handleNotification: org.jboss.deployment.SubDeployer.destroy,jmx-console.war
      2006-08-02 11:56:46,645 DEBUG [org.jboss.mx.loading.RepositoryClassLoader] Unregistering cl=org.jboss.mx.loading.UnifiedClassLoader3@128340{ url=file:/usr/jboss/jboss-4.0.3SP1/server/default/deploy/jmx-console.war/ ,addedOrder=40}
      2006-08-02 11:56:46,646 DEBUG [org.jboss.mx.loading.UnifiedLoaderRepository3] UnifiedLoaderRepository removed(true) org.jboss.mx.loading.UnifiedClassLoader3@128340{ url=file:/usr/jboss/jboss-4.0.3SP1/server/default/deploy/jmx-console.war/ ,addedOrder=40}


      Thanks,

      Scott Lance
      Interchange Corp.

      • 1952 Views
      • Tags:
        • 1. Re: JBoss automatic shutdown during boot
          peterj
          peterj Aug 2, 2006 5:50 PM (in response to scottlance)

          What changes did you make to the script? I believe that the init process requires a script to return a good result (0) when the script is successful. The 30 second delay sounds like the timeout the init process uses to determine if the initialization script hung.

          Wait a second, by any chance does it take about 30 seconds to start JBoss on your system? And did you change the script such that it doesn't complete unitl the server is up?

            • Actions
          • 2. Re: JBoss automatic shutdown during boot
            scottlance
            scottlance Aug 2, 2006 6:00 PM (in response to scottlance)

            I made the usual changes to the startup script (i.e. JBOSS_HOME, JAVAPTH, JBOSSSH). JBOSSSH got set to: $JBOSS_HOME/bin/run.sh -c default &". JBOSSUS got set to RUNASIS.

            Thats all the changes I made to jboss script.

            Wait a second, by any chance does it take about 30 seconds to start JBoss on your system?


            As you can see in the trace above its taking around 1 minute for JBoss to complete its startup. I'll have to check but when it comes up successfully it takes about the same amount of time.

            And did you change the script such that it doesn't complete unitl the server is up?


            I only made the changes above.

            Thanks for the help.

              • Actions
            • 3. Re: JBoss automatic shutdown during boot
              peterj
              peterj Aug 2, 2006 7:06 PM (in response to scottlance)

              I wonder if the '&' at the end of JBOSSSH is causing a problem. The ampersand should not be needed there since it is later added when the command is finally submitted, at which point it is appended after the redirect of stdout and stderr. Not sure what happens if the ampersand is in font of the redirects.

              I should have looked at the log output more closely. I can see that in about 90 seconds after jboss is started, it is mysteriously being asked to shut down.

              I looked at AbstractDeploymentScanner, and it appears that the message from it only appears if the shutdown request comes while it is in the middle of deploying something. And, of course, the shutdown request is coming in as an INT signal (e.g., ctrl-c). Any chance the app you have deployed might be causing something like this?

                • Actions
              • 4. Re: JBoss automatic shutdown during boot
                scottlance
                scottlance Aug 2, 2006 7:21 PM (in response to scottlance)

                 

                I looked at AbstractDeploymentScanner, and it appears that the message from it only appears if the shutdown request comes while it is in the middle of deploying something. And, of course, the shutdown request is coming in as an INT signal (e.g., ctrl-c). Any chance the app you have deployed might be causing something like this?


                The installation is a standard JBoss distribution. The server that I'm currently testing on doesn't deploy anything that didn't come with the distribution, we are only use that server for messaging.


                  • Actions
                • 5. Re: JBoss automatic shutdown during boot
                  peterj
                  peterj Aug 3, 2006 1:07 PM (in response to scottlance)

                  I am out of ideas. Well, that is not completely true. I would try modifying key JBoss source files to get more logging information about what is going on. That might or might not tell me anything. Ideally, I would want some way to determine who or what caused the INT signal, but I don't know how to get that kind of information out of RedHat. If I knew who sent the INT, I could then try to determine why it was sent.

                  Sorry I couldn't help more.

                    • Actions
                  • 6. Re: JBoss automatic shutdown during boot
                    scottlance
                    scottlance Aug 3, 2006 4:14 PM (in response to scottlance)

                    Its all right, I do appreciate the help that you have offered. Its just kind of weird that it comes up sometime and other times not, while it will never shut down when I launch it manually.

                    Perhaps I'll just write a monitor that will relaunch Jboss when the process goes down or something like that.

                    Thanks again,

                    Scott Lance
                    Interchange Corp.

                      • Actions