Jboss does not start after OS reboot with Runtime shutdown hook called, forceHalt: true
All,
I inherited this JBoss environment 4.0.3SP1. Our service provider asked us to reboot the server for maintenance. The server had not been rebooted for a year. After the reboot, we cannot start the JBoss application server anymore. Everytime we get these messages (boot.log) and the process gets killed. There is nothing in dmesg, messages showing a memory issue. Please help with any suggestions.
We are running the run.sh & to start the server. User used to start is root
Thanks
Jishnu
02:10:25,064 INFO [Server] Starting JBoss (MX MicroKernel)...
02:10:25,066 INFO [Server] Release ID: JBoss [Zion] 4.0.3SP1 (build: CVSTag=JBoss_4_0_3_SP1 date=200510231751)
02:10:25,066 DEBUG [Server] Using config: org.jboss.system.server.ServerConfigImpl@6e70c7
02:10:25,066 DEBUG [Server] Server type: class org.jboss.system.server.ServerImpl
02:10:25,066 DEBUG [Server] Server loaded through: org.jboss.system.server.NoAnnotationURLClassLoader
02:10:25,066 DEBUG [Server] Boot URLs:
02:10:25,067 DEBUG [Server] file:/usr/eko/jboss-4.0.3SP1/lib/endorsed/xml-apis.jar
02:10:25,067 DEBUG [Server] file:/usr/eko/jboss-4.0.3SP1/lib/endorsed/resolver.jar
02:10:25,067 DEBUG [Server] file:/usr/eko/jboss-4.0.3SP1/lib/endorsed/xercesImpl.jar
02:10:25,067 DEBUG [Server] file:/usr/eko/jboss-4.0.3SP1/lib/endorsed/xalan.jar
02:10:25,067 DEBUG [Server] file:/usr/eko/jboss-4.0.3SP1/lib/jboss-jmx.jar
02:10:25,067 DEBUG [Server] file:/usr/eko/jboss-4.0.3SP1/lib/dom4j.jar
02:10:25,067 DEBUG [Server] file:/usr/eko/jboss-4.0.3SP1/lib/jaxen.jar
02:10:25,067 DEBUG [Server] file:/usr/eko/jboss-4.0.3SP1/lib/gnu-regexp.jar
02:10:25,067 DEBUG [Server] file:/usr/eko/jboss-4.0.3SP1/lib/concurrent.jar
02:10:25,067 DEBUG [Server] file:/usr/eko/jboss-4.0.3SP1/lib/log4j-boot.jar
02:10:25,067 DEBUG [Server] file:/usr/eko/jboss-4.0.3SP1/lib/jboss-common.jar
02:10:25,067 DEBUG [Server] file:/usr/eko/jboss-4.0.3SP1/lib/jboss-system.jar
02:10:25,067 DEBUG [Server] file:/usr/eko/jboss-4.0.3SP1/lib/namespace.jar
02:10:25,067 INFO [Server] Home Dir: /usr/eko/jboss-4.0.3SP1
02:10:25,068 INFO [Server] Home URL: file:/usr/eko/jboss-4.0.3SP1/
02:10:25,068 DEBUG [Server] Library URL: file:/usr/eko/jboss-4.0.3SP1/lib/
02:10:25,069 INFO [Server] Patch URL: null
02:10:25,069 INFO [Server] Server Name: default
02:10:25,069 INFO [Server] Server Home Dir: /usr/eko/jboss-4.0.3SP1/server/default
02:10:25,069 INFO [Server] Server Home URL: file:/usr/eko/jboss-4.0.3SP1/server/default/
02:10:25,070 DEBUG [Server] Server Data Dir: /usr/eko/jboss-4.0.3SP1/server/default/data
02:10:25,070 INFO [Server] Server Temp Dir: /usr/eko/jboss-4.0.3SP1/server/default/tmp
02:10:25,070 DEBUG [Server] Server Config URL: file:/usr/eko/jboss-4.0.3SP1/server/default/conf/
02:10:25,070 DEBUG [Server] Server Library URL: file:/usr/eko/jboss-4.0.3SP1/server/default/lib/
02:10:25,070 INFO [Server] Root Deployment Filename: jboss-service.xml
02:10:25,073 DEBUG [Server] Starting General Purpose Architecture (GPA)...
02:10:25,308 DEBUG [Server] Created MBeanServer: org.jboss.mx.server.MBeanServerImpl@1968e23[ defaultDomain='jboss' ]
02:10:25,390 DEBUG [Server] Boot url list: [file:/usr/eko/jboss-4.0.3SP1/server/default/conf/]
02:10:25,407 DEBUG [Server] Creating loader for URL: file:/usr/eko/jboss-4.0.3SP1/server/default/conf/
02:10:25,420 DEBUG [UnifiedClassLoader] New jmx UCL with url file:/usr/eko/jboss-4.0.3SP1/server/default/conf/
02:10:25,420 DEBUG [RepositoryClassLoader] setRepository, repository=org.jboss.mx.loading.UnifiedLoaderRepository3@111a3a4, cl=org.jboss.mx.loading.UnifiedClassLoader3@10e790c{ url=file:/usr/eko/jboss-4.0.3SP1/server/default/conf/ ,addedOrder=0}
02:10:25,420 DEBUG [RepositoryClassLoader] setRepository, repository=org.jboss.mx.loading.UnifiedLoaderRepository3@111a3a4, cl=org.jboss.mx.loading.UnifiedClassLoader3@10e790c{ url=file:/usr/eko/jboss-4.0.3SP1/server/default/conf/ ,addedOrder=0}
02:10:25,421 DEBUG [UnifiedLoaderRepository3] Adding org.jboss.mx.loading.UnifiedClassLoader3@10e790c{ url=file:/usr/eko/jboss-4.0.3SP1/server/default/conf/ ,addedOrder=0}
02:10:25,750 DEBUG [Server] Failed to create xmbean for: org.jboss.system.server.ServerInfo
02:10:25,773 INFO [ServerInfo] Java version: 1.5.0_14,Sun Microsystems Inc.
02:10:25,773 INFO [ServerInfo] Java VM: Java HotSpot(TM) Server VM 1.5.0_14-b03,Sun Microsystems Inc.
02:10:25,773 INFO [ServerInfo] OS-System: Linux 2.6.9-89.ELsmp,i386
02:10:25,773 DEBUG [ServerInfo] Full System Properties Dump
02:10:25,774 DEBUG [ServerInfo] java.runtime.name: Java(TM) 2 Runtime Environment, Standard Edition
02:10:25,774 DEBUG [ServerInfo] jboss.server.base.dir: /usr/eko/jboss-4.0.3SP1/server
02:10:25,774 DEBUG [ServerInfo] java.protocol.handler.pkgs: org.jboss.net.protocol
02:10:25,774 DEBUG [ServerInfo] sun.boot.library.path: /usr/java/jdk1.5.0_14/jre/lib/i386
02:10:25,774 DEBUG [ServerInfo] jboss.server.lib.url: file:/usr/eko/jboss-4.0.3SP1/server/default/lib/
02:10:25,774 DEBUG [ServerInfo] java.vm.version: 1.5.0_14-b03
02:10:25,774 DEBUG [ServerInfo] javax.management.builder.initial: org.jboss.mx.server.MBeanServerBuilderImpl
02:10:25,774 DEBUG [ServerInfo] java.vm.vendor: Sun Microsystems Inc.
02:10:25,774 DEBUG [ServerInfo] java.vendor.url: http://java.sun.com/
02:10:25,774 DEBUG [ServerInfo] path.separator: :
02:10:25,774 DEBUG [ServerInfo] java.vm.name: Java HotSpot(TM) Server VM
02:10:25,774 DEBUG [ServerInfo] file.encoding.pkg: sun.io
02:10:25,775 DEBUG [ServerInfo] user.country: US
02:10:25,775 DEBUG [ServerInfo] sun.java.launcher: SUN_STANDARD
02:10:25,775 DEBUG [ServerInfo] sun.os.patch.level: unknown
02:10:25,775 DEBUG [ServerInfo] program.name: run.sh
02:10:25,775 DEBUG [ServerInfo] java.vm.specification.name: Java Virtual Machine Specification
02:10:25,775 DEBUG [ServerInfo] user.dir: /usr/eko/jboss-4.0.3SP1/bin
02:10:25,775 DEBUG [ServerInfo] jboss.server.base.url: file:/usr/eko/jboss-4.0.3SP1/server/
02:10:25,775 DEBUG [ServerInfo] java.runtime.version: 1.5.0_14-b03
02:10:25,775 DEBUG [ServerInfo] java.awt.graphicsenv: sun.awt.X11GraphicsEnvironment
02:10:25,775 DEBUG [ServerInfo] java.endorsed.dirs: /usr/eko/jboss-4.0.3SP1/lib/endorsed
02:10:25,775 DEBUG [ServerInfo] os.arch: i386
02:10:25,775 DEBUG [ServerInfo] java.io.tmpdir: /tmp
02:10:25,775 DEBUG [ServerInfo] line.separator:
02:10:25,775 DEBUG [ServerInfo] jbossmx.loader.repository.class: org.jboss.mx.loading.UnifiedLoaderRepository3
02:10:25,775 DEBUG [ServerInfo] java.vm.specification.vendor: Sun Microsystems Inc.
02:10:25,775 DEBUG [ServerInfo] os.name: Linux
02:10:25,776 DEBUG [ServerInfo] jboss.bind.address: 0.0.0.0
02:10:25,776 DEBUG [ServerInfo] jboss.server.temp.dir: /usr/eko/jboss-4.0.3SP1/server/default/tmp
02:10:25,776 DEBUG [ServerInfo] jboss.home.dir: /usr/eko/jboss-4.0.3SP1
02:10:25,776 DEBUG [ServerInfo] sun.jnu.encoding: UTF-8
02:10:25,776 DEBUG [ServerInfo] java.library.path: /usr/java/jdk1.5.0_14/jre/lib/i386/server:/usr/java/jdk1.5.0_14/jre/lib/i386:/usr/java/jdk1.5.0_14/jre/../lib/i386:/usr/eko/jboss-4.0.3SP1/cyberPlatConfig/lib
02:10:25,776 DEBUG [ServerInfo] jboss.server.home.dir: /usr/eko/jboss-4.0.3SP1/server/default
02:10:25,776 DEBUG [ServerInfo] java.class.version: 49.0
02:10:25,776 DEBUG [ServerInfo] java.specification.name: Java Platform API Specification
02:10:25,776 DEBUG [ServerInfo] sun.management.compiler: HotSpot Server Compiler
02:10:25,776 DEBUG [ServerInfo] jboss.server.config.url: file:/usr/eko/jboss-4.0.3SP1/server/default/conf/
02:10:25,776 DEBUG [ServerInfo] os.version: 2.6.9-89.ELsmp
02:10:25,776 DEBUG [ServerInfo] jboss.home.url: file:/usr/eko/jboss-4.0.3SP1/
02:10:25,776 DEBUG [ServerInfo] user.home: /root
02:10:25,776 DEBUG [ServerInfo] user.timezone: GMT+05:30
02:10:25,776 DEBUG [ServerInfo] java.awt.printerjob: sun.print.PSPrinterJob
02:10:25,776 DEBUG [ServerInfo] java.specification.version: 1.5
02:10:25,776 DEBUG [ServerInfo] file.encoding: UTF-8
02:10:25,777 DEBUG [ServerInfo] jboss.server.home.url: file:/usr/eko/jboss-4.0.3SP1/server/default/
02:10:25,777 DEBUG [ServerInfo] user.name: root
02:10:25,777 DEBUG [ServerInfo] java.class.path: /usr/eko/jboss-4.0.3SP1/bin/run.jar:/usr/java/jdk1.5.0_14/lib/tools.jar
02:10:25,777 DEBUG [ServerInfo] jboss.lib.url: file:/usr/eko/jboss-4.0.3SP1/lib/
02:10:25,777 DEBUG [ServerInfo] jboss.server.name: default
02:10:25,777 DEBUG [ServerInfo] java.vm.specification.version: 1.0
02:10:25,777 DEBUG [ServerInfo] sun.arch.data.model: 32
02:10:25,777 DEBUG [ServerInfo] java.home: /usr/java/jdk1.5.0_14/jre
02:10:25,777 DEBUG [ServerInfo] java.specification.vendor: Sun Microsystems Inc.
02:10:25,777 DEBUG [ServerInfo] user.language: en
02:10:25,777 DEBUG [ServerInfo] java.vm.info: mixed mode
02:10:25,777 DEBUG [ServerInfo] java.version: 1.5.0_14
02:10:25,777 DEBUG [ServerInfo] java.ext.dirs: /usr/java/jdk1.5.0_14/jre/lib/ext
02:10:25,777 DEBUG [ServerInfo] jboss.server.data.dir: /usr/eko/jboss-4.0.3SP1/server/default/data
02:10:25,777 DEBUG [ServerInfo] sun.boot.class.path: /usr/eko/jboss-4.0.3SP1/lib/endorsed/xml-apis.jar:/usr/eko/jboss-4.0.3SP1/lib/endorsed/resolver.jar:/usr/eko/jboss-4.0.3SP1/lib/endorsed/xercesImpl.jar:/usr/eko/jboss-4.0.3SP1/lib/endorsed/xalan.jar:/usr/java/jdk1.5.0_14/jre/lib/rt.jar:/usr/java/jdk1.5.0_14/jre/lib/i18n.jar:/usr/java/jdk1.5.0_14/jre/lib/sunrsasign.jar:/usr/java/jdk1.5.0_14/jre/lib/jsse.jar:/usr/java/jdk1.5.0_14/jre/lib/jce.jar:/usr/java/jdk1.5.0_14/jre/lib/charsets.jar:/usr/java/jdk1.5.0_14/jre/classes
02:10:25,778 DEBUG [ServerInfo] java.vendor: Sun Microsystems Inc.
02:10:25,778 DEBUG [ServerInfo] file.separator: /
02:10:25,778 DEBUG [ServerInfo] java.vendor.url.bug: http://java.sun.com/cgi-bin/bugreport.cgi
02:10:25,778 DEBUG [ServerInfo] sun.cpu.endian: little
02:10:25,778 DEBUG [ServerInfo] sun.io.unicode.encoding: UnicodeLittle
02:10:25,778 DEBUG [ServerInfo] sun.cpu.isalist:
02:10:25,807 DEBUG [Server] Created system MBean: jboss.system:type=ServerInfo
02:10:25,815 DEBUG [Server] Failed to create xmbean for: org.jboss.system.ServiceController
02:10:25,857 DEBUG [ServiceController] Controller MBean online
02:10:25,857 DEBUG [Server] Created system MBean: jboss.system:service=ServiceController
02:10:26,163 DEBUG [Server] Created system XMBean: jboss.system:service=MainDeployer
02:10:26,163 DEBUG [ServiceController] Creating service jboss.system:service=MainDeployer
02:10:26,181 DEBUG [MainDeployer] Creating jboss.system:service=MainDeployer
02:10:26,223 DEBUG [MainDeployer] Created jboss.system:service=MainDeployer
02:10:26,223 DEBUG [ServiceController] Creating dependent components for: jboss.system:service=MainDeployer dependents are: []
02:10:26,224 DEBUG [ServiceController] starting service jboss.system:service=MainDeployer
02:10:26,224 DEBUG [MainDeployer] Starting jboss.system:service=MainDeployer
02:10:26,224 DEBUG [MainDeployer] Started jboss.system:service=MainDeployer
02:10:26,224 DEBUG [ServiceController] Starting dependent components for: jboss.system:service=MainDeployer dependent components: []
02:10:26,225 DEBUG [Server] Shutdown hook added
02:10:26,392 INFO [Server] Runtime shutdown hook called, forceHalt: true
02:10:26,393 INFO [Server] JBoss SHUTDOWN: Undeploying all packages
02:10:26,393 DEBUG [MainDeployer] Undeployed 0 deployed packages
02:10:26,393 DEBUG [Server] Shutting down all services
02:10:26,394 DEBUG [ServiceController] Stopping 2 services
02:10:26,394 DEBUG [ServiceController] removing service: jboss.system:service=MainDeployer
02:10:26,394 DEBUG [ServiceController] removing jboss.system:service=MainDeployer from server
02:10:26,558 DEBUG [Server] Created system XMBean: jboss.system:service=JARDeployer
02:10:26,559 DEBUG [ServiceController] Creating service jboss.system:service=JARDeployer
02:10:26,559 DEBUG [JARDeployer] Creating jboss.system:service=JARDeployer
02:10:26,641 DEBUG [JARDeployer] Created jboss.system:service=JARDeployer
02:10:26,641 DEBUG [ServiceController] Creating dependent components for: jboss.system:service=JARDeployer dependents are: []
02:10:26,641 DEBUG [ServiceController] starting service jboss.system:service=JARDeployer
02:10:26,641 DEBUG [JARDeployer] Starting jboss.system:service=JARDeployer
02:10:26,642 DEBUG [MainDeployer] Adding deployer: org.jboss.deployment.JARDeployer@c09554
02:10:26,642 DEBUG [SuffixOrderHelper] Static suffix exists; ignoring request for adding enhanced suffix: 758.jar
02:10:26,642 DEBUG [JARDeployer] Started jboss.system:service=JARDeployer
02:10:26,642 DEBUG [ServiceController] Starting dependent components for: jboss.system:service=JARDeployer dependent components: []
02:10:26,712 DEBUG [Server] Created system XMBean: jboss.system:service=ServiceDeployer
02:10:26,712 DEBUG [ServiceController] Creating service jboss.system:service=ServiceDeployer
02:10:26,712 DEBUG [SARDeployer] Creating jboss.system:service=ServiceDeployer
02:10:26,719 DEBUG [ServiceController] Ignoring request to remove nonexistent service: jboss.system:service=MainDeployer
02:10:26,720 DEBUG [ServiceController] Stopped 1 services
02:10:26,720 DEBUG [Server] Deleting server tmp/deploy directory
02:10:26,721 INFO [Server] Shutdown complete
02:10:26,727 DEBUG [SARDeployer] Created jboss.system:service=ServiceDeployer
02:10:26,727 DEBUG [ServiceController] Creating dependent components for: jboss.system:service=ServiceDeployer dependents are: []
02:10:26,728 DEBUG [Server] Failed to start
javax.management.InstanceNotFoundException: jboss.system:service=ServiceController is not registered.
at org.jboss.mx.server.registry.BasicMBeanRegistry.get(BasicMBeanRegistry.java:508)
at org.jboss.mx.server.MBeanServerImpl.invoke(MBeanServerImpl.java:638)
at org.jboss.system.server.ServerImpl.startBootService(ServerImpl.java:536)
at org.jboss.system.server.ServerImpl.doStart(ServerImpl.java:438)
at org.jboss.system.server.ServerImpl.start(ServerImpl.java:330)
at org.jboss.Main.boot(Main.java:187)
at org.jboss.Main$1.run(Main.java:438)
at java.lang.Thread.run(Thread.java:595)
