Jboss EAP 6.2 crashed silently
arunav.sinha Sep 28, 2015 12:39 PMWe are using jboss-eap-6.2 in which multiple web applications are deployed in it. JDK used is jdk1.7.0_51 and linux version is SUSE Linux.
From past few months we are facing issue with JBoss server, the JBoss process goes down silently without logging any error or exception in any server or application logs after running for some days. The behavior is not periodic as some time it goes down after 5 days after running continuously and sometime it goes down after 20 days. As suggested in many places we have set "-Xrs" at VM start up, but this also did not helped and Jboss crashed again with same behaviour. In order to further debug issues, we have enabled GC logs and VM thread logs. Exactly at time of crash I could see below logs.Its been observed at time of crash nothing much is happening on the server and mostly its ideal at that time.
<dependency_failed type='abstract_with_unique_concrete_subtype' ctxk='com/sun/xml/bind/v2/runtime/Coordinator' x='com/sun/xml/bind/v2/runtime/unmarshaller/UnmarshallingContext' witness='com/sun/xml/bind/v2/runtime/XMLSerializer' stamp='92528.135'/>
<dependency_failed type='abstract_with_unique_concrete_subtype' ctxk='com/sun/xml/bind/v2/runtime/Coordinator' x='com/sun/xml/bind/v2/runtime/unmarshaller/UnmarshallingContext' witness='com/sun/xml/bind/v2/runtime/XMLSerializer' stamp='92528.135'/>
<dependency_failed type='abstract_with_unique_concrete_subtype' ctxk='com/sun/xml/bind/v2/runtime/Coordinator' x='com/sun/xml/bind/v2/runtime/unmarshaller/UnmarshallingContext' witness='com/sun/xml/bind/v2/runtime/XMLSerializer' stamp='92528.135'/>
<dependency_failed type='abstract_with_unique_concrete_subtype' ctxk='com/sun/xml/bind/v2/runtime/Coordinator' x='com/sun/xml/bind/v2/runtime/unmarshaller/UnmarshallingContext' witness='com/sun/xml/bind/v2/runtime/XMLSerializer' stamp='92528.135'/>
From initial analysis we are quite sure that issue is not related to out of memory. The JVM is started with -Xms2048m -Xmx8192m -XX:MaxPermSize=1024m -Xrs -verbose:gc -XX:+PrintGCDateStamps -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath= /logs/jbossOutOfMemory -Xrs -XX:+UnlockDiagnosticVMOptions -XX:+LogVMOutput -XX:LogFile=/logs/threadStackTrace.log
We are not able to replicate the issue and not sure what's root cause of the Jboss crash. Any help would be very much appreciated.
Thanks in Advance.