JVM crashes on AS 7 with Native libraries using mod_jk
Hello,
I’m currently having a trouble with the JBoss Native libraries on JBoss AS7.
First, here is my environment:
OS: Linux CentOS 6 x64
Apache version: Apache/2.2.15
Java version: JDK 1.6.0_26 x64
mod_jk: built from sources, tomcat-connectors-1.2.32-src.tar.gz
JBoss AS: 7.1.0 nighly built (not the latest version, from late August)
JBoss Native, I’ve tried both versions jboss-native-2.0.9-linux2-x64-ssl.tar.gz and jboss-native-2.0.10-linux2-x64-ssl.tar.gz
Installation method: I’m not really sure if I install mod_jk the right way. I untared the binaries into my JBoss AS 7 installation (which is in /opt/jboss-as/) and added the following parameter to standalone.conf
JAVA_OPTS="$JAVA_OPTS -Djava.library.path=/opt/jboss-as/bin/native"
However, when starting the server, it still logged that it couldn’t find the native libraries and I figured out that I had to do the following symlink:
mkdir -p /opt/jboss-as/standalone/tmp/bin/META-INF/lib/linux2
cd /opt/jboss-as/standalone/tmp/bin/META-INF/lib/linux2
ln -s ../../../../../../bin/native x64
(Note: in a Windows environment, JBoss found the native libraries without that symlink. But I haven’t tested using mod_jk under Windows)
Without the native libraries, I can use mod_jk perfectly. But since I install them as above, at the first HTTP request I get a JVM crash. Here is the summary of the crash, I can post the full details if needed
#
# A fatal error has been detected by the Java Runtime Environment:
#
# SIGSEGV (0xb) at pc=0x00007f654c963378, pid=16452, tid=140070285702912
#
# JRE version: 6.0_26-b03
# Java VM: Java HotSpot(TM) 64-Bit Server VM (20.1-b02 mixed mode linux-amd64 compressed oops)
# Problematic frame:
# C [libapr-1.so.0.3.12+0x1d378] signed char+0x68
#
# If you would like to submit a bug report, please visit:
# http://java.sun.com/webapps/bugreport/crash.jsp
# The crash happened outside the Java Virtual Machine in native code.
# See problematic frame for where to report the bug.
#
--------------- T H R E A D ---------------
Current thread (0x00007f64ec143800): JavaThread "ajp--127.0.0.1-8009-Poller" daemon [_thread_in_native, id=16498, stack(0x00007f64a78f7000,0x00007f64a79f8000)]
siginfo:si_signo=SIGSEGV: si_errno=0, si_code=1 (SEGV_MAPERR), si_addr=0x0000000000000008
Stack: [0x00007f64a78f7000,0x00007f64a79f8000], sp=0x00007f64a79f6810, free space=1022k
Native frames: (J=compiled Java code, j=interpreted, Vv=VM code, C=native code)
C [libapr-1.so.0.3.12+0x1d378] signed char+0x68
Java frames: (J=compiled Java code, j=interpreted, Vv=VM code)
j org.apache.tomcat.jni.Poll.add(JJI)I+0
j org.apache.tomcat.util.net.AprEndpoint$Poller.addToPoller(JI)Z+35
j org.apache.tomcat.util.net.AprEndpoint$Poller.run()V+464
j java.lang.Thread.run()V+11
--------------- P R O C E S S ---------------
Java Threads: ( => current thread )
=>0x00007f64ec143800 JavaThread "ajp--127.0.0.1-8009-Poller" daemon [_thread_in_native, id=16498, stack(0x00007f64a78f7000,0x00007f64a79f8000)]
VM Arguments:
jvm_args: -Xms512m -Xmx2000m -XX:MaxPermSize=256M -Djava.net.preferIPv4Stack=true -Dorg.jboss.resolver.warning=true -Dsun.rmi.dgc.client.gcInterval=3600000 -Dsun.rmi.dgc.server.gcInterval=3600000 -Djboss.modules.system.pkgs=org.jboss.byteman -Djava.awt.headless=true -Djava.library.path=/opt/jboss-as/bin/native -Djbo.use.pers.coll=false -Djbo.passivationstore=file -Djbo.tmpdir=/temp/jbo/ -XX:CompileCommand=exclude,oracle/sql/NUMBER,toBytes -Duser.country=US -Duser.language=en -DNLS_LANG=AMERICAN_AMERICA.UTF8 -Dfile.encoding=UTF-8 -Djavax.xml.bind.JAXBContext=com.sun.xml.bind.v2.ContextFactory -Dorg.jboss.boot.log.file=/opt/jboss-as/standalone/log/boot.log -Dlogging.configuration=file:/opt/jboss-as/standalone/configuration/logging.properties
java_command: /opt/jboss-as/jboss-modules.jar -mp /opt/jboss-as/modules -logmodule org.jboss.logmanager -jaxpmodule javax.xml.jaxp-provider org.jboss.as.standalone -Djboss.home.dir=/opt/jboss-as
Launcher Type: SUN_STANDARD
Environment Variables:
PATH=/sbin:/bin:/usr/sbin:/usr/bin
USERNAME=jboss
LD_LIBRARY_PATH=/usr/java/jdk1.6.0_26/jre/lib/amd64/server:/usr/java/jdk1.6.0_26/jre/lib/amd64:/usr/java/jdk1.6.0_26/jre/../lib/amd64
SHELL=/bin/bash
Signal Handlers:
SIGSEGV: [libjvm.so+0x85f800], sa_mask[0]=0x7ffbfeff, sa_flags=0x10000004
SIGBUS: [libjvm.so+0x85f800], sa_mask[0]=0x7ffbfeff, sa_flags=0x10000004
SIGFPE: [libjvm.so+0x70e300], sa_mask[0]=0x7ffbfeff, sa_flags=0x10000004
SIGPIPE: [libjvm.so+0x70e300], sa_mask[0]=0x7ffbfeff, sa_flags=0x10000004
SIGXFSZ: [libjvm.so+0x70e300], sa_mask[0]=0x7ffbfeff, sa_flags=0x10000004
SIGILL: [libjvm.so+0x70e300], sa_mask[0]=0x7ffbfeff, sa_flags=0x10000004
SIGUSR1: SIG_DFL, sa_mask[0]=0x00000000, sa_flags=0x00000000
SIGUSR2: [libjvm.so+0x711110], sa_mask[0]=0x00000000, sa_flags=0x10000004
SIGHUP: [libjvm.so+0x710d10], sa_mask[0]=0x7ffbfeff, sa_flags=0x10000004
SIGINT: SIG_IGN, sa_mask[0]=0x00000000, sa_flags=0x00000000
SIGTERM: [libjvm.so+0x710d10], sa_mask[0]=0x7ffbfeff, sa_flags=0x10000004
SIGQUIT: [libjvm.so+0x710d10], sa_mask[0]=0x7ffbfeff, sa_flags=0x10000004
--------------- S Y S T E M ---------------
OS:CentOS Linux release 6.0 (Final)
uname:Linux 2.6.32-71.el6.x86_64 #1 SMP Fri May 20 03:51:51 BST 2011 x86_64
libc:glibc 2.12 NPTL 2.12
rlimit: STACK 10240k, CORE 0k, NPROC 1024, NOFILE 1024, AS infinity
load average:0.56 0.28 0.10
/proc/meminfo:
MemTotal: 16317844 kB
MemFree: 9626772 kB
Buffers: 179080 kB
Cached: 4108912 kB
SwapCached: 0 kB
Active: 3052908 kB
Inactive: 2881584 kB
Active(anon): 1646884 kB
Inactive(anon): 1676 kB
Active(file): 1406024 kB
Inactive(file): 2879908 kB
Unevictable: 0 kB
Mlocked: 0 kB
SwapTotal: 18563064 kB
SwapFree: 18563064 kB
Dirty: 128 kB
Writeback: 0 kB
AnonPages: 1654892 kB
Mapped: 59948 kB
Shmem: 2036 kB
Slab: 659732 kB
SReclaimable: 300400 kB
SUnreclaim: 359332 kB
KernelStack: 3224 kB
PageTables: 20336 kB
NFS_Unstable: 0 kB
Bounce: 0 kB
WritebackTmp: 0 kB
CommitLimit: 26721984 kB
Committed_AS: 1781628 kB
VmallocTotal: 34359738367 kB
VmallocUsed: 310208 kB
VmallocChunk: 34350788272 kB
HardwareCorrupted: 0 kB
AnonHugePages: 1017856 kB
HugePages_Total: 0
HugePages_Free: 0
HugePages_Rsvd: 0
HugePages_Surp: 0
Hugepagesize: 2048 kB
DirectMap4k: 7652 kB
DirectMap2M: 16756736 kB
CPU:total 8 (4 cores per cpu, 1 threads per core) family 6 model 26 stepping 5, cmov, cx8, fxsr, mmx, sse, sse2, sse3, ssse3, sse4.1, sse4.2, popcnt
Memory: 4k page, physical 16317844k(9626772k free), swap 18563064k(18563064k free)
vm_info: Java HotSpot(TM) 64-Bit Server VM (20.1-b02) for linux-amd64 JRE (1.6.0_26-b03), built on May 4 2011 01:13:47 by "java_re" with gcc 3.2.2 (SuSE Linux)
time: Fri Sep 9 10:43:18 2011
elapsed time: 126 seconds
