9 Replies Latest reply on Jun 17, 2015 5:34 AM by jfclere

    JVM crashes on AS 7 with Native libraries using mod_jk




      I’m currently having a trouble with the JBoss Native libraries on JBoss AS7.


      First, here is my environment:

      OS: Linux CentOS 6 x64

      Apache version: Apache/2.2.15

      Java version: JDK 1.6.0_26 x64

      mod_jk: built from sources, tomcat-connectors-1.2.32-src.tar.gz

      JBoss AS: 7.1.0 nighly built (not the latest version, from late August)


      JBoss Native, I’ve tried both versions jboss-native-2.0.9-linux2-x64-ssl.tar.gz and jboss-native-2.0.10-linux2-x64-ssl.tar.gz


      Installation method: I’m not really sure if I install mod_jk the right way. I untared the binaries into my JBoss AS 7 installation (which is in /opt/jboss-as/) and added the following parameter to standalone.conf

      JAVA_OPTS="$JAVA_OPTS -Djava.library.path=/opt/jboss-as/bin/native"

      However, when starting the server, it still logged that it couldn’t find the native libraries and I figured out that I had to do the following symlink:

      mkdir -p /opt/jboss-as/standalone/tmp/bin/META-INF/lib/linux2

      cd /opt/jboss-as/standalone/tmp/bin/META-INF/lib/linux2

      ln -s ../../../../../../bin/native x64


      (Note: in a Windows environment, JBoss found the native libraries without that symlink. But I haven’t tested using mod_jk under Windows)


      Without the native libraries, I can use mod_jk perfectly. But since I install them as above, at the first HTTP request I get a JVM crash. Here is the summary of the crash, I can post the full details if needed



      # A fatal error has been detected by the Java Runtime Environment:


      #  SIGSEGV (0xb) at pc=0x00007f654c963378, pid=16452, tid=140070285702912


      # JRE version: 6.0_26-b03

      # Java VM: Java HotSpot(TM) 64-Bit Server VM (20.1-b02 mixed mode linux-amd64 compressed oops)

      # Problematic frame:

      # C  [libapr-1.so.0.3.12+0x1d378]  signed char+0x68


      # If you would like to submit a bug report, please visit:

      # http://java.sun.com/webapps/bugreport/crash.jsp

      # The crash happened outside the Java Virtual Machine in native code.

      # See problematic frame for where to report the bug.



      ---------------  T H R E A D ---------------


      Current thread (0x00007f64ec143800):  JavaThread "ajp--" daemon [_thread_in_native, id=16498, stack(0x00007f64a78f7000,0x00007f64a79f8000)]


      siginfo:si_signo=SIGSEGV: si_errno=0, si_code=1 (SEGV_MAPERR), si_addr=0x0000000000000008





      Stack: [0x00007f64a78f7000,0x00007f64a79f8000], sp=0x00007f64a79f6810,  free space=1022k

      Native frames: (J=compiled Java code, j=interpreted, Vv=VM code, C=native code)

      C  [libapr-1.so.0.3.12+0x1d378]  signed char+0x68


      Java frames: (J=compiled Java code, j=interpreted, Vv=VM code)

      j org.apache.tomcat.jni.Poll.add(JJI)I+0

      j org.apache.tomcat.util.net.AprEndpoint$Poller.addToPoller(JI)Z+35

      j org.apache.tomcat.util.net.AprEndpoint$Poller.run()V+464

      j java.lang.Thread.run()V+11


      ---------------  P R O C E S S ---------------


      Java Threads: ( => current thread )

      =>0x00007f64ec143800 JavaThread "ajp--" daemon [_thread_in_native, id=16498, stack(0x00007f64a78f7000,0x00007f64a79f8000)]



      VM Arguments:

      jvm_args: -Xms512m -Xmx2000m -XX:MaxPermSize=256M -Djava.net.preferIPv4Stack=true -Dorg.jboss.resolver.warning=true -Dsun.rmi.dgc.client.gcInterval=3600000 -Dsun.rmi.dgc.server.gcInterval=3600000 -Djboss.modules.system.pkgs=org.jboss.byteman -Djava.awt.headless=true -Djava.library.path=/opt/jboss-as/bin/native -Djbo.use.pers.coll=false -Djbo.passivationstore=file -Djbo.tmpdir=/temp/jbo/ -XX:CompileCommand=exclude,oracle/sql/NUMBER,toBytes -Duser.country=US -Duser.language=en -DNLS_LANG=AMERICAN_AMERICA.UTF8 -Dfile.encoding=UTF-8 -Djavax.xml.bind.JAXBContext=com.sun.xml.bind.v2.ContextFactory -Dorg.jboss.boot.log.file=/opt/jboss-as/standalone/log/boot.log -Dlogging.configuration=file:/opt/jboss-as/standalone/configuration/logging.properties

      java_command: /opt/jboss-as/jboss-modules.jar -mp /opt/jboss-as/modules -logmodule org.jboss.logmanager -jaxpmodule javax.xml.jaxp-provider org.jboss.as.standalone -Djboss.home.dir=/opt/jboss-as

      Launcher Type: SUN_STANDARD


      Environment Variables:






      Signal Handlers:

      SIGSEGV: [libjvm.so+0x85f800], sa_mask[0]=0x7ffbfeff, sa_flags=0x10000004

      SIGBUS: [libjvm.so+0x85f800], sa_mask[0]=0x7ffbfeff, sa_flags=0x10000004

      SIGFPE: [libjvm.so+0x70e300], sa_mask[0]=0x7ffbfeff, sa_flags=0x10000004

      SIGPIPE: [libjvm.so+0x70e300], sa_mask[0]=0x7ffbfeff, sa_flags=0x10000004

      SIGXFSZ: [libjvm.so+0x70e300], sa_mask[0]=0x7ffbfeff, sa_flags=0x10000004

      SIGILL: [libjvm.so+0x70e300], sa_mask[0]=0x7ffbfeff, sa_flags=0x10000004

      SIGUSR1: SIG_DFL, sa_mask[0]=0x00000000, sa_flags=0x00000000

      SIGUSR2: [libjvm.so+0x711110], sa_mask[0]=0x00000000, sa_flags=0x10000004

      SIGHUP: [libjvm.so+0x710d10], sa_mask[0]=0x7ffbfeff, sa_flags=0x10000004

      SIGINT: SIG_IGN, sa_mask[0]=0x00000000, sa_flags=0x00000000

      SIGTERM: [libjvm.so+0x710d10], sa_mask[0]=0x7ffbfeff, sa_flags=0x10000004

      SIGQUIT: [libjvm.so+0x710d10], sa_mask[0]=0x7ffbfeff, sa_flags=0x10000004



      ---------------  S Y S T E M ---------------


      OS:CentOS Linux release 6.0 (Final)


      uname:Linux 2.6.32-71.el6.x86_64 #1 SMP Fri May 20 03:51:51 BST 2011 x86_64

      libc:glibc 2.12 NPTL 2.12

      rlimit: STACK 10240k, CORE 0k, NPROC 1024, NOFILE 1024, AS infinity

      load average:0.56 0.28 0.10



      MemTotal:       16317844 kB

      MemFree:         9626772 kB

      Buffers:          179080 kB

      Cached:          4108912 kB

      SwapCached:            0 kB

      Active:          3052908 kB

      Inactive:        2881584 kB

      Active(anon):    1646884 kB

      Inactive(anon):     1676 kB

      Active(file):    1406024 kB

      Inactive(file):  2879908 kB

      Unevictable:           0 kB

      Mlocked:               0 kB

      SwapTotal: 18563064 kB

      SwapFree:       18563064 kB

      Dirty:               128 kB

      Writeback:             0 kB

      AnonPages:       1654892 kB

      Mapped:            59948 kB

      Shmem:              2036 kB

      Slab:             659732 kB

      SReclaimable:     300400 kB

      SUnreclaim:       359332 kB

      KernelStack: 3224 kB

      PageTables:        20336 kB

      NFS_Unstable:          0 kB

      Bounce:                0 kB

      WritebackTmp:          0 kB

      CommitLimit:    26721984 kB

      Committed_AS:    1781628 kB

      VmallocTotal:   34359738367 kB

      VmallocUsed:      310208 kB

      VmallocChunk:   34350788272 kB

      HardwareCorrupted:     0 kB

      AnonHugePages: 1017856 kB

      HugePages_Total:       0

      HugePages_Free:        0

      HugePages_Rsvd:        0

      HugePages_Surp:        0

      Hugepagesize:       2048 kB

      DirectMap4k:        7652 kB

      DirectMap2M: 16756736 kB



      CPU:total 8 (4 cores per cpu, 1 threads per core) family 6 model 26 stepping 5, cmov, cx8, fxsr, mmx, sse, sse2, sse3, ssse3, sse4.1, sse4.2, popcnt




      Memory: 4k page, physical 16317844k(9626772k free), swap 18563064k(18563064k free)


      vm_info: Java HotSpot(TM) 64-Bit Server VM (20.1-b02) for linux-amd64 JRE (1.6.0_26-b03), built on May  4 2011 01:13:47 by "java_re" with gcc 3.2.2 (SuSE Linux)


      time: Fri Sep  9 10:43:18 2011

      elapsed time: 126 seconds