0 Replies Latest reply on Apr 8, 2014 7:48 AM by Harneet Singh

    JBOSS6.1 crashes / AIX6.1 / Oracle_client11.2.0.3 / java6_64

    Harneet Singh Newbie

      Hi all,

       

      We are using JBOSS6.1 Final version in our environment

      AIX  1 6

      java6_64

      oracle 11.2.0.3 client

      -------------------------------------

      Every now and then the JVM crashes with the following javacore output.  I doubt that the client is the culprit but not able to pinpoint the reason . Can you help or advise me as how to proceed further. javacore are also attached. If you guys need any more information then do let me know.

      ---------------------------------------

      NULL           ------------------------------------------------------------------------

      0SECTION       TITLE subcomponent dump routine

      NULL           ===============================

      1TISIGINFO     Dump Event "gpf" (00002000) received

      1TIDATETIME    Date:                 2014/04/08 at 10:35:15

      1TIFILENAME    Javacore filename:    /hahome/phoenix/jboss6/jboss6_webgrensesnitt/bin/javacore.20140408.103450.14024892.0004.txt

      1TIREQFLAGS    Request Flags: 0x81 (exclusive+preempt)

      1TIPREPSTATE   Prep State: 0x0

      1TIPREPINFO    Exclusive VM access not taken: data may not be consistent across javacore sections

      NULL           ------------------------------------------------------------------------

      0SECTION       GPINFO subcomponent dump routine

      NULL           ================================

      2XHOSLEVEL     OS Level         : AIX 6.1

      2XHCPUS        Processors -

      3XHCPUARCH       Architecture   : ppc64

      3XHNUMCPUS       How Many       : 16

      3XHNUMASUP       NUMA is either not supported or has been disabled by user

      NULL

      1XHEXCPCODE    J9Generic_Signal_Number: 00000004

      1XHEXCPCODE    Signal_Number: 0000000B

      1XHEXCPCODE    Error_Value: 00000000

      1XHEXCPCODE    Signal_Code: 00000033

      1XHEXCPCODE    Handler1: 09001000A0AAAA20

      1XHEXCPCODE    Handler2: 09001000A0AA1E58

      NULL

      1XHEXCPMODULE  Module: /progs/app/oracle/product/11203/client/lib/libclntsh.so

      1XHEXCPMODULE  Module_base_address: 0900000005684000

      NULL

      1XHREGISTERS   Registers:

      2XHREGISTER      R0: 09000000058BFC10

      2XHREGISTER      R1: 00000001236DAC90

      ..

      .

      .

      .

      .

      1XMCURTHDINFO  Current thread

      NULL           ----------------------

      3XMTHREADINFO      "WorkerThread#0[10.179.120.84:47451]" J9VMThread:0x0000000119A3EB00, j9thread_t:0x0000000119998980, java/lang/T

      hread:0x070000013DAA08C0, state:R, prio=5

      3XMTHREADINFO1            (native thread ID:0x36D00C5, native priority:0x5, native policy:UNKNOWN)

      3XMTHREADINFO3           Java callstack:

      4XESTACKTRACE                at oracle/jdbc/driver/T2CStatement.t2cDefineExecuteFetch(Native Method)

      4XESTACKTRACE                at oracle/jdbc/driver/T2CPreparedStatement.doDefineExecuteFetch(T2CPreparedStatement.java:1138)

      4XESTACKTRACE                at oracle/jdbc/driver/T2CPreparedStatement.executeForRows(T2CPreparedStatement.java:725)

      4XESTACKTRACE                at oracle/jdbc/driver/OracleStatement.executeMaybeDescribe(OracleStatement.java:1199(Compiled Code))

      4XESTACKTRACE                at oracle/jdbc/driver/OracleStatement.doExecuteWithTimeout(OracleStatement.java:1289(Compiled Code))

      4XESTACKTRACE                at oracle/jdbc/driver/OraclePreparedStatement.executeInternal(OraclePreparedStatement.java:3584)

      4XESTACKTRACE                at oracle/jdbc/driver/OraclePreparedStatement.executeQuery(OraclePreparedStatement.java:3628)

      4XESTACKTRACE                at oracle/jdbc/driver/OraclePreparedStatementWrapper.executeQuery(OraclePreparedStatementWrapper.java

      :1493)

      4XESTACKTRACE                at steria/fap/as/dbaccess/SessionDbConnection.verify(SessionDbConnection.java:599)

      4XESTACKTRACE                at steria/fap/as/pool/SubPool.getPoolItem(SubPool.java:248)

      4XESTACKTRACE                at steria/fap/as/pool/SubPool.getFreePoolItem(SubPool.java:279)

      4XESTACKTRACE                at steria/fap/as/pool/Pool.getFreePoolItem(Pool.java:47)

      4XESTACKTRACE                at steria/fap/as/dbaccess/DBConnManager.getConnection(DBConnManager.java:101)

      4XESTACKTRACE                at steria/fap/as/dbaccess/DBConnManager.getFormsSDBAccess(DBConnManager.java:86)

      4XESTACKTRACE                at steria/fap/as/defmanager/DefManager.getApplicationObject(DefManager.java:1305)

      4XESTACKTRACE                at steria/fap/as/defmanager/DefManager.getApplicationObject(DefManager.java:1289)

      4XESTACKTRACE                at steria/fap/as/defmanager/DefManager.getApplicationObject(DefManager.java:1284)

      4XESTACKTRACE                at steria/fap/as/operations/Operations.openSearch(Operations.java:1169)

      4XESTACKTRACE                at steria/fap/as/FapAsBean.openSearch(FapAsBean.java:459)

      4XESTACKTRACE                at sun/reflect/NativeMethodAccessorImpl.invoke0(Native Method)

      4XESTACKTRACE                at sun/reflect/NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:48(Compiled Code))

      4XESTACKTRACE                at sun/reflect/DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:37(Compiled Code

      ))

      4XESTACKTRACE                at java/lang/reflect/Method.invoke(Method.java:600(Compiled Code))

      4XESTACKTRACE                at org/jboss/invocation/Invocation.performCall(Invocation.java:386)

      4XESTACKTRACE                at org/jboss/ejb/StatelessSessionContainer$ContainerInterceptor.invoke(StatelessSessionContainer.java

      :233)

      4XESTACKTRACE                at org/jboss/resource/connectionmanager/CachedConnectionInterceptor.invoke(CachedConnectionIntercepto

      r.java:156)

      4XESTACKTRACE                at org/jboss/ejb/plugins/StatelessSessionInstanceInterceptor.invoke(StatelessSessionInstanceIntercept

      or.java:173)

      4XESTACKTRACE                at org/jboss/ejb/plugins/CallValidationInterceptor.invoke(CallValidationInterceptor.java:63)

      4XESTACKTRACE                at org/jboss/ejb/plugins/AbstractTxInterceptor.invokeNext(AbstractTxInterceptor.java:121)

      4XESTACKTRACE                at org/jboss/ejb/plugins/TxInterceptorCMT.runWithTransactions(TxInterceptorCMT.java:350)

      4XESTACKTRACE                at org/jboss/ejb/plugins/TxInterceptorCMT.invoke(TxInterceptorCMT.java:181(Compiled Code))

      4XESTACKTRACE                at org/jboss/ejb/plugins/SecurityInterceptor.process(SecurityInterceptor.java:228)

      4XESTACKTRACE                at org/jboss/ejb/plugins/SecurityInterceptor.invoke(SecurityInterceptor.java:211)

      4XESTACKTRACE                at org/jboss/ejb/plugins/security/PreSecurityInterceptor.process(PreSecurityInterceptor.java:100)

      4XESTACKTRACE                at org/jboss/ejb/plugins/security/PreSecurityInterceptor.invoke(PreSecurityInterceptor.java:84)

      4XESTACKTRACE                at org/jboss/ejb/plugins/LogInterceptor.invoke(LogInterceptor.java:205(Compiled Code))

      4XESTACKTRACE                at org/jboss/ejb/plugins/ProxyFactoryFinderInterceptor.invoke(ProxyFactoryFinderInterceptor.java:138)

      4XESTACKTRACE                at org/jboss/ejb/SessionContainer.internalInvoke(SessionContainer.java:650)

      4XESTACKTRACE                at org/jboss/ejb/Container.invoke(Container.java:1072)

      4XESTACKTRACE                at sun/reflect/GeneratedMethodAccessor288.invoke(Bytecode PC:40)

      4XESTACKTRACE                at sun/reflect/DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:37(Compiled Code

      ))

      4XESTACKTRACE                at java/lang/reflect/Method.invoke(Method.java:600(Compiled Code))

      4XESTACKTRACE                at org/jboss/mx/interceptor/ReflectedDispatcher.invoke(ReflectedDispatcher.java:157)

      4XESTACKTRACE                at org/jboss/mx/server/Invocation.dispatch(Invocation.java:96(Compiled Code))

      4XESTACKTRACE                at org/jboss/mx/server/Invocation.invoke(Invocation.java:87(Compiled Code))

      4XESTACKTRACE                at org/jboss/mx/server/AbstractMBeanInvoker.invoke(AbstractMBeanInvoker.java:200(Compiled Code))

      4XESTACKTRACE                at org/jboss/mx/server/MBeanServerImpl.invoke(MBeanServerImpl.java:670)

      4XESTACKTRACE                at org/jboss/invocation/unified/server/UnifiedInvoker.invoke(UnifiedInvoker.java:232)

      4XESTACKTRACE                at org/jboss/remoting/ServerInvoker.invoke(ServerInvoker.java:967)

      4XESTACKTRACE                at org/jboss/remoting/transport/socket/ServerThread.completeInvocation(ServerThread.java:791)

      4XESTACKTRACE                at org/jboss/remoting/transport/socket/ServerThread.processInvocation(ServerThread.java:744)

      4XESTACKTRACE                at org/jboss/remoting/transport/socket/ServerThread.dorun(ServerThread.java:586(Compiled Code))

      4XESTACKTRACE                at org/jboss/remoting/transport/socket/ServerThread.run(ServerThread.java:234)

      3XMTHREADINFO3           Native callstack:

      4XENATIVESTACK               sdbgrfbibf_io_block_file+0x164 (0x09000000058B9C48 [libclntsh.so+0x235c48])

      4XENATIVESTACK               dbgrfrbf_read_block_file+0x218 (0x0900000005805ADC [libclntsh.so+0x181adc])

      4XENATIVESTACK               dbgrmflrp_read_page+0x88 (0x09000000059F140C [libclntsh.so+0x36d40c])

      4XENATIVESTACK               dbgrmblgmp_get_many_pages+0x14c (0x09000000059B9F10 [libclntsh.so+0x335f10])

      4XENATIVESTACK               dbgrmmdrrmd_read_relation_meta_data+0x150 (0x09000000059FAAD4 [libclntsh.so+0x376ad4])

      4XENATIVESTACK               dbgrmmdora_open_record_access_full+0xe34 (0x0900000005A06058 [libclntsh.so+0x382058])

      4XENATIVESTACK               dbgriporc_openrel_wcreate+0x134 (0x0900000005972D98 [libclntsh.so+0x2eed98])

      4XENATIVESTACK               dbgrip_open_relation_access+0x88 (0x090000000597310C [libclntsh.so+0x2ef10c])

      4XENATIVESTACK               dbgrip_start_iterator+0x184 (0x0900000005977FC8 [libclntsh.so+0x2f3fc8])

      4XENATIVESTACK               dbgrip_relation_iterator+0x15c (0x0900000005979620 [libclntsh.so+0x2f5620])

      4XENATIVESTACK               dbgruprac_read_adrctl+0xc8 (0x09000000059633CC [libclntsh.so+0x2df3cc])

      4XENATIVESTACK               dbgrsc_set_createid+0x240 (0x0900000005786584 [libclntsh.so+0x102584])

      4XENATIVESTACK               dbgrcas_create_adr_schema+0x34 (0x0900000005787FD8 [libclntsh.so+0x103fd8])

      4XENATIVESTACK               dbgraid_adr_init_disk+0x150 (0x0900000005787A94 [libclntsh.so+0x103a94])

      4XENATIVESTACK               dbgrcaod_check_adr_ondisk+0xc8 (0x090000000578994C [libclntsh.so+0x10594c])

      4XENATIVESTACK               dbgrlWriteAlertDetail_int+0x8c (0x09000000058951F0 [libclntsh.so+0x2111f0])

      4XENATIVESTACK               dbgrlWriteAlertDetail+0x150 (0x09000000058935D4 [libclntsh.so+0x20f5d4])

      4XENATIVESTACK               dbgrlWriteAlertText+0x24 (0x09000000058933E8 [libclntsh.so+0x20f3e8])

      4XENATIVESTACK               nlddwrtlog+0x114 (0x09000000056A5BB8 [libclntsh.so+0x21bb8])

      4XENATIVESTACK               nldsadrvfp+0x4c (0x0900000005F91B50 [libclntsh.so+0x90db50])

      4XENATIVESTACK               nldsfprintf+0xf4 (0x0900000005F90258 [libclntsh.so+0x90c258])

      4XENATIVESTACK               niqlce1+0xa0 (0x0900000006FE17E4 [libclntsh.so+0x195d7e4])

      4XENATIVESTACK               nioqer+0x9ec (0x0900000006FD67D0 [libclntsh.so+0x19527d0])

      4XENATIVESTACK               nioqrc+0x26b8 (0x0900000006FC527C [libclntsh.so+0x194127c])

      4XENATIVESTACK               ttcdrv+0x3dc (0x0900000005EE23C0 [libclntsh.so+0x85e3c0])

      4XENATIVESTACK               nioqwa+0x4c (0x0900000006FE1270 [libclntsh.so+0x195d270])

      4XENATIVESTACK               upirtrc+0x97c (0x0900000005EFA6C0 [libclntsh.so+0x8766c0])

      4XENATIVESTACK               kpurcsc+0x6c (0x0900000005EF8E90 [libclntsh.so+0x874e90])

      4XENATIVESTACK               kpuexec+0xc10 (0x0900000005EF4B34 [libclntsh.so+0x870b34])

      4XENATIVESTACK               OCIStmtExecute+0x20 (0x0900000006500E04 [libclntsh.so+0xe7ce04])

      4XENATIVESTACK               eooExecute+0x1788 (0x0900000003151F4C [libocijdbc11.so+0x9f4c])

      4XENATIVESTACK               Java_oracle_jdbc_driver_T2CStatement_t2cDefineExecuteFetch+0x2e8 (0x090000000315696C [libocijdbc11.so

      +0xe96c])