-
1. Re: Periodic Recovery - ARJUNA016037 and ARJUNA016038
ochaloup Aug 19, 2014 9:21 AM (in response to tbronzan)1 of 1 people found this helpfulHi,
I think that this issue was discussed several times in other threads. I would recommend to check explanation here: norecoveryxa | Planet JBoss Developer
Or other reason for this warning could be that there are some unfinished transactions in jboss transaction log bound to datasource that the app server does not know. I mean something like that configuration of standalone.xml does not contain datasource with jndi "java:jboss/jdbc/realtime". But it does not seem being the case here.
I would say that the best solution is use jboss cli. It could be started by $JBOSS_HOME/bin/jboss-cli.sh from command line.
First you need to "load" data from the transaction log by probing log-store
/subsystem=transactions/log-store=log-store:probe()
Then you can see which transactions are unfinished there:
/subsystem=transactions/log-store=log-store:read-resource(recursive=true)
And you can delete such transaction one by one from the log store. This means that they will be forgotten. There will be no rollback called on any resource that participated in transaction. But there will be no in-doubt transaction in log so transaction manager will not complain about them anymore.
/subsystem=transactions/log-store=log-store/transactions=0\:ffffc0a815a7\:-7aef0ab8\:53dbd0f0\:b02:delete()
Ondra
-
2. Re: Periodic Recovery - ARJUNA016037 and ARJUNA016038
tbronzan Aug 29, 2014 10:54 AM (in response to ochaloup)Thanks for the response. Unfortunately using jboss cli is not an option because we need something that will happen automatically. We can't expect a customer to know how WIldFly works and how to handle problems like these manually. Furthermore, we cannot monitor thousands of customers to see if they are getting these log messages. We are looking for a solution that will automatically discard the transactions. I'm trying to avoid deleting the the data/tx-object-store folder on startup, but it's starting to look like I may have to.
-
3. Re: Periodic Recovery - ARJUNA016037 and ARJUNA016038
j_ri Sep 1, 2014 10:14 AM (in response to ochaloup)Yes, it was discussed several times, but there are no good answers;-(
I tested with Wildfly, EAP 6.1.1, EAP 6.2.4 and EAP 6.3.0
Although there are these entries in the log
16:03:02,301 WARN [com.arjuna.ats.jta] (Periodic Recovery) ARJUNA016037: Could not find new XAResource to use for recovering non-serializable XAResource XAResourceRecord < resource:null, txid:< formatId=131077, gtrid_length=39, bqual_length=36, tx_uid=0:ffff96854d53:-362d13b0:54047c43:10, node_name=rk3983node1, branch_uid=0:ffff96854d53:-362d13b0:54047c43:1b, subordinatenodename=null, eis_name=java:jboss/datasources/XATestNode11_Informix_DS >, heuristic: TwoPhaseOutcome.FINISH_OK, product: Informix Dynamic Server/11.50.FC7, jndiName: java:jboss/datasources/XATestNode11_Informix_DS com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord@136e8bfd >16:03:02,302 WARN [com.arjuna.ats.jta] (Periodic Recovery) ARJUNA016037: Could not find new XAResource to use for recovering non-serializable XAResource XAResourceRecord < resource:null, txid:< formatId=131077, gtrid_length=39, bqual_length=36, tx_uid=0:ffff96854d53:-362d13b0:54047c43:10, node_name=rk3983node1, branch_uid=0:ffff96854d53:-362d13b0:54047c43:29, subordinatenodename=null, eis_name=java:jboss/datasources/XATestNode21_Informix_DS >, heuristic: TwoPhaseOutcome.FINISH_OK, product: Informix Dynamic Server/11.50.FC7, jndiName: java:jboss/datasources/XATestNode21_Informix_DS com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord@544017e9 >16:03:02,302 WARN [com.arjuna.ats.jta] (Periodic Recovery) ARJUNA016037: Could not find new XAResource to use for recovering non-serializable XAResource XAResourceRecord < resource:null, txid:< formatId=131077, gtrid_length=39, bqual_length=36, tx_uid=0:ffff96854d53:-362d13b0:54047c43:10, node_name=rk3983node1, branch_uid=0:ffff96854d53:-362d13b0:54047c43:2e, subordinatenodename=null, eis_name=java:jboss/datasources/XATestNode22_DS >, heuristic: TwoPhaseOutcome.FINISH_OK, product: Oracle/Oracle Database 11g Enterprise Edition Release 11.2.0.3.0 - 64bit Production, jndiName: java:jboss/datasources/XATestNode22_DS com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord@3be1ed35 >16:03:02,302 WARN [com.arjuna.ats.jta] (Periodic Recovery) ARJUNA016038: No XAResource to recover < formatId=131077, gtrid_length=39, bqual_length=36, tx_uid=0:ffff96854d53:-362d13b0:54047c43:10, node_name=rk3983node1, branch_uid=0:ffff96854d53:-362d13b0:54047c43:1b, subordinatenodename=null, eis_name=java:jboss/datasources/XATestNode11_Informix_DS >16:03:02,302 WARN [com.arjuna.ats.jta] (Periodic Recovery) ARJUNA016038: No XAResource to recover < formatId=131077, gtrid_length=39, bqual_length=36, tx_uid=0:ffff96854d53:-362d13b0:54047c43:10, node_name=rk3983node1, branch_uid=0:ffff96854d53:-362d13b0:54047c43:29, subordinatenodename=null, eis_name=java:jboss/datasources/XATestNode21_Informix_DS >16:03:02,302 WARN [com.arjuna.ats.jta] (Periodic Recovery) ARJUNA016038: No XAResource to recover < formatId=131077, gtrid_length=39, bqual_length=36, tx_uid=0:ffff96854d53:-362d13b0:54047c43:10, node_name=rk3983node1, branch_uid=0:ffff96854d53:-362d13b0:54047c43:2e, subordinatenodename=null, eis_name=java:jboss/datasources/XATestNode22_DS >
the CLI commands
/subsystem=transactions/log-store=log-store:probe()
and
/subsystem=transactions/log-store=log-store:read-resource(recursive=true)
don't find any results.
after deleting the "tx-object-store" folder and starting the server again, the pending transactions get rollbacked....but this should happen before....
Why is "public void rollback(Xid xid) throws XAException" not called before (e.g. after reconnection to the database after a network crash )??
Is this really the way the recovery manager should work?
Regards,
Jochen
-
4. Re: Periodic Recovery - ARJUNA016037 and ARJUNA016038
tomjenkinson Sep 2, 2014 10:59 AM (in response to tbronzan)Hi guys,
This should be resolved in JBTM-860.
Hope it helps,
Tom
-
5. Re: Periodic Recovery - ARJUNA016037 and ARJUNA016038
ochaloup Sep 3, 2014 4:02 AM (in response to tbronzan)Hi Troy,
There is property xaAssumeRecoveryComplete which assumes the recovery being completed and unknown txn log entries could be forgotten. This should work with JTA but I think that it does not work for JTS. You put it as property on start of JBoss AS/WildFly like -DJTAEnvironmentBean.xaAssumeRecoveryComplete=true
I'm not sure if this helps and whether this would be a safe settings for you - but maybe as workaround it could work (till JBTM-860 will be implemented)
Detailed info on the option could be found in TM recovery guide here
Failure Recovery Guide - Assumed complete
Ondra
-
6. Re: Periodic Recovery - ARJUNA016037 and ARJUNA016038
tbronzan Sep 8, 2014 2:44 PM (in response to ochaloup)Thank you for the response. Setting that property to true seems to have resolved the issue.