XA Recovery

[benr]

Hi folks. I'm running JBoss 5.1.0 and WebSphere MQ's JCA resource adapter on x86 Linux, with a MDB pulling messages from a WMQ queue and putting them to a response queue under an XA transaction. This all works fine.

I then kill the queue manager when the MDB is under heavy load (100s messages/second). As expected, this results in a few in-doubt transactions from messages where xa_prepare had completed but xa_commit failed with XAER_RMFAIL as the connection to the RM was lost. I then bring the queue manager up, the MDB is reconnected by the resource adapter and delivery resumes.

However, I would expect to see a call to the ResourceAdapter method

public XAResource[] getXAResources(ActivationSpec[] specs)throws ResourceException

to resolve the in-doubt transactions, but this never seems to happen (there's no sign of it in resource adapter trace). Instead I see a looping message in the server output saying

14:09:28,308 WARN  [loggerI18N] [com.arjuna.ats.internal.jta.resources.arjunacore.norecoveryxa] [com.arjuna.ats.internal.jta.resources.arjunacore.norecoveryxa] Could not find new XAResource to use for recovering non-serializable XAResource < 131075, 27, 25, 494557495256534897589748101100585299509952975655585099555749525653489758974810110058529950995297565558509956 >

<snip>

14:11:38,348 WARN  [loggerI18N] [com.arjuna.ats.internal.jta.resources.arjunacore.norecoveryxa] [com.arjuna.ats.internal.jta.resources.arjunacore.norecoveryxa] Could not find new XAResource to use for recovering non-serializable XAResource < 131075, 27, 25, 494557495256534897589748101100585299509952975655585099555749525653489758974810110058529950995297565558509956 >

Restarting the server doesn't help - the messages continue post-restart.

If I use the WMQ command dspmqtrn/rsvmqtrn to manually resolve the transaction then the in-doubt messages become available on the queue.

Is this behaviour expected, or have I failed to do some configuration of either the server or resource adapter? I've found some discussion but it's from a couple of years ago and i'm not sure if it's current. If I repeat the test with JBoss 4.2.3 then I get similar behaviour, although I also hit a javax.transaction.HeuristicMixedException when the connection failure happens and appear to lose messages as a result.

[jesper.pedersen]

Did you set it up ? http://community.jboss.org/wiki/JBossTSRecoveryInAS

Otherwise try and the latest SVN snapshot of AS/trunk - its done automatically there - http://community.jboss.org/wiki/XARecoveryinJCA

[benr]

Jesper Pedersen wrote:

Did you set it up ? http://community.jboss.org/wiki/JBossTSRecoveryInAS

Otherwise try and the latest SVN snapshot of AS/trunk - its done automatically there - http://community.jboss.org/wiki/XARecoveryinJCA

The honest answer is that i'm not sure if it's setup correctly - i'd read that link, but it states "This article applies to JBossAS 4.2 and EAP versions based on is (EAP 4.2, 4.3)" and, as i'm on 5.1, I wasn't certain if it was still relevant. I consistently hit a HeuristicMixed exception on failure in 4.2, so had given up on that and moved to 5.1

I'd also read http://community.jboss.org/wiki/XARecoveryinJCA

too, but again was uncertain of the relevance. In particular, it's talking about outbound JCA recovery (via a MCF) whereas i'm looking at inbound recovery (via the ResourceAdapter.getXAResouces(...) call) and from the perspective of an application writer I can't directly do anything with a org.jboss.tm.XAResourceRecovery as the WMQ resource adapter is providing the XAResource implementation.

I've spent a day or so (on and off) wrestling with the docs to try and understand what needs doing. I'm coming from the perspective of a JEE application developer here checking application portability (this is also running in WAS, Geronimo and Glassfish) and it came as something of a surprise that this might not work 'out of the box'. Indeed, my experiences largely match

http://management-platform.blogspot.com/2008/11/transaction-recovery-in-jbossas.html

so i'm trying to understand what I can (and can't) do here. For legal reasons I can't look at the source code, so i'm reliant on the docs.

[jhalliday]

> i'm looking at inbound recovery (via the  ResourceAdapter.getXAResouces(...) call) and from the perspective of an  application writer

Application writer? Last I heard you were an MQ dev at IBM, not an end user.  The first part of that sentence does make sense though - you're basically referring to JCA 1.5 section 12.5.2 I think.

> and  it came as something of a surprise that this might not work 'out of the  box'.

If you can say that with a straight face you've not been watching enough youtube :-)

As far an I can tell from a quick poke though the JCA code, it does not call ResourceAdapter.getXAResources(javax.resource.spi.ActivationSpec[] activationSpecs)  I'm guessing that's an implementation decision based on the assumption that a call to ManagedConnection.getXAResource().recover() will return equivalent information instead.

[benr]

Jonathan Halliday wrote:

Application writer? Last I heard you were an MQ dev at IBM, not an end user. 

It's not mutually exclusive. If people are using this in production unaware that they can't do transaction recovery then i'm quite keen to know about it and (if necessary) point them at the relevant config.

Jonathan Halliday wrote:

- you're basically referring to JCA 1.5 section 12.5.2 I think.

ResourceAdapter.getXAResources(javax.resource.spi.ActivationSpec[] activationSpecs)  I'm guessing that's an implementation decision based on the assumption that a call to ManagedConnection.getXAResource().recover() will return equivalent information instead.

I don't think it's a safe assumption though - you can define an activationSpec without a corresponding MCF, so there's no guarantee you can get the necessary information to drive recovery

12.5.2 reads to me as if the server must call ResourceAdapter.getXAResources(...) to do recovery, although there may be something elsewhere that overrides that section - it's been a while since I read the spec in detail.

[ben.cotton1]

Re: XA Recovery

Jonathan  Halliday wrote:

Application writer? Last I heard you were an MQ dev at IBM,  not an end user.

It's not mutually exclusive. If people are  using this in production unaware that they can't do transaction recovery  then i'm quite keen to know about it and (if necessary) point them at  the relevant config.

Ben - First time I have seen you post here, it is great to know that you are now directly contributing your expertise to this forum.   I recall the many times  (e.g. http://community.jboss.org/message/525230#525230 ) you have generously taken your time to  help us config JBoss connectivity to our legacy WSMQ tier via JCA.  We couldn't have gotten it done without you.    Again, great to see you here.

[clauritsen]

I don't think it's a safe assumption though - you can define an activationSpec without a corresponding MCF, so there's no guarantee you can get the necessary information to drive recovery

12.5.2 reads to me as if the server must call ResourceAdapter.getXAResources(...) to do recovery, although there may be something elsewhere that overrides that section - it's been a while since I read the spec in detail.

So if this was the assumption, the question remains: Is there someway to get inbound recovery working with wmq.jmsra.rar? I'm using 5.1.0.GA, but can upgrade to 6 if it has what I need.

If I have to provide a custom jar to enable it somehow its fine. Any pointers in that endeavor would help greatly.

Thanks.