[mazz]

Take a look at what the TS docs say about BasicXARecovery.

OK, I found those at: http://www.redhat.com/docs/en-US/JBoss_Enterprise_Application_Platform/4.2.0.cp05/html/Transactions_JTA_Programmers_Guide/sect-Transactions_JTA_Programmers_Guide-Examples-Failure_recovery_example.html

It says:

The parameter supplied in setParameters can contain arbitrary information necessary to initialize the class once created. In this instance it contains the name of the property file in which the db connection information is specified

Yup - that's exactly what I see JDBCXARecovery doing as I step thru the code. And if you give an invalid parameter (i.e. its not a URI to a prop file), you'll get the full jbossjta-properties.xml file props.

I am going to assume, this bug aside, that JDBCXARecovery does what I need it to do. I just have to get my data source to be a [xa-datasource> and I think it should work.

[mazz]

BTW: the topic of this thread "Disable XA" has now completely turned into the opposite of what I'm doing now :) Whoever is the admin of this forum should consider editing the thread's title to be "Enable XA" :)

[marklittle]

https://jira.jboss.org/jira/browse/JBTM-435

[mazz]

Crossed over all the hurdles previously discussed, but now hit the next one.

In JDBCXARecovery, there is this in method "createDataSource":

if (_dataSource == null) {
 Context context = new InitialContext();
 _dataSource = (XADataSource) context.lookup(_dbName);

I am stepping thru in a debugger and here is what I see. _dbName is correct, its my JNDI name. The lookup does return a non-null object, but it is not of type "XADataSource" so that line that is casting to XADataSource throws up a ClassCastException. The class it is getting back is "org.jboss.resource.adapter.jdbc.WrapperDataSource".

I step thru and for sure that's the object its getting back. Now, I know I have XA setup properly (I read through all the JBoss docs, converted my local-tx-datasource to a valid xa-datasource and see that it deploys fine). I also know this is XA because in my debugger, I inspect the returned object from this lookup and I see it has a private member "cm" (the connection manager) that in turn has a "cmName" member which is the object name whose value is "jboss.jca:service=XATxCM,name=RHQDS". That's the first indication its XA. My looked up object also has a "mcf" member and its type is "XAManagedConnectionFactory" - again, XA. In this object, I can inspect my XA DataSourceClass that I configured in my <xa-datasource> definition along with other XA things.

So, this lookup is definitely returning an object that is XA capable. Its just not returning what this JDBCXARecovery object is expecting - its not a XADataSource.

I have no idea what to do now - this looks like another bug in the JDBCXARecovery class because I can't see what I'm doing wrong. Everything seems configured properly, and my JNDI lookup is returning what looks like a valid XA-enabled object (just not a XADataSource object).

[jhalliday]

For recovery inside JBossAS you may be better off with AppServerJDBCXARecovery:

https://jira.jboss.org/jira/browse/JBTM-319

[mazz]

Ah... thanks for the JIRA link - now THAT is an important missing piece. I just found this forum thread, linked from that JIRA:

http://www.jboss.com/index.html?module=bb&op=viewtopic&t=126628&start=0&postdays=postDays&postorder=postOrder&highlight=highlight

Again, I go back to the fact that JBossTM's JBossAS integration docs are lacking. We really need a simple, single wiki page that discusses this issue - I've yet to find a good one. But I see there are some open JIRAs in JBTM to address some of these doc issues already.

In the end, I guess I was hoping the JBossTM integration was as easy as setting some configuration properties and "just go". It isn't (at least in JBossAS 4.2.1)

Now I really am curious to know how many people are ACTUALLY using the transaction manager's XA recovery feature in their deployed apps running in JBossAS. People may think they are (thinking that just by deploying an XA data source they get it), but I doubt many people really are. As far as I can tell, there is no simple way of enabling recovery short of writing your own code (or finding code submitted via JIRAs) and deploying it properly.

[mazz]

For the record, here's the source of AppServerJDBCXARecovery:

http://anonsvn.labs.jboss.com/labs/jbosstm/branches/JBOSSTS_4_2_3_GA_SP/atsintegration/classes/com/arjuna/ats/internal/jbossatx/jta/AppServerJDBCXARecovery.java

[marklittle]

Hi John. You can track this here https://jira.jboss.org/jira/browse/JBTM-436

[jhalliday]

> We really need a simple, single wiki page that discusses this issue - I've yet to find a good one

https://www.jboss.org/community/docs/DOC-10789

Feel free to make changes if it's not up to your high standards :-)

[mazz]

I still can't believe I can't get this right.

OK, I've done everything possible that I know of - deploying XA data source, configuring jbossjta-properties.xml with the appropriate AppServerJDBCXARecovery - I even step through this code to make sure its actually getting valid XADataSource objects (it is).

My app runs fine - and I verified it is using XA data sources and it can do all the database things that it needs to do - hibernate is working fine.

In this test, I'm using Postgres 8.3. As the app is running, I shutdown the database - a hard kill, the process is gone, database server is down. I let the app run for a minute or two, and it gets very angry at me, spitting out tons of exceptions as its session beans try but fail to connect to the database, as expected. After that, I restart the database. The app becomes happy again, and I can see it can access the database fine.

But, now, instead of getting periodic "XAResource is not serializable" messages, I'm getting periodic messages that look like this, roughly every 2 minutes:

01:33:33,171 WARN [loggerI18N] [com.arjuna.ats.internal.jta.recovery.xarecovery1] Local XARecoveryModule.xaRecovery got XA exception org.postgresql.xa.PGXAException: Error during recover, XAException.XAER_RMERR
01:35:43,171 WARN [loggerI18N] [com.arjuna.ats.internal.jta.recovery.xarecovery1] Local XARecoveryModule.xaRecovery got XA exception org.postgresql.xa.PGXAException: Error during recover, XAException.XAER_RMERR
01:37:53,171 WARN [loggerI18N] [com.arjuna.ats.internal.jta.recovery.xarecovery1] Local XARecoveryModule.xaRecovery got XA exception org.postgresql.xa.PGXAException: Error during recover, XAException.XAER_RMERR

Any ideas what this means?

P.S. Its been a half hour, and I still get these every 2 minutes... when do they stop? I assume at some point, the recovery system will give up, but when is that cutoff point?

[mazz]

And just for giggles, I just tried the same test on Oracle10g and I get the exact same behavior. I get the same XAException.XAER_RMERR warning message every 2 minutes.

So it appears this isn't specific to one database vendor. Postgres 8.3 or Oracle 10g - I get the same problem.

I must have something else misconfigured. But I have no idea what that could be.

Any hints as to what this warning message is telling me would be greatly appreciated - even more appreciated would be a solution to it :)

[jhalliday]

> Any hints as to what this warning message is telling me would be greatly appreciated

RMERR is a generic XA error code that covers a wide range of errors that can be collectively described as 'the resource manager is sulking'. It may be because the resource instance you have is on a connection that died when the db went down and did not reconnect. Or it may be because the user you are connecting as does not have the right permissions on the db to do recovery. Or maybe it's just in a bad mood. Try bouncing the app server too, that should help narrow down the possibilities.

[marklittle]

Looks like you've got an error when terminating the recovery scan. Not sure why and I don't recall ever having seen that happen before (not if the start scan succeeded).

Yes eventually the recovery manager will move the log elsewhere and require manual intervention. I think I mentioned that earlier in this thread. I think the default is about 12 hours, but can be set in the config file. Read the docs for more details.

[mazz]

"jhalliday" wrote:
Or it may be because the user you are connecting as does not have the right permissions on the db to do recovery.

Can you give more details on that? When I was searching the net for the causes of this, I came across something that mentioned "recovery permissions" too, but it didn't go into detail and I really didn't know what they were referring to other than "you must be the dba to recover".

Does this mean that for XA recovery to work, that my datasource's user/pass must be the credentials to a dba/admin user? Or the user must simply be granted some specific set of "recovery" permissions. What types of permissions are these?

I realize this question is probably database vendor specific and the answer is "depends on the database" - but if you have any specific information on Oracle and/Postgres related to that, please fill me in.

[jhalliday]

Depending on the flavour of Oracle you have, you'll need to provide the user some additional permissions along the lines of:

grant select on sys.dba_pending_transactions to foo;
grant select on sys.pending_trans$ to foo;
grant select on sys.dba_2pc_pending to foo;
grant execute on sys.dbms_system to foo;
grant select on v$xatrans$ to foo;

Think of it as giving them sudo for recovery functions rather than full on dba privs.