1 2 Previous Next 18 Replies Latest reply on Nov 15, 2007 2:46 PM by jantzen

    Non Serialized XAResources Warnings with JBoss 4.2, PostgreS

    jantzen Newbie

      I'm trying to migrate from 4.0.5 to 4.2, and I'm hitting problems with my datasources. I learned in this process that 4.2 does not support multiple local TX datasources (http://www.jboss.com/index.html?module=bb&op=viewtopic&p=3988662). So, I changed my datasources to be XA's. But, now I get these warnings every few minutes for all three of my datasources.

      21:31:10,147 WARN [loggerI18N] [com.arjuna.ats.internal.jta.resources.arjunacore.norecoveryxa] [com.arjuna.ats.internal.jta.resources.arjunacore.norecoveryxa] Could not find new XAResource to use for recovering non-serializable XAResource < 131075, 28, 26, 1-7f000001:ba8a:469ed3e6:2097f000001:ba8a:469ed3e6:213
      


      My questions:
      What does this mean?
      What problems will this lead to?
      Is there an inexpensive solution?

      Thanks

        • 1. Re: Non Serialized XAResources Warnings with JBoss 4.2, Post
          jantzen Newbie

          Yay for title truncation. That's PostgreSQL 8.1.9

          • 2. Re: Non Serialized XAResources Warnings with JBoss 4.2, Post
            Mark Little Master

             

            "jantzen" wrote:
            I'm trying to migrate from 4.0.5 to 4.2, and I'm hitting problems with my datasources. I learned in this process that 4.2 does not support multiple local TX datasources (http://www.jboss.com/index.html?module=bb&op=viewtopic&p=3988662).


            That's an old article. It does now.

            http://wiki.jboss.org/wiki/Wiki.jsp?page=Multiple1PC


            So, I changed my datasources to be XA's. But, now I get these warnings every few minutes for all three of my datasources.

            21:31:10,147 WARN [loggerI18N] [com.arjuna.ats.internal.jta.resources.arjunacore.norecoveryxa] [com.arjuna.ats.internal.jta.resources.arjunacore.norecoveryxa] Could not find new XAResource to use for recovering non-serializable XAResource < 131075, 28, 26, 1-7f000001:ba8a:469ed3e6:2097f000001:ba8a:469ed3e6:213
            


            My questions:
            What does this mean?


            It means that you've had a crash and the TM is trying to recover. Unfortunately it has a stale XAResource reference and needs you to provide a new one so it can drive recovery.

            Check out

            http://wiki.jboss.org/wiki/Wiki.jsp?page=JBossTSRecovery

            and the Failure Recovery Guide as well as the JTA Programmers Guide.


            What problems will this lead to?


            You've had a failure. Until you provide a new XAResource, the transaction cannot complete. Locks and other resources will be retained.


            Is there an inexpensive solution?


            Since you're seeing this I'm assuming you've got more than one participant in the same transaction. If that's the case you needed XAResources in the first place, or you risk getting data consistency problems. You really only have two choices: a) go back to multiple one-phase resources and continue to run the risk of data corruption, b) fix the warning by adding in the necessary recovery classes.

            I'd recommend b).

            • 3. Re: Non Serialized XAResources Warnings with JBoss 4.2, Post
              jantzen Newbie

              Hi Mark, thank you for the detailed explanation, I appreciate it.

              • 4. Re: Non Serialized XAResources Warnings with JBoss 4.2, Post
                Mark Little Master

                I think I definitely need to add a wiki page around this, because it comes up enough.

                • 5. Re: Non Serialized XAResources Warnings with JBoss 4.2, Post
                  jantzen Newbie

                  Hi Again,

                  I'm returning to this problem after a few months' hiatus. Has anyone created a wiki page or forum post with a how-to describing multiple one phase setup in JBoss 4.2?

                  Thanks

                  • 6. Re: Non Serialized XAResources Warnings with JBoss 4.2, Post
                    Jonathan Halliday Master

                     

                    "jantzen" wrote:
                    Hi Again,

                    Has anyone created a wiki page or forum post with a how-to describing multiple one phase setup in JBoss 4.2?

                    Thanks


                    Is there something wrong with the one Mark mentioned back in July?
                    http://wiki.jboss.org/wiki/Wiki.jsp?page=Multiple1PC



                    • 7. Re: Non Serialized XAResources Warnings with JBoss 4.2, Post
                      jantzen Newbie

                      I've read that wiki page, along with the sections of the Transaction Manager manual that it references. None of these resources contain a simple example of how to accomplish the multiple 1 phase datasource.

                      I'm considering a premature upgrade to 4.2 due a bug in 4.0.5 (http://jira.jboss.com/jira/browse/JASSIST-28). Sorry to sound like a loser here, but I don't have the luxury of a week to bone up on the JTA internals and write a special implementation of com.arjuna.ats.jta.resources.LastResourceCommitOptimisation. If there's not a copy n'paste workaround here I've gotta look at other options.

                      • 8. Re: Non Serialized XAResources Warnings with JBoss 4.2, Post
                        Mark Little Master

                        You don't have to "bone up" on anything. Just do what the documentation says:

                        "[set] com.arjuna.ats.jta.allowMultipleLastResources to true."

                        • 9. Re: Non Serialized XAResources Warnings with JBoss 4.2, Post
                          jantzen Newbie

                          Okay, thanks. I updated the Multiple1PC wiki page with a small clarification on how to do this.

                          But, now I'm seeing these errors:

                          11:47:51,510 ERROR [STDERR] Caused by: javax.persistence.PersistenceException: org.hibernate.exception.GenericJDBCException:
                          Cannot open connection
                          


                          11:47:51,838 ERROR [STDERR] org.jboss.resource.connectionmanager.JBossLocalXAException: Trying to start a new tx when old is not complete! old: < 131075, 28, 26, 1-7f000001:880e:47375c0b:19a7f000001:880e:47375c0b:1ad >,
                          new < 131075, 28, 26,
                          1-7f000001:880e:47375c0b:1f07f000001:880e:47375c0b:1f7 >, flags 0
                          


                          11:47:52,128 ERROR [JDBCExceptionReporter] Could not enlist in transaction on entering meta-aware object!; - nested throwable: (javax.transaction.SystemException: java.lang.Throwable: Unabled to enlist resource, see the previous warnings. tx=TransactionImple < ac, BasicAction: 7f000001:880e:47375c0b:200 status:
                          ActionStatus.ABORT_ONLY >); - nested throwable: (org.jboss.resource.JBossResourceException: Could not enlist in transaction on entering meta-aware object!; - nested throwable:
                          (javax.transaction.SystemException: java.lang.Throwable: Unabled to enlist resource, see the previous warnings. tx=TransactionImple < ac, BasicAction: 7f000001:880e:47375c0b:200 status: ActionStatus.ABORT_ONLY >))
                          


                          Maybe if I explain what I'm trying to do you can point out my folly. I have a PostgreSQL database with multiple schemas. I want to read large sets of data from one schema, process the data in memory, write a subset of that data to the second schema, and update the first schema with a record that it has been imported. Here's a simple outline:
                          Postgres Database
                          ------------------------------------------------------------------------------------------------
                           Raw Source Data Schema | Processed Authoritative Data
                           ---reads--> Source Data Processor App ---writes-->
                           <--writes---
                          



                          Large result sets for new data are read from Raw Source into memory using default "Required" JTA transaction.

                          JBoss application processes the data deciding what to do with it.

                          application chooses not to use the data, does nothing
                          application chooses to write the data to Authoritative

                          updates Authority in a "RequiresNew" JTA transaction
                          in the same transaction updates the original Raw Source record with the date on which it was processed. The source record is retrieved by id inside of this transaction, and the record as it exists in the dataset in step 1 is untouched.



                          I'm seeing exceptions now in all steps inside step 2. The same code works fine in 4.0.5. I'm using RequiresNew when writing authoritative records because on these large datasets the transaction gets to be gigantic and I want to keep it small and narrowly focused.

                          Any ideas why multiple one phase data sources fail in 4.2.1 even with the new property set? Or why I'd have transaction failures using XA transactions?

                          Thanks for your help.

                          • 10. Re: Non Serialized XAResources Warnings with JBoss 4.2, Post
                            jantzen Newbie

                            Oh now this is fun: every single property in that jbossjta-properties.xml uses YES or NO except for this: name="com.arjuna.ats.jta.allowMultipleLastResources" value="true".

                            Now it's working with multiple local datasources.

                            Mark, I'd still truly appreciate any light you could shed on why my use of transactions doesn't work with XA datasources. Thanks.

                            • 11. Re: Non Serialized XAResources Warnings with JBoss 4.2, Post
                              Mark Little Master

                               

                              "jantzen" wrote:
                              Oh now this is fun: every single property in that jbossjta-properties.xml uses YES or NO except for this: name="com.arjuna.ats.jta.allowMultipleLastResources" value="true".


                              It's even better than that: some of them use ON or OFF too ;-)


                              Now it's working with multiple local datasources.

                              Mark, I'd still truly appreciate any light you could shed on why my use of transactions doesn't work with XA datasources. Thanks.


                              It's difficult to tell for sure from the errors you posted, but it looks like something isn't terminating the transaction associated with a thread before that thread tries to start a new transaction. The fact the transaction is set to ABORT_ONLY probably points at something somewhere calling setRollbackOnly and assuming it disassociates the thread-to-transaction mapping, which it doesn't do.

                              • 12. Re: Non Serialized XAResources Warnings with JBoss 4.2, Post
                                jantzen Newbie

                                It's looking like one transaction, perhaps the read operation from the source data schema, is being marked for rollback, while the write operations later succeed, and the whole thing gets rolled back on the basis of a "mixed heuristic". I don't see any actual errors during execution though, so it makes no sense to me that any of the transactions would be rolled back.

                                What's the best way to determine the cause of the rollback? Enable trace logging in the arjuna package?

                                • 13. Re: Non Serialized XAResources Warnings with JBoss 4.2, Post
                                  Mark Little Master

                                   

                                  "jantzen" wrote:
                                  What's the best way to determine the cause of the rollback? Enable trace logging in the arjuna package?


                                  Yes, that would be a good place to start.

                                  • 14. Re: Non Serialized XAResources Warnings with JBoss 4.2, Post
                                    jantzen Newbie

                                    With debug logging on in "org.hibernate" and "com.arjuna" we were able to trace the call to setRollbackOnly to a class we're using to page over result sets. The method in question is this:

                                     private void getNextResults() {
                                     query.setFirstResult(tripCounter++ * maxResults);
                                     query.setMaxResults(maxResults);
                                     currentResults = query.getResultList();
                                     currentResultsIter = currentResults.iterator();
                                     }
                                    


                                    "query" is a javax.persistence.Query object, and the method operates over the result set pulling out "maxResults" (ie 100) at a time. For some reason this operation encounters an error periodically inside of Query.getResultList(), which tries then to roll back the transaction.

                                    This seems completely innocuous and there isn't an accompanying exception we can see. Could it be a timeout on the read operation? What could possibly be going wrong in iterating over a resultset?

                                    1 2 Previous Next