9 Replies Latest reply on Jan 9, 2012 3:43 AM by watchamcb

    HeuristicMixedException: cause and remedy?

    kennardconsulting

      Hi guys,

      I'm having a hard time finding good documentation on javax.transaction.HeuristicMixedException as it relates to JBoss Messaging. JBoss Messaging seems to throw this exception intermittently while processing large volumes of messages in a batch (say, 2000 messages over 3 cluster nodes):

      javax.transaction.HeuristicMixedException
       at com.arjuna.ats.internal.jta.transaction.arjunacore.TransactionImple.commitAndDisassociate(TransactionImple.java:1431)
       at com.arjuna.ats.internal.jta.transaction.arjunacore.BaseTransaction.commit(BaseTransaction.java:137)
       at com.arjuna.ats.jbossatx.BaseTransactionManagerDelegate.commit(BaseTransactionManagerDelegate.java:75)
       at org.jboss.resource.adapter.jms.inflow.JmsServerSession$XATransactionDemarcationStrategy.end(JmsServerSession.java:657)
       at org.jboss.resource.adapter.jms.inflow.JmsServerSession.run(JmsServerSession.java:243)
       at org.jboss.resource.work.WorkWrapper.execute(WorkWrapper.java:205)
       at org.jboss.util.threadpool.BasicTaskWrapper.run(BasicTaskWrapper.java:260)
       at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
       at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
       at java.lang.Thread.run(Thread.java:619)
      


      The worst thing is, once the exception is thrown, that node is effectively dead. No more messages get processed. My app has multiple queues, but once one queue on the node fails (say, the e-mail queue) then all queues stop working (say, the reports queues): they all report HeuristicMixedException whenever a messages is sent to them.

      I had this same error with JBoss MQ, and upgraded to JBoss Messaging in hopes it would be fixed. But no luck! Can anybody suggest why this error happens, and whether there is some setting to automatically recover from it? If I manually restart the node it works fine.

      Regards,

      Richard

        • 1. Re: HeuristicMixedException: cause and remedy?
          gaohoward

          Hi Richard,

          Do you have more detailed log around this exception? I didn't see the exception was thrown from JMS code.

          • 2. Re: HeuristicMixedException: cause and remedy?
            kennardconsulting

            Howard,

            Wow. I swear you guys reply faster than any support forum I have ever used. Thank you!

            No, I'm afraid the Exception I posted is the entire thing. I assume it's JBoss Messaging related because it comes about at the end of the JmsServerSession, but if this question is better posted elsewhere I'd be happy to.

            Regards,

            Richard.

            • 3. Re: HeuristicMixedException: cause and remedy?
              timfox

              The exception is being thrown from the transaction manager code (JBoss Transactions) which has been called from the JCA adapter with a call to commit a transaction.

              I'd say your best bet is to try the JCA user forum. (or JBoss TS forum)

              • 4. Re: HeuristicMixedException: cause and remedy?
                kennardconsulting

                Tim,

                Thanks. I have posted a message on the JCA forum.

                Please note I may not be the only one seeing this. This JIRA seems to indicate they see it intermittently too:

                https://jira.jboss.org/jira/browse/JBESB-2484

                It says "JBoss Messaging sometimes causes a HeuristicMixedException to be raised by the transaction manager during commit. Once this happens the associated session would appear to be destabilised in that subsequent use does not appear possible... The exceptions are always generated with the same session id which would suggest that there is an underlying issue in the JBoss Messaging codebase. We need to handle this until the issue is resolved within JBM."

                Their JIRA is more concerned with working around the problem, but it was only opened in March 2009. Perhaps you could ask the ESB guys about it?

                • 5. Re: HeuristicMixedException: cause and remedy?
                  vink

                  Hello,

                  I'm also receiving HeuristicMixedException. Please suggest.
                  I'm using Messaging 1.4.4.GA & Remoting 2.2.3.GA.

                  <log4j:event logger="com.arjuna.ats.jta.logging.loggerI18N" timestamp="1248757363467" level="WARN" thread="WorkManager(2)-34">
                  <log4j:message><![CDATA[[com.arjuna.ats.internal.jta.resources.arjunacore.preparefailed] [com.arjuna.ats.internal.jta.resources.arjunacore.preparefailed] XAResourceRecord.prepare - prepare failed with exception XAException.XAER_RMERR]]></log4j:message>
                  </log4j:event>
                  
                  <log4j:event logger="com.arjuna.ats.arjuna.logging.arjLoggerI18N" timestamp="1248757363483" level="WARN" thread="WorkManager(2)-34">
                  <log4j:message><![CDATA[[com.arjuna.ats.arjuna.coordinator.BasicAction_50] - Prepare phase of action -6961b6a3:7d5:4a6e852d:4fd received heuristic decision: TwoPhaseOutcome.HEURISTIC_HAZARD]]></log4j:message>
                  </log4j:event>
                  
                  <log4j:event logger="com.arjuna.ats.arjuna.logging.arjLoggerI18N" timestamp="1248757363483" level="WARN" thread="WorkManager(2)-34">
                  <log4j:message><![CDATA[[com.arjuna.ats.arjuna.coordinator.BasicAction_36] - BasicAction.End() - prepare phase of action-id -6961b6a3:7d5:4a6e852d:4fd failed.]]></log4j:message>
                  </log4j:event>
                  
                  <log4j:event logger="com.arjuna.ats.arjuna.logging.arjLoggerI18N" timestamp="1248757363483" level="WARN" thread="WorkManager(2)-34">
                  <log4j:message><![CDATA[[com.arjuna.ats.arjuna.coordinator.BasicAction_37] - Received heuristic: TwoPhaseOutcome.HEURISTIC_HAZARD .]]></log4j:message>
                  </log4j:event>
                  
                  <log4j:event logger="com.arjuna.ats.arjuna.logging.arjLoggerI18N" timestamp="1248757363483" level="WARN" thread="WorkManager(2)-34">
                  <log4j:message><![CDATA[[com.arjuna.ats.arjuna.coordinator.BasicAction_38] - Action Aborting]]></log4j:message>
                  </log4j:event>
                  
                  <log4j:event logger="org.jboss.resource.adapter.jms.inflow.JmsServerSession" timestamp="1248757363779" level="ERROR" thread="WorkManager(2)-31">
                  <log4j:message><![CDATA[org.jboss.resource.adapter.jms.inflow.JmsServerSession@113da2c failed to commit/rollback]]></log4j:message>
                  <log4j:throwable><![CDATA[javax.transaction.HeuristicMixedException
                   at com.arjuna.ats.internal.jta.transaction.arjunacore.TransactionImple.commitAndDisassociate(TransactionImple.java:1397)
                   at com.arjuna.ats.internal.jta.transaction.arjunacore.BaseTransaction.commit(BaseTransaction.java:135)
                   at com.arjuna.ats.jbossatx.BaseTransactionManagerDelegate.commit(BaseTransactionManagerDelegate.java:87)
                   at org.jboss.resource.adapter.jms.inflow.JmsServerSession$XATransactionDemarcationStrategy.end(JmsServerSession.java:494)
                   at org.jboss.resource.adapter.jms.inflow.JmsServerSession.run(JmsServerSession.java:248)
                   at org.jboss.resource.work.WorkWrapper.execute(WorkWrapper.java:204)
                   at org.jboss.util.threadpool.BasicTaskWrapper.run(BasicTaskWrapper.java:275)
                   at EDU.oswego.cs.dl.util.concurrent.PooledExecutor$Worker.run(PooledExecutor.java:756)
                   at java.lang.Thread.run(Thread.java:619)
                  ]]></log4j:throwable>
                  </log4j:event>



                  Thanks

                  Vinay

                  • 6. Re: HeuristicMixedException: cause and remedy?
                    kennardconsulting
                    • 7. Re: HeuristicMixedException: cause and remedy?
                      timfox

                      I have commented on the JIRA, but I do not understand why you think this a JBoss Messaging issue.

                      Not one of the classes on the provided stack trace is a JBoss Messaging class!

                      • 8. Re: HeuristicMixedException: cause and remedy?
                        varsha7

                        Has anyone found a solution to this issue?

                         

                        Thanks

                        • 9. Re: HeuristicMixedException: cause and remedy?
                          watchamcb

                          The cause of this issue in our production environment appears to be result of the maxSession attribute on the MDB being greater than the max-pool-size in the DataSource descriptor. Under heavy load the MDB was unable to retrieve a connection from the pool and would try and roll-back the transaction which would then throw HeuristicMixedException. In JBoss 4.2.2 the stack trace on the actual cause is below, issue was resolved by increasing the pool size in *-ds.xml descriptor to be greater than the MDB session size.

                           

                          Caused by: javax.resource.ResourceException: No ManagedConnections available within configured blocking timeout ( 30000 [ms] )

                                  at org.jboss.resource.connectionmanager.InternalManagedConnectionPool.getConnection(InternalManagedConnectionPool.java:301)

                                  at org.jboss.resource.connectionmanager.JBossManagedConnectionPool$BasePool.getConnection(JBossManagedConnectionPool.java:538)

                                  at org.jboss.resource.connectionmanager.BaseConnectionManager2.getManagedConnection(BaseConnectionManager2.java:341)

                                  at org.jboss.resource.connectionmanager.TxConnectionManager.getManagedConnection(TxConnectionManager.java:315)

                                  at org.jboss.resource.connectionmanager.BaseConnectionManager2.allocateConnection(BaseConnectionManager2.java:396)

                                  at org.jboss.resource.connectionmanager.BaseConnectionManager2$ConnectionManagerProxy.allocateConnection(BaseConnectionManager2.java:842)

                                  at org.jboss.resource.adapter.jdbc.WrapperDataSource.getConnection(WrapperDataSource.java:88)

                           

                          Hope this helps...