Random JMS Message Delivery Failures.

[stewar]

I have deployed a JMS producer and Consumer in Jboss As 6.1.0 final.

Some times i see exceptions on the logs like :

2012-09-10 17:37:13,039 ERROR [org.hornetq.ra.inflow.HornetQMessageHandler] Failed to deliver message

java.lang.IllegalStateException: This message endpoint + ExampleMDB is already in use by another thread Thread[Thread-3 (group:HornetQ-client-global-threads-47122773),5,]

        at org.jboss.ejb3.mdb.inflow.MessageInflowLocalProxy.invoke(MessageInflowLocalProxy.java:118)

        at $Proxy2015.beforeDelivery(Unknown Source)

        at org.hornetq.ra.inflow.HornetQMessageHandler.onMessage(HornetQMessageHandler.java:267)

        at org.hornetq.core.client.impl.ClientConsumerImpl.callOnMessage(ClientConsumerImpl.java:866)

        at org.hornetq.core.client.impl.ClientConsumerImpl.access$100(ClientConsumerImpl.java:44)

        at org.hornetq.core.client.impl.ClientConsumerImpl$Runner.run(ClientConsumerImpl.java:983)

        at org.hornetq.utils.OrderedExecutorFactory$OrderedExecutor$1.run(OrderedExecutorFactory.java:100)

        at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)

        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)

        at java.lang.Thread.run(Thread.java:662)

2012-09-10 17:37:13,371 ERROR [org.hornetq.ra.inflow.HornetQMessageHandler] Failed to deliver message

java.lang.IllegalStateException: This message endpoint + ExampleMDB is already in use by another thread Thread[Thread-4 (group:HornetQ-client-global-threads-47122773),5,]

        at org.jboss.ejb3.mdb.inflow.MessageInflowLocalProxy.invoke(MessageInflowLocalProxy.java:118)

        at $Proxy2015.beforeDelivery(Unknown Source)

        at org.hornetq.ra.inflow.HornetQMessageHandler.onMessage(HornetQMessageHandler.java:267)

        at org.hornetq.core.client.impl.ClientConsumerImpl.callOnMessage(ClientConsumerImpl.java:866)

        at org.hornetq.core.client.impl.ClientConsumerImpl.access$100(ClientConsumerImpl.java:44)

        at org.hornetq.core.client.impl.ClientConsumerImpl$Runner.run(ClientConsumerImpl.java:983)

        at org.hornetq.utils.OrderedExecutorFactory$OrderedExecutor$1.run(OrderedExecutorFactory.java:100)

        at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)

        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)

        at java.lang.Thread.run(Thread.java:662)

2012-09-10 17:37:13,509 ERROR [org.hornetq.ra.inflow.HornetQMessageHandler] Failed to deliver message

java.lang.IllegalStateException: This message endpoint + ExampleMDB is already in use by another thread Thread[Thread-6 (group:HornetQ-client-global-threads-47122773),5,]

        at org.jboss.ejb3.mdb.inflow.MessageInflowLocalProxy.invoke(MessageInflowLocalProxy.java:118)

        at $Proxy2015.beforeDelivery(Unknown Source)

        at org.hornetq.ra.inflow.HornetQMessageHandler.onMessage(HornetQMessageHandler.java:267)

        at org.hornetq.core.client.impl.ClientConsumerImpl.callOnMessage(ClientConsumerImpl.java:866)

        at org.hornetq.core.client.impl.ClientConsumerImpl.access$100(ClientConsumerImpl.java:44)

        at org.hornetq.core.client.impl.ClientConsumerImpl$Runner.run(ClientConsumerImpl.java:983)

        at org.hornetq.utils.OrderedExecutorFactory$OrderedExecutor$1.run(OrderedExecutorFactory.java:100)

        at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)

        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)

        at java.lang.Thread.run(Thread.java:662)

and when i check the Queue using twiddle script i see... messages are getting accumulated which are not delivered.

the output looks like this.

Selector=null

Paused=false

JNDIBindings=[Ljava.lang.String;@682bc3f5

DeadLetterAddress=jms.queue.DLQ

ExpiryAddress=jms.queue.ExpiryQueue

ConsumerCount=15

ScheduledCount=0

Address=jms.queue.ExampleQueue

Name=ExampleQueue

Temporary=false

MessageCount=1664

DeliveringCount=1664

MessagesAdded=8377

from the above its clear that Messages added were 8377 but 1664 are still unprocessed... and this has stayed for over 24 hrs now.

So they are stuck.

I googled and found that https://community.jboss.org/thread/149368  the above issue could be because of an exception which has occured before.

but i see the exceptions one after the other...

So not able to find cause of this?

[clebert.suconic]

There were a few issues that we fixed on the reousrce adapter after 2.2.5. It's hard for us to tell...

Anyway you could move to Jboss7, since it's easier to migrate to a newer version there?

[stewar]

Upgrading to Jboss 7 seems an option i dont have.

However i just drilled up the log files and saw an exception on same MDB earlier :

11:41:47,688 ERROR [org.hornetq.ra.inflow.HornetQMessageHandler] Failed to deliver message

javax.ejb.EJBTransactionRolledbackException: Component ExampleComponent rejected message.

Caused by: org.springframework.jdbc.UncategorizedSQLException: Hibernate flushing: could not update: [com.example.ExampleAgent#835aaa67-ede7-4f99-a0a7-9b0c3f683af5]; uncategorized SQLException for SQL [update agent set active=?, agent_address=?, agent_host_address=?, agent_host_name=?, customer_id=?, embedded_ind=?, heartbeat_interval_num=?, heartbeat_status_code=?, heartbeat_ts=?, heartbeat_threshold_num=?, other_flag=?, solution_element_ref_num=?, supported_product_list=? where agent_ref_num=?]; SQL state [null]; error code [0]; Error; - nested throwable: (java.lang.reflect.UndeclaredThrowableException); nested exception is org.jboss.util.NestedSQLException: Error; - nested throwable: (java.lang.reflect.UndeclaredThrowableException)

        at org.springframework.jdbc.support.SQLStateSQLExceptionTranslator.translate(SQLStateSQLExceptionTranslator.java:124)

        at org.springframework.jdbc.support.SQLErrorCodeSQLExceptionTranslator.translate(SQLErrorCodeSQLExceptionTranslator.java:322)

        at org.springframework.orm.hibernate3.HibernateTransactionManager.convertJdbcAccessException(HibernateTransactionManager.java:750)

        at org.springframework.orm.hibernate3.HibernateTransactionManager.convertHibernateAccessException(HibernateTransactionManager.java:736)

        at org.springframework.orm.hibernate3.HibernateTransactionManager.doCommit(HibernateTransactionManager.java:614)

        at org.springframework.transaction.support.AbstractPlatformTransactionManager.processCommit(AbstractPlatformTransactionManager.java:709)

        at org.springframework.transaction.support.AbstractPlatformTransactionManager.commit(AbstractPlatformTransactionManager.java:678)

        at org.springframework.transaction.interceptor.TransactionAspectSupport.commitTransactionAfterReturning(TransactionAspectSupport.java:321)

        at org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:116)

        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:171)

        at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:204)

        at $Proxy1921.update(Unknown Source)

        at com.example.ExampleComponent.updateHB(ExampleComponent.java:346)

        at com.transport.enterprise.ejb.ExampleMDB.onMessage(TransportMDB.java:241)

        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)

        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)

        at java.lang.reflect.Method.invoke(Method.java:597)

        at org.jboss.aop.joinpoint.MethodInvocation.invokeTarget(MethodInvocation.java:122)

        at org.jboss.aop.joinpoint.MethodInvocation.invokeNext(MethodInvocation.java:111)

        at org.jboss.ejb3.interceptors.container.ContainerMethodInvocationWrapper.invokeNext(ContainerMethodInvocationWrapper.java:72)

at org.jboss.ejb3.interceptors.aop.InterceptorSequencer.invoke(InterceptorSequencer.java:76)

        at org.jboss.ejb3.interceptors.aop.InterceptorSequencer.aroundInvoke(InterceptorSequencer.java:62)

        at sun.reflect.GeneratedMethodAccessor1617.invoke(Unknown Source)

        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)

        at java.lang.reflect.Method.invoke(Method.java:597)

        at org.jboss.aop.advice.PerJoinpointAdvice.invoke(PerJoinpointAdvice.java:174)

        at org.jboss.aop.joinpoint.MethodInvocation.invokeNext(MethodInvocation.java:102)

        at org.jboss.ejb3.interceptors.aop.InvocationContextInterceptor.fillMethod(InvocationContextInterceptor.java:74)

        at org.jboss.aop.advice.org.jboss.ejb3.interceptors.aop.InvocationContextInterceptor_z_fillMethod_1394272285.invoke(InvocationContextInterceptor_z_fillMethod_1394272285.java)

        at org.jboss.aop.joinpoint.MethodInvocation.invokeNext(MethodInvocation.java:102)

        at org.jboss.ejb3.interceptors.aop.InvocationContextInterceptor.setup(InvocationContextInterceptor.java:90)

        at org.jboss.aop.advice.org.jboss.ejb3.interceptors.aop.InvocationContextInterceptor_z_setup_1394272285.invoke(InvocationContextInterceptor_z_setup_1394272285.java)

        at org.jboss.aop.joinpoint.MethodInvocation.invokeNext(MethodInvocation.java:102)

        at org.jboss.ejb3.async.impl.interceptor.AsynchronousServerInterceptor.invoke(AsynchronousServerInterceptor.java:110)

        at org.jboss.aop.joinpoint.MethodInvocation.invokeNext(MethodInvocation.java:102)

        at org.jboss.ejb3.connectionmanager.CachedConnectionInterceptor.invoke(CachedConnectionInterceptor.java:62)

        at org.jboss.aop.joinpoint.MethodInvocation.invokeNext(MethodInvocation.java:102)

        at org.jboss.ejb3.entity.TransactionScopedEntityManagerInterceptor.invoke(TransactionScopedEntityManagerInterceptor.java:56)

        at org.jboss.aop.joinpoint.MethodInvocation.invokeNext(MethodInvocation.java:102)

        at org.jboss.ejb3.AllowedOperationsInterceptor.invoke(AllowedOperationsInterceptor.java:47)

        at org.jboss.aop.joinpoint.MethodInvocation.invokeNext(MethodInvocation.java:102)

        at org.jboss.ejb3.tx.NullInterceptor.invoke(NullInterceptor.java:42)

        at org.jboss.aop.joinpoint.MethodInvocation.invokeNext(MethodInvocation.java:102)

        at org.jboss.ejb3.stateless.StatelessInstanceInterceptor.invoke(StatelessInstanceInterceptor.java:68)

        at org.jboss.aop.joinpoint.MethodInvocation.invokeNext(MethodInvocation.java:102)

        at org.jboss.ejb3.core.context.InvocationContextAdapter.proceed(InvocationContextAdapter.java:70)

        at org.jboss.ejb3.tx2.impl.CMTTxInterceptor.invokeInCallerTx(CMTTxInterceptor.java:223)

        at org.jboss.ejb3.tx2.impl.CMTTxInterceptor.required(CMTTxInterceptor.java:353)

        at org.jboss.ejb3.tx2.impl.CMTTxInterceptor.invoke(CMTTxInterceptor.java:209)

        at org.jboss.ejb3.tx2.aop.CMTTxInterceptorWrapper.invoke(CMTTxInterceptorWrapper.java:52)

        at org.jboss.aop.joinpoint.MethodInvocation.invokeNext(MethodInvocation.java:102)

        at org.jboss.ejb3.tx.NullInterceptor.invoke(NullInterceptor.java:42)

        at org.jboss.aop.joinpoint.MethodInvocation.invokeNext(MethodInvocation.java:102)

        at org.jboss.ejb3.security.Ejb3AuthenticationInterceptorv2.invoke(Ejb3AuthenticationInterceptorv2.java:79)

        at org.jboss.aop.joinpoint.MethodInvocation.invokeNext(MethodInvocation.java:102)

        at org.jboss.ejb3.BlockContainerShutdownInterceptor.invoke(BlockContainerShutdownInterceptor.java:67)

        at org.jboss.aop.joinpoint.MethodInvocation.invokeNext(MethodInvocation.java:102)

        at org.jboss.ejb3.core.context.CurrentInvocationContextInterceptor.invoke(CurrentInvocationContextInterceptor.java:47)

        at org.jboss.aop.joinpoint.MethodInvocation.invokeNext(MethodInvocation.java:102)

        at org.jboss.aspects.currentinvocation.CurrentInvocationInterceptor.invoke(CurrentInvocationInterceptor.java:67)

        at org.jboss.aop.joinpoint.MethodInvocation.invokeNext(MethodInvocation.java:102)

          at org.jboss.ejb3.interceptor.EJB3TCCLInterceptor.invoke(EJB3TCCLInterceptor.java:86)

        at org.jboss.aop.joinpoint.MethodInvocation.invokeNext(MethodInvocation.java:102)

        at org.jboss.ejb3.mdb.MessagingContainer.localInvoke(MessagingContainer.java:306)

        at org.jboss.ejb3.mdb.inflow.MessageInflowLocalProxy.delivery(MessageInflowLocalProxy.java:299)

        at org.jboss.ejb3.mdb.inflow.MessageInflowLocalProxy.invoke(MessageInflowLocalProxy.java:152)

        at $Proxy1889.onMessage(Unknown Source)

        at org.hornetq.ra.inflow.HornetQMessageHandler.onMessage(HornetQMessageHandler.java:278)

        at org.hornetq.core.client.impl.ClientConsumerImpl.callOnMessage(ClientConsumerImpl.java:866)

        at org.hornetq.core.client.impl.ClientConsumerImpl.access$100(ClientConsumerImpl.java:44)

        at org.hornetq.core.client.impl.ClientConsumerImpl$Runner.run(ClientConsumerImpl.java:983)

        at org.hornetq.utils.OrderedExecutorFactory$OrderedExecutor$1.run(OrderedExecutorFactory.java:100)

        at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)

        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)

        at java.lang.Thread.run(Thread.java:662)

Caused by: org.jboss.util.NestedSQLException: Error; - nested throwable: (java.lang.reflect.UndeclaredThrowableException)

        at org.jboss.resource.adapter.jdbc.WrappedConnection.checkException(WrappedConnection.java:887)

        at org.jboss.resource.adapter.jdbc.WrappedConnection.prepareStatement(WrappedConnection.java:257)

        at org.hibernate.jdbc.AbstractBatcher.getPreparedStatement(AbstractBatcher.java:534)

        at org.hibernate.jdbc.AbstractBatcher.prepareStatement(AbstractBatcher.java:116)

        at org.hibernate.jdbc.AbstractBatcher.prepareStatement(AbstractBatcher.java:109)

        at org.hibernate.jdbc.AbstractBatcher.prepareBatchStatement(AbstractBatcher.java:244)

        at org.hibernate.persister.entity.AbstractEntityPersister.update(AbstractEntityPersister.java:2541)

        at org.hibernate.persister.entity.AbstractEntityPersister.updateOrInsert(AbstractEntityPersister.java:2494)

        at org.hibernate.persister.entity.AbstractEntityPersister.update(AbstractEntityPersister.java:2821)

        at org.hibernate.action.EntityUpdateAction.execute(EntityUpdateAction.java:113)

        at org.hibernate.engine.ActionQueue.execute(ActionQueue.java:273)

        at org.hibernate.engine.ActionQueue.executeActions(ActionQueue.java:265)

        at org.hibernate.engine.ActionQueue.executeActions(ActionQueue.java:185)

        at org.hibernate.event.def.AbstractFlushingEventListener.performExecutions(AbstractFlushingEventListener.java:321)

        at org.hibernate.event.def.DefaultFlushEventListener.onFlush(DefaultFlushEventListener.java:51)

        at org.hibernate.impl.SessionImpl.flush(SessionImpl.java:1216)

        at org.hibernate.impl.SessionImpl.managedFlush(SessionImpl.java:383)

        at org.hibernate.transaction.JTATransaction.commit(JTATransaction.java:143)

        at org.springframework.orm.hibernate3.HibernateTransactionManager.doCommit(HibernateTransactionManager.java:606)

        ... 75 more

Caused by: java.lang.reflect.UndeclaredThrowableException

        at $Proxy727.prepareStatement(Unknown Source)

        at org.jboss.resource.adapter.jdbc.BaseWrapperManagedConnection.doPrepareStatement(BaseWrapperManagedConnection.java:467)

        at org.jboss.resource.adapter.jdbc.BaseWrapperManagedConnection.prepareStatement(BaseWrapperManagedConnection.java:462)

        at org.jboss.resource.adapter.jdbc.WrappedConnection.prepareStatement(WrappedConnection.java:253)

        ... 92 more

Caused by: java.lang.reflect.InvocationTargetException

at sun.reflect.GeneratedMethodAccessor1464.invoke(Unknown Source)

        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)

        at java.lang.reflect.Method.invoke(Method.java:597)

        at org.postgresql.xa.PGXAConnection$ConnectionHandler.invoke(PGXAConnection.java:146)

        ... 96 more

Caused by: org.postgresql.util.PSQLException: Connection has been closed.

        at org.postgresql.ds.jdbc23.AbstractJdbc23PooledConnection$ConnectionHandler.invoke(AbstractJdbc23PooledConnection.java:297)

        at $Proxy727.prepareStatement(Unknown Source)

        ... 100 more

any ideas why i am seeing this exceptions, and after a few of these exceptions i actually hit the "MDB already in use errors" as reported before.

ERROR [org.hornetq.ra.inflow.HornetQMessageHandler] Failed to deliver message

java.lang.IllegalStateException: This message endpoint + ExampleMDB is already in use by another thread Thread[Thread-3 (group:HornetQ-client-global-threads-47122773),5,]

        at org.jboss.ejb3.mdb.inflow.MessageInflowLocalProxy.invoke(MessageInflowLocalProxy.java:118)

        at $Proxy2015.beforeDelivery(Unknown Source)

        at org.hornetq.ra.inflow.HornetQMessageHandler.onMessage(HornetQMessageHandler.java:267)

        at org.hornetq.core.client.impl.ClientConsumerImpl.callOnMessage(ClientConsumerImpl.java:866)

        at org.hornetq.core.client.impl.ClientConsumerImpl.access$100(ClientConsumerImpl.java:44)

        at org.hornetq.core.client.impl.ClientConsumerImpl$Runner.run(ClientConsumerImpl.java:983)

        at org.hornetq.utils.OrderedExecutorFactory$OrderedExecutor$1.run(OrderedExecutorFactory.java:100)

        at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)

        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)

        at java.lang.Thread.run(Thread.java:662)

[jbertram]

It seems clear to me that the MDB is failing initially because of an issue with the JDBC connection it is trying to use.  The first thing I would do (if you haven't already) is configure connection validation for your JDBC datasource so that it doesn't get a dead connection out of the pool.  That should mitigate the risk of this happening in the future.  Then I would fix your MDB so it doesn't throw any kind of RuntimeException out of onMessage() as this is a no-no in Java EE.  Once you do that let us know if you see this issue any more.

[stewar]

Thanks a ton Justin,

I am not seeing the Caused by: org.postgresql.util.PSQLException: Connection has been closed. exceptions any more.

after adding the connection validation for JDBC datasource.

i also removed the RuntimeExceptions which were thrown from the MDB onMessage()

but now iam seeing the following exceptions :

2012-09-18 02:47:57,531 ERROR [org.hornetq.ra.inflow.HornetQMessageHandler] Failed to deliver message

java.lang.reflect.UndeclaredThrowableException

        at $Proxy1782.afterDelivery(Unknown Source)

        at org.hornetq.ra.inflow.HornetQMessageHandler.onMessage(HornetQMessageHandler.java:287)

        at org.hornetq.core.client.impl.ClientConsumerImpl.callOnMessage(ClientConsumerImpl.java:866)

        at org.hornetq.core.client.impl.ClientConsumerImpl.access$100(ClientConsumerImpl.java:44)

        at org.hornetq.core.client.impl.ClientConsumerImpl$Runner.run(ClientConsumerImpl.java:983)

        at org.hornetq.utils.OrderedExecutorFactory$OrderedExecutor$1.run(OrderedExecutorFactory.java:100)

        at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)

        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)

        at java.lang.Thread.run(Thread.java:662)

Caused by: javax.resource.ResourceException: javax.transaction.HeuristicMixedException

        at org.jboss.ejb3.mdb.inflow.MessageInflowLocalProxy.after(MessageInflowLocalProxy.java:263)

        at org.jboss.ejb3.mdb.inflow.MessageInflowLocalProxy.invoke(MessageInflowLocalProxy.java:140)

        ... 9 more

Caused by: javax.transaction.HeuristicMixedException

        at com.arjuna.ats.internal.jta.transaction.arjunacore.TransactionImple.commitAndDisassociate(TransactionImple.java:1166)

        at com.arjuna.ats.internal.jta.transaction.arjunacore.BaseTransaction.commit(BaseTransaction.java:119)

        at com.arjuna.ats.jbossatx.BaseTransactionManagerDelegate.commit(BaseTransactionManagerDelegate.java:75)

        at org.jboss.ejb3.mdb.inflow.MessageInflowLocalProxy.endTransaction(MessageInflowLocalProxy.java:464)

        at org.jboss.ejb3.mdb.inflow.MessageInflowLocalProxy.finish(MessageInflowLocalProxy.java:343)

        at org.jboss.ejb3.mdb.inflow.MessageInflowLocalProxy.after(MessageInflowLocalProxy.java:259)

        ... 10 more

2012-09-18 02:47:57,534 ERROR [org.hornetq.core.client.impl.ClientConsumerImpl] Failed to call onMessage()

java.lang.IllegalStateException: afterDelivery without a previous beforeDelivery for message endpoint ExampleMDB

        at org.jboss.ejb3.mdb.inflow.MessageInflowLocalProxy.after(MessageInflowLocalProxy.java:254)

        at org.jboss.ejb3.mdb.inflow.MessageInflowLocalProxy.invoke(MessageInflowLocalProxy.java:140)

        at $Proxy1782.afterDelivery(Unknown Source)

        at org.hornetq.ra.inflow.HornetQMessageHandler.onMessage(HornetQMessageHandler.java:336)

        at org.hornetq.core.client.impl.ClientConsumerImpl.callOnMessage(ClientConsumerImpl.java:866)

        at org.hornetq.core.client.impl.ClientConsumerImpl.access$100(ClientConsumerImpl.java:44)

        at org.hornetq.core.client.impl.ClientConsumerImpl$Runner.run(ClientConsumerImpl.java:983)

        at org.hornetq.utils.OrderedExecutorFactory$OrderedExecutor$1.run(OrderedExecutorFactory.java:100)

        at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)

        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)

        at java.lang.Thread.run(Thread.java:662)

[jbertram]

Is the javax.transaction.HeuristicMixedException the only exception you see up until that point in your log?

What resources are you enlisting in your transaction?

[stewar]

Yes unfortunately this is the only exception that i see.

I have some @EJB's.

and i have a injected bean which inturn uses

@Resource

private javax.ejb.SessionContext ctx;

Iam using this sessionContext object to get the TimerService instance for creating timers.

other than this i dont have any other resources in the transaction.

Also i see some exceptions around this which i think are not related, so added in a new thread these are related to Jboss TimerService, but just had second thoughts so added to this thread as well:

https://community.jboss.org/message/761257#761257

[clebert.suconic]

Your best option on the community would be moving to AS7.

If you need to keep on AS5/6 style, I would recommend you going to EAP5. This seems a bug on the EJB container to me.

[stewar]

unfortunately i cant move to any other version, its very late in the cycle.

can you point me to a appropriate fix/workaround which i can fix in the source code if possible.?

[clebert.suconic]

It seems the issue is on the EJB container.

For HornetQ, you could try replacing hornetq on AS6. But that's probably more work than you upgrading to AS7 or using EAP5 which is closer to the setup you have.

EAP5 is a paid version, but it would probably be cheaper for you to use EAP5 instead of burning a lot of time on fixing stuff yourself.