How to prevent ApplicationDeadlockException with READ-ONLY m

[aparaapara]

I am running jboss 3.2.7 on Linux.

I have examined my application and marked all read-only method with a tag (in jboss.xml):

<method-attributes>

<method-name>get*</method-name>
<read-only>true</read-only>

</method-attributes>

I also added similar tags for is* and any other read-only operations. I have added these to the Remote Session and Local Entity objects.

But I am still getting ApplicationDeadlockException during what should be read-only operations. To avoid multiple SELECT statements from the database, I am using standard Requires or Supports TX attributes as far as standard J2EE is concerned.

Can someone help me understand or tell me how to trace this issue? What else should I check?

Here is the exception which I get:

06:36:50,406 ERROR [LogInterceptor] TransactionRolledbackLocalException in method: public abstract boolean com.myprofiles.model.profile.ProfileEntity.isPrivateProfile(), causedBy:
org.jboss.util.deadlock.ApplicationDeadlockException: Application deadlock detected, resource=org.jboss.ejb.plugins.lock.QueuedPessimisticEJBLock@3b5a5b, bean=Profile, id=com.standardset.ejb.common.base.EntityKey@311dacf id: 65581, refs=4, tx=TransactionImpl:XidImpl [FormatId=257, GlobalId=cp//14782, BranchQual=], synched=Thread[http-0.0.0.0-8080-Processor49,5,jboss], timeout=5000, queue=[TXLOCK waitingTx=TransactionImpl:XidImpl [FormatId=257, GlobalId=cp//14786, BranchQual=] id=0 thread=Thread[http-0.0.0.0-8080-Processor39,5,jboss] queued=true, TXLOCK waitingTx=TransactionImpl:XidImpl [FormatId=257, GlobalId=cp//14784, BranchQual=] id=0 thread=Thread[http-0.0.0.0-8080-Processor2,5,jboss] queued=true], holder=TransactionImpl:XidImpl [FormatId=257, GlobalId=cp//14788, BranchQual=], waitingResource=org.jboss.ejb.plugins.lock.QueuedPessimisticEJBLock@1ef5dfc, bean=PictureValue, id=com.standardset.ejb.common.base.EntityKey@311de71 id: 65671, refs=2, tx=TransactionImpl:XidImpl [FormatId=257, GlobalId=cp//14773, BranchQual=], synched=null, timeout=5000, queue=[TXLOCK waitingTx=TransactionImpl:XidImpl [FormatId=257, GlobalId=cp//14782, BranchQual=] id=0 thread=Thread[http-0.0.0.0-8080-Processor37,5,jboss] queued=true], waitingResourceHolder=TransactionImpl:XidImpl [FormatId=257, GlobalId=cp//14773, BranchQual=]
at org.jboss.util.deadlock.DeadlockDetector.deadlockDetection(DeadlockDetector.java:48)
at org.jboss.ejb.plugins.lock.QueuedPessimisticEJBLock.waitForTx(QueuedPessimisticEJBLock.java:301)
at org.jboss.ejb.plugins.lock.QueuedPessimisticEJBLock.doSchedule(QueuedPessimisticEJBLock.java:236)
at org.jboss.ejb.plugins.lock.QueuedPessimisticEJBLock.schedule(QueuedPessimisticEJBLock.java:183)
at org.jboss.ejb.plugins.EntityLockInterceptor.invoke(EntityLockInterceptor.java:85)
at org.jboss.ejb.plugins.EntityCreationInterceptor.invoke(EntityCreationInterceptor.java:54)
at org.jboss.ejb.plugins.AbstractTxInterceptor.invokeNext(AbstractTxInterceptor.java:84)
at org.jboss.ejb.plugins.TxInterceptorCMT.runWithTransactions(TxInterceptorCMT.java:343)
at org.jboss.ejb.plugins.TxInterceptorCMT.invoke(TxInterceptorCMT.java:150)
at org.jboss.ejb.plugins.SecurityInterceptor.invoke(SecurityInterceptor.java:111)
at org.jboss.ejb.plugins.LogInterceptor.invoke(LogInterceptor.java:192)
at org.jboss.ejb.plugins.ProxyFactoryFinderInterceptor.invoke(ProxyFactoryFinderInterceptor.java:122)
at org.jboss.ejb.EntityContainer.internalInvoke(EntityContainer.java:484)
at org.jboss.ejb.Container.invoke(Container.java:709)
at org.jboss.ejb.plugins.local.BaseLocalProxyFactory.invoke(BaseLocalProxyFactory.java:419)
at org.jboss.ejb.plugins.local.EntityProxy.invoke(EntityProxy.java:44)
at $Proxy171.isPrivateProfile(Unknown Source)
at com.myprofiles.model.security.SecuritySessionBean.assertMemberCanSeeComponent(SecuritySessionBean.java:272)
at sun.reflect.GeneratedMethodAccessor116.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:324)
at org.jboss.ejb.StatelessSessionContainer$ContainerInterceptor.invoke(StatelessSessionContainer.java:683)
at org.jboss.resource.connectionmanager.CachedConnectionInterceptor.invoke(CachedConnectionInterceptor.java:186)
at org.jboss.ejb.plugins.StatelessSessionInstanceInterceptor.invoke(StatelessSessionInstanceInterceptor.java:72)
at org.jboss.ejb.plugins.AbstractTxInterceptor.invokeNext(AbstractTxInterceptor.java:84)
at org.jboss.ejb.plugins.TxInterceptorCMT.runWithTransactions(TxInterceptorCMT.java:343)
at org.jboss.ejb.plugins.TxInterceptorCMT.invoke(TxInterceptorCMT.java:150)
at org.jboss.ejb.plugins.SecurityInterceptor.invoke(SecurityInterceptor.java:111)
at org.jboss.ejb.plugins.LogInterceptor.invoke(LogInterceptor.java:192)
at org.jboss.ejb.plugins.ProxyFactoryFinderInterceptor.invoke(ProxyFactoryFinderInterceptor.java:122)
at org.jboss.ejb.StatelessSessionContainer.internalInvoke(StatelessSessionContainer.java:331)
at org.jboss.ejb.Container.invoke(Container.java:709)
at org.jboss.ejb.plugins.local.BaseLocalProxyFactory.invoke(BaseLocalProxyFactory.java:419)
at org.jboss.ejb.plugins.local.StatelessSessionProxy.invoke(StatelessSessionProxy.java:83)
at $Proxy211.assertMemberCanSeeComponent(Unknown Source)
at com.myprofiles.service.component.MemberComponentServiceBean.getPictureComponentData(MemberComponentServiceBean.java:624)
at sun.reflect.GeneratedMethodAccessor115.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:324)
at org.jboss.ejb.StatelessSessionContainer$ContainerInterceptor.invoke(StatelessSessionContainer.java:683)
at org.jboss.resource.connectionmanager.CachedConnectionInterceptor.invoke(CachedConnectionInterceptor.java:186)
at org.jboss.ejb.plugins.StatelessSessionInstanceInterceptor.invoke(StatelessSessionInstanceInterceptor.java:72)
at org.jboss.ejb.plugins.AbstractTxInterceptor.invokeNext(AbstractTxInterceptor.java:84)
at org.jboss.ejb.plugins.TxInterceptorCMT.runWithTransactions(TxInterceptorCMT.java:317)
at org.jboss.ejb.plugins.TxInterceptorCMT.invoke(TxInterceptorCMT.java:150)
at org.jboss.ejb.plugins.SecurityInterceptor.invoke(SecurityInterceptor.java:111)
at org.jboss.ejb.plugins.LogInterceptor.invoke(LogInterceptor.java:192)
at org.jboss.ejb.plugins.ProxyFactoryFinderInterceptor.invoke(ProxyFactoryFinderInterceptor.java:122)
at org.jboss.ejb.StatelessSessionContainer.internalInvoke(StatelessSessionContainer.java:331)
at org.jboss.ejb.Container.invoke(Container.java:709)
at sun.reflect.GeneratedMethodAccessor84.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:324)
at org.jboss.mx.server.ReflectedDispatcher.dispatch(ReflectedDispatcher.java:60)
at org.jboss.mx.server.Invocation.dispatch(Invocation.java:62)
at org.jboss.mx.server.Invocation.dispatch(Invocation.java:54)
at org.jboss.mx.server.Invocation.invoke(Invocation.java:82)
at org.jboss.mx.server.AbstractMBeanInvoker.invoke(AbstractMBeanInvoker.java:198)
at org.jboss.mx.server.MBeanServerImpl.invoke(MBeanServerImpl.java:473)
at org.jboss.invocation.local.LocalInvoker.invoke(LocalInvoker.java:97)
at org.jboss.invocation.InvokerInterceptor.invokeLocal(InvokerInterceptor.java:115)
at org.jboss.invocation.InvokerInterceptor.invoke(InvokerInterceptor.java:101)
at org.jboss.proxy.TransactionInterceptor.invoke(TransactionInterceptor.java:46)
at org.jboss.proxy.SecurityInterceptor.invoke(SecurityInterceptor.java:55)
at org.jboss.proxy.ejb.StatelessSessionInterceptor.invoke(StatelessSessionInterceptor.java:100)
at org.jboss.proxy.ClientContainer.invoke(ClientContainer.java:86)
at $Proxy218.getPictureComponentData(Unknown Source)
at com.myprofiles.client.web.picture.PictureAction.execute(PictureAction.java:75)
at org.apache.struts.action.RequestProcessor.processActionPerform(RequestProcessor.java:484)
at org.apache.struts.action.RequestProcessor.process(RequestProcessor.java:274)
at org.apache.struts.action.ActionServlet.process(ActionServlet.java:1482)
at org.apache.struts.action.ActionServlet.doGet(ActionServlet.java:507)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:697)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:810)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:252)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:173)
at com.standardset.web.filter.JBossTomCatPrincipalFilter.doFilter(JBossTomCatPrincipalFilter.java:114)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:202)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:173)
at com.standardset.web.filter.UserLoginTriggerFilter.doFilter(UserLoginTriggerFilter.java:134)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:202)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:173)
at org.jboss.web.tomcat.filters.ReplyHeaderFilter.doFilter(ReplyHeaderFilter.java:75)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:202)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:173)
at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:214)
at org.apache.catalina.core.StandardValveContext.invokeNext(StandardValveContext.java:104)
at org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.java:520)
at org.apache.catalina.core.StandardContextValve.invokeInternal(StandardContextValve.java:198)
at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:152)
at org.apache.catalina.core.StandardValveContext.invokeNext(StandardValveContext.java:104)
at org.jboss.web.tomcat.security.CustomPrincipalValve.invoke(CustomPrincipalValve.java:66)
at org.apache.catalina.core.StandardValveContext.invokeNext(StandardValveContext.java:102)
at org.jboss.web.tomcat.security.SecurityAssociationValve.invoke(SecurityAssociationValve.java:162)
at org.apache.catalina.core.StandardValveContext.invokeNext(StandardValveContext.java:102)
at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:462)
at org.apache.catalina.core.StandardValveContext.invokeNext(StandardValveContext.java:102)
at org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.java:520)
at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:137)
at org.apache.catalina.core.StandardValveContext.invokeNext(StandardValveContext.java:104)
at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:118)
at org.apache.catalina.core.StandardValveContext.invokeNext(StandardValveContext.java:102)
at org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:535)
at org.apache.catalina.core.StandardValveContext.invokeNext(StandardValveContext.java:102)
at org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.java:520)
at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109)
at org.apache.catalina.core.StandardValveContext.invokeNext(StandardValveContext.java:104)
at org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.java:520)
at org.apache.catalina.core.ContainerBase.invoke(ContainerBase.java:929)
at org.apache.coyote.tomcat5.CoyoteAdapter.service(CoyoteAdapter.java:160)
at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:799)
at org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.processConnection(Http11Protocol.java:705)
at org.apache.tomcat.util.net.TcpWorkerThread.runIt(PoolTcpEndpoint.java:577)
at org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run(ThreadPool.java:684)
at java.lang.Thread.run(Thread.java:534)

[aparaapara]

Has anyone seen this problem? What would cause a deadlock exception when all the methods are marked as READ-ONLY?

Thanks.
-AP_

[plutus]

Hi,

we have _exactly_ the same problem. To our server there are ~100 clients connected and these deadlocks make us crazy and our customers very angry. After the appearance of "deadlock detections" the whole server blocks and only 'kill -9' helps :-(. This is so bad. Please support this thread, please provide better documentation...!

Marcel

[ereze]

I have the same problem: methods marked with read-only but still lots of deadlocks which obviously make our customer mad.

Any question I had regarding this issue was left unanswered by JBoss which I guess does not have a clue too.
Maybe I need to come down from the JBoss and go with something else. It's been fun with < 100 online clients but now it's really not going anywhere.

I hope someone from the BIG JBoss would come down to us earthy developers and gave some answers. But I guess that will not happen too.

[khamre]

We too have _exactly_ the same problem. In our case we even suspect that the condition for a deadlock is not present. Two resources does not seem to be reserved in opposite order. The exception is triggered when a message bean eats consequtive events from an event queue. Is it possible that deadlock can occur when two identical (message)beans run the identical piece of code on identical events???

[vickyk]

"ereze" wrote:
I have the same problem: methods marked with read-only but still lots of deadlocks which obviously make our customer mad.

I am not sure but the read only will be used in the optimization technique for the 2P-Protocol ..It will not help in deadlock situation .

Regards
Vicky

[odg]

Hi, at my company we had similair problems but we solved them by marking the whole bean as read-only ( luky for us they were :-) ) this prevents the beans to be associated with the transaction and therfore prevents deadlocks

[_monkey_]

Hi there,

I think you have to make JBOSS use SimpleReadWriteEJBLock instead of the default PessimisticQueueLock, by including the following block in jboss.xml:

<container-configuration extends="Standard CMP 2.x EntityBean">
<container-name>Standard CMP 2.x EntityBean</container-name> <locking-policy>org.jboss.ejb.plugins.lock.SimpleReadWriteEJBLock</locking-policy>
</container-configuration>

This is what I did, and it did not give me any more transaction locking errors, but I do find this approach slower, it seems to load all the data from a table instead of just pulling the required row for me... I just posted a query on this forum about this and hope someone can help me with that :)