1 Reply Latest reply on Jun 3, 2004 1:49 AM by jdenly

Problem with concurrent CMP creates in cluster

jdenly Jun 3, 2004 1:21 AM

Hi all,

I have JBoss 3.2.3 setup in a cluster fronted by Apache 2.0.3 using the mod_jk2 connector. I am using commit option A in conjunction with the distributed cache invalidation framework, backed by a SQL Server 2000 cluster using the I-Net Sprinta JDBC driver. In addition, Apache is set for sticky sessions and no Http session replication is used. Everything seems to work fine apart from concurrent creates of entities. The system has multiple CMP entity beans, fronted by stateless session beans, with tx required on their methods. The entity beans use local interfaces and have tx supports on their methods. All of the getters of the entity beans are marked as read-only for performance reasons. If multiple creates occur simultaniously, I get the following exception:

14:51:16,819 ERROR [Engine] StandardWrapperValve[action]: Servlet.service() for servlet action threw exception
org.jboss.tm.JBossTransactionRolledbackException: null; nested exception is:
 org.jboss.tm.JBossRollbackException: Unable to commit, tx=TransactionImpl:XidImpl [FormatId=257, GlobalId=ausyduxdev00//261, BranchQual=] status=STATUS_NO_TRANSACTION; - nested throwable: (javax.ejb.EJBException: Update failed. Expected one affected row: rowsAffected=0id=45); - nested throwable: (org.jboss.tm.JBossRollbackException: Unable to commit, tx=TransactionImpl:XidImpl [FormatId=257, GlobalId=ausyduxdev00//261, BranchQual=] status=STATUS_NO_TRANSACTION; - nested throwable: (javax.ejb.EJBException: Update failed. Expected one affected row: rowsAffected=0id=45))
 at org.jboss.ejb.plugins.TxInterceptorCMT.throwJBossException(TxInterceptorCMT.java:489)
 at org.jboss.ejb.plugins.TxInterceptorCMT.endTransaction(TxInterceptorCMT.java:403)
 at org.jboss.ejb.plugins.TxInterceptorCMT.runWithTransactions(TxInterceptorCMT.java:277)
 at org.jboss.ejb.plugins.TxInterceptorCMT.invoke(TxInterceptorCMT.java:128)
 at org.jboss.ejb.plugins.SecurityInterceptor.invoke(SecurityInterceptor.java:118)
 at org.jboss.ejb.plugins.LogInterceptor.invoke(LogInterceptor.java:191)
 at org.jboss.ejb.plugins.ProxyFactoryFinderInterceptor.invoke(ProxyFactoryFinderInterceptor.java:122)
 at org.jboss.ejb.StatelessSessionContainer.internalInvoke(StatelessSessionContainer.java:331)
 at org.jboss.ejb.Container.invoke(Container.java:700)
 at sun.reflect.GeneratedMethodAccessor79.invoke(Unknown Source)
 at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
 at java.lang.reflect.Method.invoke(Method.java:324)
 at org.jboss.mx.capability.ReflectedMBeanDispatcher.invoke(ReflectedMBeanDispatcher.java:284)
 at org.jboss.mx.server.MBeanServerImpl.invoke(MBeanServerImpl.java:546)
 at org.jboss.invocation.local.LocalInvoker.invoke(LocalInvoker.java:101)
 at org.jboss.invocation.InvokerInterceptor.invoke(InvokerInterceptor.java:90)
 at org.jboss.proxy.TransactionInterceptor.invoke(TransactionInterceptor.java:46)
 at org.jboss.proxy.SecurityInterceptor.invoke(SecurityInterceptor.java:45)
 at org.jboss.proxy.ejb.StatelessSessionInterceptor.invoke(StatelessSessionInterceptor.java:100)
 at org.jboss.proxy.ClientContainer.invoke(ClientContainer.java:85)
 at $Proxy84.updateCDE(Unknown Source)
 at au.com.eclipsegroup.egem.actions.cde.epr.UpdateEPRAction.perform(UpdateEPRAction.java:760)
 at org.apache.struts.action.ActionServlet.processActionPerform(ActionServlet.java:1787)
 at org.apache.struts.action.ActionServlet.process(ActionServlet.java:1586)
 at org.apache.struts.action.ActionServlet.doPost(ActionServlet.java:510)
 at javax.servlet.http.HttpServlet.service(HttpServlet.java:760)
 at javax.servlet.http.HttpServlet.service(HttpServlet.java:853)
 at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:247)
 at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:193)
 at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:256)
 at org.apache.catalina.core.StandardPipeline$StandardPipelineValveContext.invokeNext(StandardPipeline.java:643)
 at org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.java:480)
 at org.apache.catalina.core.ContainerBase.invoke(ContainerBase.java:995)
 at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:191)
 at org.apache.catalina.core.StandardPipeline$StandardPipelineValveContext.invokeNext(StandardPipeline.java:643)
 at org.jboss.web.tomcat.security.JBossSecurityMgrRealm.invoke(JBossSecurityMgrRealm.java:220)
 at org.apache.catalina.core.StandardPipeline$StandardPipelineValveContext.invokeNext(StandardPipeline.java:641)
 at org.apache.catalina.valves.CertificatesValve.invoke(CertificatesValve.java:246)
 at org.apache.catalina.core.StandardPipeline$StandardPipelineValveContext.invokeNext(StandardPipeline.java:641)
 at org.jboss.web.tomcat.tc4.statistics.ContainerStatsValve.invoke(ContainerStatsValve.java:76)
 at org.apache.catalina.core.StandardPipeline$StandardPipelineValveContext.invokeNext(StandardPipeline.java:641)
 at org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.java:480)
 at org.apache.catalina.core.ContainerBase.invoke(ContainerBase.java:995)
 at org.apache.catalina.core.StandardContext.invoke(StandardContext.java:2417)
 at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:180)
 at org.apache.catalina.core.StandardPipeline$StandardPipelineValveContext.invokeNext(StandardPipeline.java:643)
 at org.apache.catalina.valves.ErrorDispatcherValve.invoke(ErrorDispatcherValve.java:171)
 at org.apache.catalina.core.StandardPipeline$StandardPipelineValveContext.invokeNext(StandardPipeline.java:641)
 at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:172)
 at org.apache.catalina.core.StandardPipeline$StandardPipelineValveContext.invokeNext(StandardPipeline.java:641)
 at org.jboss.web.tomcat.security.SecurityAssociationValve.invoke(SecurityAssociationValve.java:65)
 at org.apache.catalina.core.StandardPipeline$StandardPipelineValveContext.invokeNext(StandardPipeline.java:641)
 at org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:577)
 at org.apache.catalina.core.StandardPipeline$StandardPipelineValveContext.invokeNext(StandardPipeline.java:641)
 at org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.java:480)
 at org.apache.catalina.core.ContainerBase.invoke(ContainerBase.java:995)
 at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:174)
 at org.apache.catalina.core.StandardPipeline$StandardPipelineValveContext.invokeNext(StandardPipeline.java:643)
 at org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.java:480)
 at org.apache.catalina.core.ContainerBase.invoke(ContainerBase.java:995)
 at org.apache.coyote.tomcat4.CoyoteAdapter.service(CoyoteAdapter.java:197)
 at org.apache.jk.server.JkCoyoteHandler.invoke(JkCoyoteHandler.java:309)
 at org.apache.jk.common.HandlerRequest.invoke(HandlerRequest.java:387)
 at org.apache.jk.common.ChannelSocket.invoke(ChannelSocket.java:673)
 at org.apache.jk.common.ChannelSocket.processConnection(ChannelSocket.java:615)
 at org.apache.jk.common.SocketConnection.runIt(ChannelSocket.java:786)
 at org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run(ThreadPool.java:677)
 at java.lang.Thread.run(Thread.java:536)
Caused by: org.jboss.tm.JBossRollbackException: Unable to commit, tx=TransactionImpl:XidImpl [FormatId=257, GlobalId=ausyduxdev00//261, BranchQual=] status=STATUS_NO_TRANSACTION; - nested throwable: (javax.ejb.EJBException: Update failed. Expected one affected row: rowsAffected=0id=45)
 at org.jboss.tm.TransactionImpl.commit(TransactionImpl.java:413)
 at org.jboss.ejb.plugins.TxInterceptorCMT.endTransaction(TxInterceptorCMT.java:398)
 ... 66 more
Caused by: javax.ejb.EJBException: Update failed. Expected one affected row: rowsAffected=0id=45
 at org.jboss.ejb.plugins.cmp.jdbc.JDBCStoreEntityCommand.execute(JDBCStoreEntityCommand.java:155)
 at org.jboss.ejb.plugins.cmp.jdbc.JDBCStoreManager.storeEntity(JDBCStoreManager.java:627)
 at org.jboss.ejb.plugins.CMPPersistenceManager.storeEntity(CMPPersistenceManager.java:421)
 at org.jboss.resource.connectionmanager.CachedConnectionInterceptor.storeEntity(CachedConnectionInterceptor.java:387)
 at org.jboss.ejb.EntityContainer.storeEntity(EntityContainer.java:714)
 at org.jboss.ejb.GlobalTxEntityMap.synchronizeEntities(GlobalTxEntityMap.java:149)
 at org.jboss.ejb.GlobalTxEntityMap$GlobalTxEntityMapSynchronize.beforeCompletion(GlobalTxEntityMap.java:215)
 at org.jboss.tm.TransactionImpl.doBeforeCompletion(TransactionImpl.java:1308)
 at org.jboss.tm.TransactionImpl.commit(TransactionImpl.java:347)
 ... 67 more

Interestingly, if I run exactly the same test with one node of the two node cluster shut down, everything works fine.

Any ideas?

Jon

1. Re: Problem with concurrent CMP creates in cluster

jdenly Jun 3, 2004 1:49 AM (in response to jdenly)

Hmmm - having delved a bit further by turning on the SQL debugging (as per the Wiki - very useful!), it appears as though I have a problem with deadlocking occuring that is being hidden by some suspect exception handling on my part. When creating a new entity, it is associated with the id of another entity of the same type by running a finder on the same entity type within the business method doing the create. This is deadlocking and causing the whole tx to roll back. Looks like the issue is an application level one and I might be better to look up the parent entity before getting to the save, rather than inside the same transaction. Not sure why it only shows up in the cluster though...

Thanks for listening! Nothing like explaining the problem to someone else to help in solving it ;)

Jon
Actions