3 Replies Latest reply on Mar 30, 2015 9:01 AM by akoskm

    Concurrent usage of TaskService

    akoskm

      The reproducer project I attached starts 10 threads simultaneously. Each thread creates a RuntimeManager and starts another thread. This thread uses the created RuntimeManager to resolve the user tasks in the started process.

      With enough threads (~10) sometimes the following exception appears:

       

      Exception in thread "9-ht-resolver" java.lang.RuntimeException: Unable to commit transaction
        at org.drools.persistence.jta.JtaTransactionManager.commit(JtaTransactionManager.java:229)
        at org.jbpm.services.task.persistence.TaskTransactionInterceptor.execute(TaskTransactionInterceptor.java:56)
        at org.drools.core.command.impl.AbstractInterceptor.executeNext(AbstractInterceptor.java:41)
        at org.drools.persistence.jta.TransactionLockInterceptor.execute(TransactionLockInterceptor.java:79)
        at org.jbpm.services.task.commands.TaskCommandExecutorImpl.execute(TaskCommandExecutorImpl.java:40)
        at org.jbpm.services.task.impl.command.CommandBasedTaskService.claim(CommandBasedTaskService.java:147)
        at org.jbpm.runtime.manager.impl.task.SynchronizedTaskService.claim(SynchronizedTaskService.java:102)
        at org.concurrency.HumanTaskResolver.run(ConcurrentHumanTaskTest.java:155)
        at java.lang.Thread.run(Thread.java:745)
      Caused by: bitronix.tm.internal.BitronixRollbackException: RuntimeException thrown during beforeCompletion cycle caused transaction rollback
        at bitronix.tm.BitronixTransaction.commit(BitronixTransaction.java:241)
        at bitronix.tm.BitronixTransactionManager.commit(BitronixTransactionManager.java:143)
        at org.drools.persistence.jta.JtaTransactionManager.commit(JtaTransactionManager.java:226)
        ... 8 more
      Caused by: javax.persistence.OptimisticLockException: Row was updated or deleted by another transaction (or unsaved-value mapping was incorrect): [org.jbpm.services.task.impl.model.TaskImpl#5768]
        at org.hibernate.ejb.AbstractEntityManagerImpl.wrapStaleStateException(AbstractEntityManagerImpl.java:1413)
        at org.hibernate.ejb.AbstractEntityManagerImpl.convert(AbstractEntityManagerImpl.java:1329)
        at org.hibernate.ejb.AbstractEntityManagerImpl.convert(AbstractEntityManagerImpl.java:1310)
        at org.hibernate.ejb.AbstractEntityManagerImpl.convert(AbstractEntityManagerImpl.java:1316)
        at org.hibernate.ejb.AbstractEntityManagerImpl$CallbackExceptionMapperImpl.mapManagedFlushFailure(AbstractEntityManagerImpl.java:1510)
        at org.hibernate.engine.transaction.synchronization.internal.SynchronizationCallbackCoordinatorNonTrackingImpl.beforeCompletion(SynchronizationCallbackCoordinatorNonTrackingImpl.java:114)
        at org.hibernate.engine.transaction.synchronization.internal.RegisteredSynchronization.beforeCompletion(RegisteredSynchronization.java:53)
        at bitronix.tm.BitronixTransaction.fireBeforeCompletionEvent(BitronixTransaction.java:532)
        at bitronix.tm.BitronixTransaction.commit(BitronixTransaction.java:235)
        ... 10 more
      

       

      Any help is appreciated.

      Thanks

        • 1. Re: Concurrent usage of TaskService
          akoskm

          Sometimes I see the following exception during ksession.startApplication:

           

          Exception in thread "1-process-runner" java.lang.RuntimeException: Unable to commit transaction
            at org.drools.persistence.jta.JtaTransactionManager.commit(JtaTransactionManager.java:229)
            at org.drools.persistence.SingleSessionCommandService$TransactionInterceptor.execute(SingleSessionCommandService.java:551)
            at org.drools.core.command.impl.AbstractInterceptor.executeNext(AbstractInterceptor.java:41)
            at org.drools.persistence.jpa.OptimisticLockRetryInterceptor.execute(OptimisticLockRetryInterceptor.java:73)
            at org.drools.core.command.impl.AbstractInterceptor.executeNext(AbstractInterceptor.java:41)
            at org.drools.persistence.jta.TransactionLockInterceptor.execute(TransactionLockInterceptor.java:79)
            at org.drools.persistence.SingleSessionCommandService.execute(SingleSessionCommandService.java:358)
            at org.drools.core.command.impl.CommandBasedStatefulKnowledgeSession.startProcess(CommandBasedStatefulKnowledgeSession.java:242)
            at org.concurrency.ProcessRunner.run(ConcurrentHumanTaskTest.java:128)
            at java.lang.Thread.run(Thread.java:745)
          Caused by: bitronix.tm.internal.BitronixRollbackException: RuntimeException thrown during beforeCompletion cycle caused transaction rollback
            at bitronix.tm.BitronixTransaction.commit(BitronixTransaction.java:241)
            at bitronix.tm.BitronixTransactionManager.commit(BitronixTransactionManager.java:143)
            at org.drools.persistence.jta.JtaTransactionManager.commit(JtaTransactionManager.java:226)
            ... 9 more
          Caused by: javax.persistence.PersistenceException: org.hibernate.exception.ConstraintViolationException: could not execute statement
            at org.hibernate.ejb.AbstractEntityManagerImpl.convert(AbstractEntityManagerImpl.java:1387)
            at org.hibernate.ejb.AbstractEntityManagerImpl.convert(AbstractEntityManagerImpl.java:1310)
            at org.hibernate.ejb.AbstractEntityManagerImpl.convert(AbstractEntityManagerImpl.java:1316)
            at org.hibernate.ejb.AbstractEntityManagerImpl$CallbackExceptionMapperImpl.mapManagedFlushFailure(AbstractEntityManagerImpl.java:1510)
            at org.hibernate.engine.transaction.synchronization.internal.SynchronizationCallbackCoordinatorNonTrackingImpl.beforeCompletion(SynchronizationCallbackCoordinatorNonTrackingImpl.java:114)
            at org.hibernate.engine.transaction.synchronization.internal.RegisteredSynchronization.beforeCompletion(RegisteredSynchronization.java:53)
            at bitronix.tm.BitronixTransaction.fireBeforeCompletionEvent(BitronixTransaction.java:532)
            at bitronix.tm.BitronixTransaction.commit(BitronixTransaction.java:235)
            ... 11 more
          Caused by: org.hibernate.exception.ConstraintViolationException: could not execute statement
            at org.hibernate.exception.internal.SQLStateConversionDelegate.convert(SQLStateConversionDelegate.java:129)
            at org.hibernate.exception.internal.StandardSQLExceptionConverter.convert(StandardSQLExceptionConverter.java:49)
            at org.hibernate.engine.jdbc.spi.SqlExceptionHelper.convert(SqlExceptionHelper.java:124)
            at org.hibernate.engine.jdbc.spi.SqlExceptionHelper.convert(SqlExceptionHelper.java:109)
            at org.hibernate.engine.jdbc.internal.ResultSetReturnImpl.executeUpdate(ResultSetReturnImpl.java:189)
            at org.hibernate.engine.jdbc.batch.internal.NonBatchingBatch.addToBatch(NonBatchingBatch.java:59)
            at org.hibernate.persister.entity.AbstractEntityPersister.insert(AbstractEntityPersister.java:3079)
            at org.hibernate.persister.entity.AbstractEntityPersister.insert(AbstractEntityPersister.java:3521)
            at org.hibernate.action.internal.EntityInsertAction.execute(EntityInsertAction.java:88)
            at org.hibernate.engine.spi.ActionQueue.execute(ActionQueue.java:395)
            at org.hibernate.engine.spi.ActionQueue.executeActions(ActionQueue.java:387)
            at org.hibernate.engine.spi.ActionQueue.executeActions(ActionQueue.java:303)
            at org.hibernate.event.internal.AbstractFlushingEventListener.performExecutions(AbstractFlushingEventListener.java:349)
            at org.hibernate.event.internal.DefaultFlushEventListener.onFlush(DefaultFlushEventListener.java:56)
            at org.hibernate.internal.SessionImpl.flush(SessionImpl.java:1159)
            at org.hibernate.internal.SessionImpl.managedFlush(SessionImpl.java:404)
            at org.hibernate.engine.transaction.synchronization.internal.SynchronizationCallbackCoordinatorNonTrackingImpl.beforeCompletion(SynchronizationCallbackCoordinatorNonTrackingImpl.java:109)
            ... 14 more
          Caused by: org.postgresql.util.PSQLException: ERROR: duplicate key value violates unique constraint "organizationalentity_pkey"
            Detail: Key (id)=(sales) already exists.
            at org.postgresql.core.v3.QueryExecutorImpl.receiveErrorResponse(QueryExecutorImpl.java:2161)
            at org.postgresql.core.v3.QueryExecutorImpl.processResults(QueryExecutorImpl.java:1890)
            at org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:255)
            at org.postgresql.jdbc2.AbstractJdbc2Statement.execute(AbstractJdbc2Statement.java:560)
            at org.postgresql.jdbc2.AbstractJdbc2Statement.executeWithFlags(AbstractJdbc2Statement.java:417)
            at org.postgresql.jdbc2.AbstractJdbc2Statement.executeUpdate(AbstractJdbc2Statement.java:363)
            at sun.reflect.GeneratedMethodAccessor32.invoke(Unknown Source)
            at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
            at java.lang.reflect.Method.invoke(Method.java:606)
            at bitronix.tm.resource.jdbc.BaseProxyHandlerClass.invoke(BaseProxyHandlerClass.java:64)
            at com.sun.proxy.$Proxy35.executeUpdate(Unknown Source)
            at org.hibernate.engine.jdbc.internal.ResultSetReturnImpl.executeUpdate(ResultSetReturnImpl.java:186)
            ... 26 more
          
          

           

          I wonder how this was designed to work with persistance. Obviously the same userGroupCallback is inserted twice.

          I don't know how you solved this under the hood, but seems that two threads are coming concurrently to the line where you do find/insert.

          Both queries are returning null for the string sales, and they are simultaneously trying to insert something with the same pk into the same table.

          • 2. Re: Concurrent usage of TaskService
            swiderski.maciej

            when it comes to first issue with taskimpl encountering optimistic lock exception (Stale object state) this is expected with your reproducer and here is why:

            • you run concurrently same process definitions with same set of data
            • then you run in another threads task operations where most important is getTasksAssignedAsPotentialOwner
            • that query will return all tasks for given user which means all tasks regardless which thread created them
            • then each task will get first task from the list which will always be the same one and thus you try to operate on exact same tasks from multiple threads

            That MUST throw optimistic lock exception and thus it works as designed. See attached reproducer fixed to make sure threads operate only on their tasks (created process instances tasks) and thus running smoothly

             

            Second issue with constraint violation exception is again optimistic lock exception as you try to put same data into user/group table at the same time. You can mitigate this by pre populating db with known user and groups before you execute the processes. This is considered as limitation and will remain as such because dealing with table locks to avoid this optimistic lock exception is way more difficult and causes huge performance degradation compared to the value gained from it.

             

            HTH

            • 3. Re: Concurrent usage of TaskService
              akoskm

              You're right with the first exception. Thanks!

               

              Regarding the second exception, in an environment where the user-group relations can be changed, you have to initialize those relations right before you run any processes.

              Now if your timeframe is really tight, you'll end up with the same relations for different threads, each of them trying to insert the same values.

               

              Currently the only solution I see is to synchronize the ksession.startApplication, to prevent multiple threads from inserting the same values.

               

              What do you think?

               

              Thanks,

              Akos