schdule in cluster is not singleton

[sshetty]

Hi,

We have an couple of jboss servers in a cluster and have the esb deployed on each server of the cluster. The esb has multiple schedules configured. We noticed the same schedule is triggered on every server in the cluster - we expected a schedule to be triggered only on a single server at a time, because they are clustered.

How can we change the behaviour so that a schedule is trigerred only on a single server at a time ?

Also when the schedule is triggered at the same instance on multiple servers, we see the following exception occassionaly. When the schedule is triggered, it triggers a BPM which in turn invokes different services on the esb.

15:47:39,108 INFO [STDOUT] ==========> entering node: Node(Convert to XML)
15:47:39,246 ERROR [AbstractFlushingEventListener] Could not synchronize database state with session
org.hibernate.StaleObjectStateException: Row was updated or deleted by another transaction (or unsaved-value mapping was incorrect): [org.jbpm.job.ExecuteActionJob#3459]
 at org.hibernate.persister.entity.AbstractEntityPersister.check(AbstractEntityPersister.java:1765)
 at org.hibernate.persister.entity.AbstractEntityPersister.update(AbstractEntityPersister.java:2407)
 at org.hibernate.persister.entity.AbstractEntityPersister.updateOrInsert(AbstractEntityPersister.java:2307)
 at org.hibernate.persister.entity.AbstractEntityPersister.update(AbstractEntityPersister.java:2607)
 at org.hibernate.action.EntityUpdateAction.execute(EntityUpdateAction.java:92)
 at org.hibernate.engine.ActionQueue.execute(ActionQueue.java:250)
 at org.hibernate.engine.ActionQueue.executeActions(ActionQueue.java:234)
 at org.hibernate.engine.ActionQueue.executeActions(ActionQueue.java:142)
 at org.hibernate.event.def.AbstractFlushingEventListener.performExecutions(AbstractFlushingEventListener.java:298)
 at org.hibernate.event.def.DefaultFlushEventListener.onFlush(DefaultFlushEventListener.java:27)
 at org.hibernate.impl.SessionImpl.flush(SessionImpl.java:1000)
 at org.hibernate.impl.SessionImpl.managedFlush(SessionImpl.java:338)
 at org.hibernate.transaction.CacheSynchronization.beforeCompletion(CacheSynchronization.java:59)
 at com.arjuna.ats.internal.jta.resources.arjunacore.SynchronizationImple.beforeCompletion(SynchronizationImple.java:114)
 at com.arjuna.ats.arjuna.coordinator.TwoPhaseCoordinator.beforeCompletion(TwoPhaseCoordinator.java:247)
 at com.arjuna.ats.arjuna.coordinator.TwoPhaseCoordinator.end(TwoPhaseCoordinator.java:86)
 at com.arjuna.ats.arjuna.AtomicAction.commit(AtomicAction.java:177)
 at com.arjuna.ats.internal.jta.transaction.arjunacore.TransactionImple.commitAndDisassociate(TransactionImple.java:1389)
 at com.arjuna.ats.internal.jta.transaction.arjunacore.BaseTransaction.commit(BaseTransaction.java:135)
 at com.arjuna.ats.jbossatx.BaseTransactionManagerDelegate.commit(BaseTransactionManagerDelegate.java:87)
 at org.jboss.tm.usertx.client.ServerVMClientUserTransaction.commit(ServerVMClientUserTransaction.java:140)
 at org.jbpm.persistence.jta.JtaDbPersistenceService.endJtaTransaction(JtaDbPersistenceService.java:76)
 at org.jbpm.persistence.jta.JtaDbPersistenceService.close(JtaDbPersistenceService.java:37)
 at org.jbpm.svc.Services.close(Services.java:225)
 at org.jbpm.JbpmContext.close(JbpmContext.java:139)
 at org.jbpm.job.executor.JobExecutorThread.acquireJobs(JobExecutorThread.java:144)
 at org.jbpm.job.executor.JobExecutorThread.run(JobExecutorThread.java:58)
15:47:39,247 WARN [arjLoggerI18N] [com.arjuna.ats.arjuna.coordinator.TwoPhaseCoordinator_2] TwoPhaseCoordinator.beforeCompletion - failed for com.arjuna.ats.internal.jta.resources.arjunacore.SynchronizationImple@11b5a04
org.hibernate.StaleObjectStateException: Row was updated or deleted by another transaction (or unsaved-value mapping was incorrect): [org.jbpm.job.ExecuteActionJob#3459]
 at org.hibernate.persister.entity.AbstractEntityPersister.check(AbstractEntityPersister.java:1765)
 at org.hibernate.persister.entity.AbstractEntityPersister.update(AbstractEntityPersister.java:2407)
 at org.hibernate.persister.entity.AbstractEntityPersister.updateOrInsert(AbstractEntityPersister.java:2307)
 at org.hibernate.persister.entity.AbstractEntityPersister.update(AbstractEntityPersister.java:2607)
 at org.hibernate.action.EntityUpdateAction.execute(EntityUpdateAction.java:92)
 at org.hibernate.engine.ActionQueue.execute(ActionQueue.java:250)
 at org.hibernate.engine.ActionQueue.executeActions(ActionQueue.java:234)
 at org.hibernate.engine.ActionQueue.executeActions(ActionQueue.java:142)
 at org.hibernate.event.def.AbstractFlushingEventListener.performExecutions(AbstractFlushingEventListener.java:298)
 at org.hibernate.event.def.DefaultFlushEventListener.onFlush(DefaultFlushEventListener.java:27)
 at org.hibernate.impl.SessionImpl.flush(SessionImpl.java:1000)
 at org.hibernate.impl.SessionImpl.managedFlush(SessionImpl.java:338)
 at org.hibernate.transaction.CacheSynchronization.beforeCompletion(CacheSynchronization.java:59)
 at com.arjuna.ats.internal.jta.resources.arjunacore.SynchronizationImple.beforeCompletion(SynchronizationImple.java:114)
 at com.arjuna.ats.arjuna.coordinator.TwoPhaseCoordinator.beforeCompletion(TwoPhaseCoordinator.java:247)
 at com.arjuna.ats.arjuna.coordinator.TwoPhaseCoordinator.end(TwoPhaseCoordinator.java:86)
 at com.arjuna.ats.arjuna.AtomicAction.commit(AtomicAction.java:177)
 at com.arjuna.ats.internal.jta.transaction.arjunacore.TransactionImple.commitAndDisassociate(TransactionImple.java:1389)
 at com.arjuna.ats.internal.jta.transaction.arjunacore.BaseTransaction.commit(BaseTransaction.java:135)
 at com.arjuna.ats.jbossatx.BaseTransactionManagerDelegate.commit(BaseTransactionManagerDelegate.java:87)
 at org.jboss.tm.usertx.client.ServerVMClientUserTransaction.commit(ServerVMClientUserTransaction.java:140)
 at org.jbpm.persistence.jta.JtaDbPersistenceService.endJtaTransaction(JtaDbPersistenceService.java:76)
 at org.jbpm.persistence.jta.JtaDbPersistenceService.close(JtaDbPersistenceService.java:37)
 at org.jbpm.svc.Services.close(Services.java:225)
 at org.jbpm.JbpmContext.close(JbpmContext.java:139)
 at org.jbpm.job.executor.JobExecutorThread.acquireJobs(JobExecutorThread.java:144)
 at org.jbpm.job.executor.JobExecutorThread.run(JobExecutorThread.java:58)
15:47:39,249 ERROR [Services] problem closing service 'persistence'
org.jbpm.JbpmException: couldn't commit JTA transaction
 at org.jbpm.persistence.jta.JtaDbPersistenceService.endJtaTransaction(JtaDbPersistenceService.java:78)
 at org.jbpm.persistence.jta.JtaDbPersistenceService.close(JtaDbPersistenceService.java:37)

Thanks

:)
SShetty

[massios]

Hello S Shetty,

I have seen two schedulers in the esb

1. the esb scheduler that is used in the jboss-esb.xml. This one works with the RAMstore of quartz and I do not think it can be clustered

    In our case this actually helped because we wanted our servers to do some xml file synchronization for each node.

2. the jbpm scheduler that comes with jbpm. This one can be clustered. I think you have to set up

   a) first a common jbpm data base

   b) cluster the jbpm

   c) modify inside the jbpm.esb a flile called quartz.properties and set clustered = true for quartz.

Nikos

[kconner]

1 - correct, this is local to the instance as it drives the local service deployments

2 - An alternative would be to use the JMS/JCA inflow replacement which is provided within the ESB

One issue with the standard jBPM JobExecutor is that the same job can be loaded on multiple nodes, leading to the StaleObjectStateException seen above.  By using the JMS delivery, especially with a clustered queue, this particular conflict should be removed.

There are other causes of this exception however.

Kev