[Cluster-UDP]: Cluster is not stable with a massive load

[developer80]

Hello,

I am running 2 exoplatform tomcat nodes with cluster using the default UDP mode and use a jmeter script to validate the login performance.

The cluster is running good with smaller load but when the load is increased the cluster become unstable and i got a TimeoutException of jgroups , below the stack-trace:

at java.lang.Thread.run(Thread.java:745) [na:1.7.0_67]

Caused by: org.jgroups.TimeoutException: timeout sending message to exo-node1-1821

	at org.jgroups.blocks.MessageDispatcher.sendMessage(MessageDispatcher.java:390) ~[jgroups-3.2.7.Final.jar:3.2.7.Final]
	at org.infinispan.remoting.transport.jgroups.CommandAwareRpcDispatcher.processSingleCall(CommandAwareRpcDispatcher.java:301) ~[infinispan-core-5.2.9.Final.jar:5.2.9.Final]
	at org.infinispan.remoting.transport.jgroups.CommandAwareRpcDispatcher.invokeRemoteCommand(CommandAwareRpcDispatcher.java:179) ~[infinispan-core-5.2.9.Final.jar:5.2.9.Final]
	... 91 common frames omitted

2015-12-15 15:20:23,720 | WARN  | ARJUNA016029: SynchronizationImple.afterCompletion - failed for SynchronizationAdapter{localTransaction=LocalTransaction{remoteLockedNodes=[exo-node2-3705, exo-node1-1821], isMarkedForRollback=false, lockedKeys=null, backupKeyLocks=[ChangesKey-a602dadbc0a802ec504f633177cf6b64-null], topologyId=2, isFromStateTransfer=false} org.infinispan.transaction.synchronization.SyncLocalTransaction@10f3f0} org.infinispan.transaction.synchronization.SynchronizationAdapter@10f40f with exception [com.arjuna.ats.jta<Forum-Thread>]

org.infinispan.CacheException: Could not commit.

	at org.infinispan.transaction.synchronization.SynchronizationAdapter.afterCompletion(SynchronizationAdapter.java:83) ~[infinispan-core-5.2.9.Final.jar:5.2.9.Final]
	at com.arjuna.ats.internal.jta.resources.arjunacore.SynchronizationImple.afterCompletion(SynchronizationImple.java:96) ~[jbossjta-4.16.6.Final.jar:na]
	at com.arjuna.ats.arjuna.coordinator.TwoPhaseCoordinator.afterCompletion(TwoPhaseCoordinator.java:402) ~[jbossjta-4.16.6.Final.jar:na]
	at com.arjuna.ats.arjuna.coordinator.TwoPhaseCoordinator.end(TwoPhaseCoordinator.java:103) ~[jbossjta-4.16.6.Final.jar:na]
	at com.arjuna.ats.arjuna.AtomicAction.commit(AtomicAction.java:164) ~[jbossjta-4.16.6.Final.jar:na]
	at com.arjuna.ats.internal.jta.transaction.arjunacore.TransactionImple.commitAndDisassociate(TransactionImple.java:1165) [jbossjta-4.16.6.Final.jar:na]
	at com.arjuna.ats.internal.jta.transaction.arjunacore.BaseTransaction.commit(BaseTransaction.java:117) [jbossjta-4.16.6.Final.jar:na]
	at org.infinispan.CacheImpl.executeCommandAndCommitIfNeeded(CacheImpl.java:1196) [infinispan-core-5.2.9.Final.jar:5.2.9.Final]
	at org.infinispan.CacheImpl.putInternal(CacheImpl.java:783) [infinispan-core-5.2.9.Final.jar:5.2.9.Final]
	at org.infinispan.CacheImpl.put(CacheImpl.java:777) [infinispan-core-5.2.9.Final.jar:5.2.9.Final]
	at org.infinispan.DecoratedCache.put(DecoratedCache.java:320) [infinispan-core-5.2.9.Final.jar:5.2.9.Final]
	at org.exoplatform.services.jcr.impl.core.query.ispn.ISPNIndexChangesFilter.doUpdateIndex(ISPNIndexChangesFilter.java:172) [exo.jcr.component.core-1.16.4-GA.jar:1.16.4-GA]
	at org.exoplatform.services.jcr.impl.core.query.IndexerChangesFilter.doUpdateIndex(IndexerChangesFilter.java:231) [exo.jcr.component.core-1.16.4-GA.jar:1.16.4-GA]
	at org.exoplatform.services.jcr.impl.core.query.IndexerChangesFilter.onSaveItems(IndexerChangesFilter.java:160) [exo.jcr.component.core-1.16.4-GA.jar:1.16.4-GA]
	at org.exoplatform.services.jcr.impl.core.query.SearchManager.onSaveItems(SearchManager.java:640) [exo.jcr.component.core-1.16.4-GA.jar:1.16.4-GA]
	at org.exoplatform.services.jcr.impl.dataflow.persistent.WorkspacePersistentDataManager.notifySaveItems(WorkspacePersistentDataManager.java:1300) [exo.jcr.component.core-1.16.4-GA.jar:1.16.

The infinispan version is : 5.2.9.Final

The jgroups version is : 3.2.7.Final

In attachment the jgroups configuration file used in our application.

Is there any recommended config to use in order to fix this problem?

Regards