great analysis and very good catch. This reminds me of my issue back in 2010 where I analyzed the synchronization in ENC factory is suboptimal. Could you have a look at the issue https://issues.jboss.org/browse/JBPAPP-4787 whether it is what you are seeing too?
As you see, these problems are not really fixable and havent been completely fixed.
Lets get to solution then. Really, solving this with "cluster" as in fully replicated states and etc does not make much sense. Clusters provide high availability but since you would be running all instances in the same machine, the failure of one component or network or whatnot of that particular machine makes your entire "cluster" unavailable. Therefore, I propose a solution to use multiple standalone servers (well, depends on your app if thats doable). You can either do this via different port binding or create new virtual network interfaces and bind servers to them.
Yes, this is exactly what I would like to try - create two standalone jbosses and loadbalance them using Apache, so the visitors traffic is divided between two of them.
Regarding your issues, I think it's same as mine:
Thread: ajp-0.0.0.0-8009-813 : priority:5, demon:true, threadId:1431, threadState:RUNNABLE
- locked <0x7406ce55> (a java.util.WeakHashMap)
and other threads waiting for weakhashmap. BTW, I am using JBoss 5.1 standalone, not EAP.
I suppose this solution will help (for a while) I have had a similar configuration using EJB.
For this we setup multiple instances on one power system. It will solve the 'sync' stuff but if the JBoss app scales you might bring the DB in behind under heavy load .
Will say as more performance you have as more you will need