JBossDeveloper

Do you (meaning the developers) have a QA environment with JBoss and Modeshape setup? I am curious as to why this isn't working for me. Apart from the fact that I am not using multicast (can't do it on Amazon EC2), I don't see what is preventing it from working. Maybe I'll try locally on a simple network first.

Nope, it's not multicast either, just a vanilla network with two machines won't work as-is with the configuration.

I breakpointed into the code and got the cluster configuration (as JSON) as well: https://gist.github.com/panghy/5114017. Nothing looks out of the ordinary. One thing I did notice is that modeshape, unlike JBoss EJB clustering and infinispan, does not "ask" the jgroups module for the actual channel but instead it asks for the ChannelFactory to be injected and the channel is constructed that way. I am not sure if there's any difference between doing both either way. Curiously, I can ask jgroups to create non-shared tranports for the jgroups configuration which then shows rejected messages on the "other" transports (essentially if there are 3 jgroup channels each using its own transport, they would complain seeing other's messages). This proves though that messages are being sent on the modeshape jgroups channel (since the rejection message is seen on the other 2 channels) but it never reaches the jgroup stack where it would pass off to the modeshape code. I have an inkling that it's somehow related to classloading and/or threading in a sense that the channel needs to be created by the jgroup module within it's own classloader and service thread and not with modeshape's. That' just a hunch, it would be somewhat difficult to prove unless I also look at how infinispan obtains its jgroups channel and mimmic it in modeshape (ala http://piotrnowicki.com/2013/02/using-jgroups-directly-from-jboss-as-7-component/).

Also, if one were to turn on cluster-name and cluster-stack in domain.xml with modeshape, JBoss complains about a missing dependency from modeshape to jgroups which requires adding a line to modeshape's module.xml with

<module name="org.jboss.as.clustering.jgroups"/>

I have found the actual problem within JBoss now, it's related to AS7-4415 and the JGroups channel that JBoss provides simply ignores any custome receivers.

https://github.com/jbossas/jboss-as/commit/ae32d2cc1a36839976cc270721cffb661315bef9

I guess it's now back to trying a newer version of JBoss and seeing if it works. I'll get a pull request in for the missing module in modeshape anyhow.

There should be a PSA: one CANNOT run modeshape 3.1.3.Final on JBoss 7.1.1.Final if you intend to run in a cluster. I have a patch for the 3.1.x branch that should make it work on 7.1.1.Final (not sure if the devs are releasing a 3.1.4 though since 3.2.x would target AS7.2).

I have also discovered that there is a classloader problem when running modeshape clustered in JBoss. The classloader that JGroups uses does not have the proper classes for deserialization. Looks like one would have to use a thread pool with the proper class loader set to handle the case when JGroups is supplied by the container.