Are you sure that you have the same multicast-address/port and cluster-partition name for both installations configured.
A problem might that a switch or other network-component drop the multicast packages!
A simple test can be to use two original JBoss on both systems started with run.sh -b<the IP> -c all.
In this case they should bild a cluster (see number of members within the logfile)
A good starting point will be
Here you will also find a simple test tool to check that JGroups communication will work.