I think the largest clusters I've seen (with 1.4.x) has been with close to 50 instances, but that''s using async replication.
If you want sync replication, and you can guarantee session affinity, I'd recommend buddy replication since this scales very well compared to total replication.
Hopefully in the near future we'll have a much more scalable replication mechanism http://wiki.jboss.org/wiki/Wiki.jsp?page=JBossCachePartitioning
We are currently running 4 nodes with synchronous replication of rather large data objects. (The replication is done within a transaction.)
The objects replicated are from 1k up to about 8k bytes large. These 4 nodes currently handle up to 5000 events/second which would translate into 1250 replications per second and node. (Note: I would really like to try PojoCache here for a more fine grained replication. )
We are constantly trying new jgroups stacks and so forth, but currently it seems like we get better performance with TCP. Buddy replication also helped alot (1 buddy per node).
We also run a cache shared by 8 nodes which is updated more then 5k/s, but that cache carries only small data and is updated asynchronously and is batched.
You will find that TCP is much faster when it comes to buddy replication, since replication is unicast to target buddies rather than the entire cluster.