Network failure protection with JGroups/JBossCache

[kapilanand]

hi
I am trying to use JBoss cache as a synchronously replicated cache inside jboss. There are other jboss/ejb applications running that rely on this replicated data.
I have couple of failure scenarios that I need to protect against.

1) first one is the simplest one: when the whole node goes down. This case is taken care of as other applications depending on the cache use only other nodes.

The other two scenarios are a bit complex:
2) If the nodes are running fine, but the connectivity between the JBossCache JGroup nodes is affected. That is other applications using the cache are not aware of this.

3) If the nodes are running fine, but only the multicast traffic amongst the JBossCache JGroup nodes is affected.

One possible solution to this problem is to invalidate all other cache replicas except one and then somehow route all application requests to that jboss instance. That would mean that if there are some EJBs that are clustered and they should be disabled on these invalidated cache nodes.

How can this be done?:
- Stateless session bean, since client stub participating in a different jgroup discovery might still see all the nodes alive and continue load-balancing. Is it possible to programmatically start/stop SLSB pool?
Or Stateless bean could check for cache invalidation and throw exception that hints the client to retry request, but it might still land up on the same node depending on the concurrency of requests and since stub is not aware of this exception.
- Message driven bean: Is it possible to programmatically start/stop MDB pools?
Or i guess (not sure) if MDB's could throw some exception in onMessage(), and the load-balancing logic in the external jms provider will try to route those messages to other MDBs.

Other possible solutions for high-availibilty through redundency:
(on top of the above)
Is it possible to configure a redundant network route for JGroups to fallback on?
Or is it possible to configure redundant multicast JGroups for one JBossCache and somehow bind these multicast address with different routes/nic?
Or is it possible to configure JGroups with two physical network interfaces, so it can decide to use either one of them.

thanks

[belaban]

// Is it possible to configure a redundant network route
// for JGroups to fallback on?

No, not yet. We might be able to implement this with LogicalAddresses, in one of the next releases, e.g. 2.6.

However, you might be able to achieve this with help from the OS, e.g. IP Multipathing on Solaris or IP Bonding on Linux, where you have a logical NIC, which is backed up by 2 physical IP NICs, so traffic automatically uses the second NIC if the first fails. Every NIC can point to a separate network.

// Or is it possible to configure redundant multicast
// JGroups for one JBossCache and somehow bind these
// multicast address with different routes/nic?


You can bind to multiple NICs for incoming traffic:
- receive_on_all_interfaces="true" or
- receive_interfaces="eth0,192.168.0.2,hme2"

You can also send your traffic out over multiple NICs:
- send_on_all_interfaces
- send_interfaces="eth0,eth2"

This option doesn't load balance, but copies the *entire* traffic on every interface !

// Or is it possible to configure JGroups with two
// physical network interfaces, so it can decide to use
// either one of them.

See above: use IP bonding, for example