Discussions related to http://jira.jboss.com/jira/browse/JBCACHE-311 following more IM chats between Manik and myself.
JBCACHE-311 proposes creating an abstraction separating JGroups from JBossCache. This was scheduled for 2.1, although some or all of the work may make sense for 2.0.
Some initial thoughts:
1) The JIRA mentions moving the JChannel stuff to the ReplicationInterceptor. However, state transfer in all its many flavors also needs to use the channel. Manik and I have discussed this and passing the state transfer related calls through the interceptors doesn't really fit well (e.g. replication is a push operation; state transfer is a pull). However, we'd like to simplify the TreeCache implementation class by removing the callRemoteMethods() stuff.
We were discussing creating an "RpcManager" to provide the RPC call functionality for both ReplicationInterceptor and state transfer. The TreeCache impl would be responsible for creating the RpcManager and making a ref to it available to interceptors and the state transfer code.
This RpcManager (probably renamed) is a logical portion of the JBCACHE-311 abstraction.
If we create RpcManager, we shouldn't use JGroups' MethodCall in its API; instead use JBC's own abstraction of a MethodCall.
2) Multiplexer integration is going to involve changes to the way the JChannel is configured/managed. It makes sense to consider JBCACHE-311 when we design the multiplexer integration. E.g. encapsulate all transport related config in an element that can be passed to a user specified "TransportManager" class (not really trying to think of good names here). The JGroupsTransportManager knows how to parse the config to either use the multiplexer or create a JChannel the old fashioned way.
Presumably this "TransportManager" and the previously mentioned "RpcManager" would be the same thing; need to sort out names.
3) Finally, there is the JGroups MembershipListener side of things. This includes abstracting away the MembershipListener interface, as well as direct use of Address/IpAddress.
A major concern is doing this for 2.0 is a lot of work, but OTOH when we create the multiplexer integration seems like the logical time to do it.
1) What is the benefit of an RpcManager versus having this functionality inside of RpcBaseInterceptor (superclass of ReplicationInterceptor) ?
A separate RpcManager would only be necessary if someone else than a replication interceptor needs to make remote cluster calls. Is this the case ?
2) I think with the Multiplexer, we could 'inject' (or set programmatically in standalone mode) an instance of the Multiplexer, so all cluster calls could be made through the Multiplexer. As you mentioned, this could of course again be that RpcManager interface.
Since all of these points affect interfaces, we *need* to have this done by 2.0, so I guess we will have to reshuffle some priorities...
The various varieties of state transfer make RPC calls:
1) plain vanilla initial state transfer (something of a special case as it involves an operation on the JChannel rather than the RpcDispatcher).
2) partial state transfer when regions are activated (e.g. webapp deploy)
3) complete or partial state transfer when a cache is notified it is someone's buddy (complete or partial depends on whether region-based marshalling is in effect; if region-based is used, a series of partial state transfers is done).
State transfer should IMO be handled by the ReplicationInterceptor anyway ! Replication and transfer is the same aspect
The thing that made a strange fit to me is that replication is a push of data, state transfer is a pull. So, I haven't seen the common function between the two that the replication interceptor is going to perform, besides knowing how to invoke callRemoteMethods().
I'll keep thinking about it.
I think the state transfer issue will be much clearer if all (JGroups based) state transfers involve FLUSH and JGroups partial state transfer. Right now the way remote calls are done for the different state transfer use cases is quite different (with common code used for actually preparing the state and integrating the state that comes through). If we can get rid of using an RpcDispatcher call for partial state transfer and instead use the channel's partial state transfer API that may help.
Since we need to provide a consolidated view of RPC call outside of JBossCache (inside of AS CLustering) anyway, if we have RPCManager, maybe this portion can be made public as well. Then, we don't need to maintain a separate HAPartition for the RemoteMethodCall, e.g.
Of course, whether state caching and rpc should mix or not is an philosophical issue.
Negative on making RpcManager public; we should not mix replicated state (JBossCache) and remote cluster calls
I ahree with Bela. The RpcManager should not be public.
JBossCache RPC calls will be optimised for JBossCache specific usage - with known method ids and magic numbers of internal JBC classes. Not optimised for generic RPC usage.
I agree that replication and transfer are both aspects that are very close together (and even the group membership messages I need to broadcast in Buddy Replication) - the reason why I don't see this a good fit for the BaseRPCInterceptor is this:
The interceptor stack is designed to apply aspects to method calls on the cache, such as gets and puts.
For non-cache operations - such as broadcastBuddyGroup() and state transfers - this is raw RPC and not an aspect on any cache-specific method. Implementing this in the BaseRPCInterceptor will look an awful lot like a kludge - all other interceptors will have to ignore this method call, the only one that responds to it is the BaseRPCInterceptor that performs the RPC.
My point is, that in the case of state transfers and broadcasting buddy groups, the RPC is *not* an aspect anymore but core purpose.
In replicating a get(), the RPC is just an aspect.
And this is why an RPCManager makes sense - for core RPC features, use the RPCManager directly. If your cache call requires RPC as an aspect, this comes through the BaseRpcInterceptor.
So we then end up with 3 'consumers' of the RPC service:
1. The BaseRpcInterceptor, which applies RPC as an aspect on cache calls
2. StateTransferManager, which uses RPC to move state
3. BuddyManager, which uses RPC to maintain and manage buddy groups.