As we proceed to the design of Remoting 3.0, I'd like to contribute a few thoughts on the subject, based on Ovidiu Feodorov's wiki page JBossMessagingRemotingAPIExtensions, Tom Elrod's wiki page JBossMessagingRemotingAPIExtensions_Deux, various discussions, and my experience as a Remoting "user" in the context of working with the JBossMessaging team. 

I. Current architecture.

In its current incarnation, Remoting supports an rpc invocation model with

• selectable transports

• pluggable marshalling

• round trip and oneway invocations

• push, pull, and "simulated push" asynchronous callbacks

• connection monitoring

Many of these features (probably all of them, though some of them predate my experience with Remoting) have been created in response to some explicit demand, and I think they should be preserved.  JBossMessaging, for example, uses almost all of them.  But there seems to be a lot of sentiment in favor of evolving beyond the invocation model.

Remoting meets the application world in four primary places:

• org.jboss.remoting.Client

• org.jboss.remoting.transport.Connector

• org.jboss.remoting.ServerInvocationHandler

• org.jboss.remoting.marshal.Marshaller/org.jboss.remoting.marshal.UnMarshaller

The Client class wraps, configures, and adds extra facilities to the client side of the transport level invoker and presents the client side face of Remoting to the application.  The Connector class wraps and configures the server side of the transport level invoker, but plays a lesser role than the Client class.  In fact, it almost disappears in the case of declarative server side configuration. The real server side interface between Remoting and the application is the ServerInvocationHandler, implementations of which process each invocation. The Marshaller/UnMarshaller interfaces provide a configurable layer between the transport layer and the network.  A distributed Remoting application has the following layered structure:

    Client side                    Server side
    -----------                    -----------
    application
    Client                         Connector
    transport (client invoker)     transport (server invoker)
                                   application (invocation handler)
    marshalling/unmarshalling      marshalling/unmarshalling

Note that the structure is not quite symmetric. It's appropriate for the rpc invocation model, but I think that pursuing greater symmetry is the key architectural concept for the next phase of the Remoting project.

II. Bidirectional connections.

One source of asymmetry is the treatment of callbacks.  A callback is conceptually similar to an invocation (with no return value), which is executed by a call to Client.invoke (or Client.invokeOneway()), but a callback is sent by executing org.jboss.remoting.callback.ServerInvokerCallbackHandler.handleCallback(), which (in some cases) wraps a call to Client.invoke() (or Client.invokeOneway()).  Now, the distinction is not entirely inappropriate, since an invocation and a callback, though conceptually similar, can be behaviorally very different, due to possible restrictions on the creation of connections from the server back to the client.  If there is no connection from the server to the client, then ServerInvokerCallbackHandler.handleCallback() doesn't execute Client.invoke(); it just stores the callback for later retrieval by the client. 

I believe, though, that Remoting callbacks could be simplified by hiding the behavioral differences behind a more highly functional Client.  That is, Client.invoke() (or Client.invokeOneway()) could be semantically responsible for sending a message to the other end of a connection, without specifying the implementation details of whether the message is sent by direct transmission, stored for later retrieval, etc.  So, callbacks could be sent by calling Client.invoke(), eliminating the conceptual duplication of ServerInvokerCallbackHandler.handleCallback().  Actually, this idea comes from JBossMessaging 1.0, where the ServerInocationHandler called ServerInvokerCallbackHandler.getCallbackClient() to extract the wrapped Client and then proceeded to call Client.invoke().  At first I thought this code was a strange way to use Remoting callbacks, but eventually I realized they were telling us something.

This explicit use of Client on both the client side and the server side is one step away from what I think of as a primary conceptual extension of the current form of Remoting.  Tim Fox asked for a channel model in Remoting, which I understand to mean a bidirectional pipe where bytes go in at one end and come out the other, possibly after being stored in the middle.  This pipe could present itself as a Client at either end, though for the sake of symmetry I would extend Client to a more neutrally named subclass Connection that looks something like

   public class Connection extends Client
   {
      public void send(Object payload) throws Exception {...}
      public Object receive() throws Exception {...}
      public void rawSend(Object payload) throws Exception {...}
      public Object rawReceive() throws Exception {...}
   }

where send(Object payload) wraps payload in an InvocationRequest and rawSend(Object payload) does not, and where receive() returns the response wrapped in an InvocationResponse and rawReceive() does not.  Obviously, the intention is that the bytes transmitted by Connection.send() on one end are returned by Connection.receive() at the other end, and this relationship is symmetric.  Remoting, across all of its transports, would be responsible for implementing the semantics, which, in the case of server side to client side communication, might involve the current pull callback server side storage mechanism.

Currently, a callback connection is created when Client.addListener() is invoked on the client side, which leads to the invocation, on the server side, of ServerInvocationHandler.addListener(InvokerCallbackHandler callbackHandler), which passes a ServerInvokerCallbackHandler to the application code, giving the application code access to the server side of the connection.  I would promote the server side connection endpoint to first class status by turning Connector into a Connection repository.  The server side of a client/server connection is currently represented by an InvokerLocator, which allows a client anywhere on the network to access the referenced server.  I think the client side endpoint should also have an identity, represented by an InvokerLocator, though the port component would be optional, possibly replaced by application specific parameters.  So Connection would have a constructor like

   public class Connection extends Client
   {
      ...
      public Connection(InvokerLocator clientLocator, InvokerLocator serverLocator);
      ...
   }

and Connector would be extended with query methods that allow server side application code to connect to an arbitrary client known to the Connector, methods like

   public class Connector
   {
      ...
      InvokerLocator[] getConnections() throws Exception;
      InvokerLocator getConnecttion(InvokerLocator clientLocator) throws Exception;
      ...
   }

Suppose a Connection is created by

   Connection connection = new Connection("socket://blueMonkey.diamond.com?wile.E.Client", "socket://acmeServer:5555");

That is, the client side is identified by "socket://blueMonkey.diamond.com?wile.E.Client" and the server side is identified by "socket://acmeServer:5555". Then any server side application code with access to the Connector could send an asynchronous callback by doing something like

   Connector connector = new Connector("socket://acmeServer:5555");
   connector.start();
   ...
   Connection connection = connector.getConnection("socket://blueMonkey.diamond.com?wile.E.Client");
   connection.send(new CallbackMessage());

These changes allow application code to live outside of the ServerInvocationHandler, and as a result, a distributed Remoting application has the symmetrical structure

    Client side                    Server side
    -----------                    -----------
    application                    application
    Connection                     Connection (supplied  by a Connector)
    transport                      transport
    marshalling/unmarshalling      marshalling/unmarshalling

III.  Peer to peer communications.

In fact, we could go a step further and extend Connection with query methods as well:

   public class Connection
   {
      ...
      InvokerLocator[] getConnections() throws Exception;
      InvokerLocator getConnecttion(InvokerLocator clientLocator) throws Exception;
      ...
   }

Now, suppose that the client at "socket://blueMonkey.diamond.com?wile.E.Client" has created a Connection to the server at "socket://acmeServer:5555":

   Connection connection = new Connection("socket://blueMonkey.diamond.com?wile.E.Client", "socket://acmeServer:5555");

Then any application code anywhere can do something like this:

   Connection connection = new Connection("socket://greenMonkey.diamond.com?Daff.E.Duck", "socket://acmeServer:5555");
   Connection peer = connection.getConnection("socket://blueMonkey.diamond.com?wile.E.Client");
   peer.send(new Message());

The client at "socket://greenMonkey.diamond.com?Daff.E.Duck" creates a Connection to the server at "socket://acmeServer:5555" and uses the Connection to query the server for a Connection to the other Client at "socket://blueMonkey.diamond.com?wile.E.Client". That is, peer to peer communication can be established, with a server acting as hub. Remoting would be responsible for supporting the communication from "socket://greenMonkey.diamond.com?Daff.E.Duck" to "socket://blueMonkey.diamond.com?wile.E.Client", depending on the nature of the Connections in place. For example, if messages are accumulating on "socket://acmeServer:5555" for later retrieval, then "socket://greenMonkey.diamond.com?Daff.E.Duck" could just contribute to the message store.  On the other hand, if a ServerSocket is open on "socket://blueMonkey.diamond.com?wile.E.Client", then a direct connection could be established. 

Note that I'm not aware of any requests for Remoting to support peer to peer communications, so I'm just throwing it out there as a fantasy feature.