Remoting API Extensions

This is how an ideal Remoting API would look like, from a Messaging perspective. I will walk through the proposed API changes by following a call from client to server.

The messaging client run-time creates a Remoting Client, by specifying the server's locator:

      Client client = new Client("bidirectional://jmsserverbox:7777", "EXAMPLE-SUBSYSTEM");
      client.connect()

Transports

For the purpose of this discussion, I will concentrate on only two transports:

A "bidirectional" transport

This is a new TCP socket based transport that we are proposing to be implemented. I purposely don't refer to it as "multiplex" to avoid any confusion with the current existing "multiplex" transport, based on "virtual sockets". The new "bidirectional" transport must meet two requirements:

1. It supports "bi-directionality".

That means that once a client creates a physical TCP connection to the server (possibly going throug a client-side firewall that allows outbound connections), the Remoting runtime is smart enough to know how to use the same physical connection to get data coming from the opposite direction (server to client).

2. It supports "multiplexing".

That means that the Remoting runtime is smart enough to be able to use only one physical TCP connection to the server to send data submitted by different Client instances living in the same virtual machine and having been configured with the same server locator. The concept is somehow similar with AMQP's connection and virtual channels.

The usefulness of multiplexing virtual channels in top of a physical TCP connection is somehow open to discussions. Actually, Messaging doesn't care to much how Remoting does this. A solutions similar to the actual PooledInvoker where multiple physical connections is open to the server is totally acceptable, provided that at least one of those connections allows for bidirectionality. This is not an API issue, but an implementation/configuration one.

*Note*

Apparently, the currently available "multiplex" transport supports both requirements, but it known as not very performant, if compared with the unidirectional socket transport. The API should be agnostic to that, so in a first stage, we can start testing with multiplex, followed by a different implementation, in case the performance is not satisfactory.

*TO DO*

Understand what is causing the performance problems for "multiplex".

The "http" transport

This transport is already implemented. It is important that the API semantics is preserved while using it: "http" must provide for virtual bi-directionality at API level.

Client-side callback API

The Messaging runtime should be able to use the existing Client instance to receive "push" callbacks, by just simply registering CallbackHandlers with the client:

      client.addListener(new InvokerCallbackHandler() { ... });

*Note 1*

NO callback server is declared and started, and also NO callback server locator is passed as argument to the addListener() call. There is no need for one. Both "bidirectional" and "http" transports allow for server-to-client calls: the "bidirectional" transport uses the physical TCP connection and the "http" transport uses polling on client-side.

*TO DO*

Apparently, there is already a method withing Client class that provides this behavior (public void addListener(InvokerCallbackHandler callbackhandler, Map metadata) throws Throwable ), and which is not used by Messaging yet. Need to test it to make sure it provides the semantics we need.

*Node 2*

For the sake of intuitiveness, I would suggest either addListener(new CallbackListener()) or addHandler(new CallbackHandler()). However, this is really a minor issue.

The Client's sending API

The Client interface should allow for four types of calls:

1. RPCs (Request-Response)

     
      public Object invoke(Object param) throws Throwable

This is the regular RPC, where the invoking client thread blocks until a reply or an exception arrives from the server. No changes in API here. Messaging uses this model extensively, and it will continue to use it. The Remoting runtime will continue to use underneath the pluggable marshaller mechanism and pluggable serialization, so all application that were relying on those will continue to work.

2. Synchronous raw send

   
      public byte[] send(byte[] rawBytes) throws Throwable

The method send raw bytes and blocks waiting for a reply. The method will be used by Messaging for synchronous high-throughput operations, such as sending messages from client to server. The method needs to be synchronous, because the client runtime needs an acknowledgment from the server that the message has been accepted for delivery. The method sends raw bytes, to avoid wasting time with unnecessary marshalling/unmarshalling, since both client and server know exactly what they send to each other. This will increase the overall throughput, as no unnecessary operations are performed.

*Note*

the same behavior can be obtained by sending a raw byte array using the already existing invoke(Object).

 

These are Tom Elrod's findings:

 

"I used jprofiler to see how much time was being spent in regards to marshalling when making 1000 invocations in tight loop (using socket invoker).  Not talking about the actual serialization, but the overhead of going to a marshaller instead of straight to the socket and writing. Total time for all marshalling hot spots is 2048 milliseconds which is 0.06% of the overall time.  So would roughly see a 0.06% gain in performance by exposing a invoke method that takes a byte{FOOTNOTE DEF  } and then writes directly to socket verses using a marshaller that does the same thing."

 

According to this conclusion, it is likely that a separate "raw" method that circumvents the marshaller does not make too much sense. 

3. Asynchronous invocations (no response)

Note that this extension is optional. I am not convinced this is very useful, it's here for symmetry.

      public void asynchInvoke(Object param) throws Throwable

This method call would put the invocation on the wire and the calling thread would return immediately, without blocking to wait for a reply. Similarly to the synchronous invocation, the Remoting runtime will use the pluggable marshaller mechanism and pluggable serialization, allowing for a great degree of flexibility in what is actually sent on the wire. Messaging will use this mechanism for acknowledgeBatch() type of invocations, which not need to be synchronous, but can benefit from the convenience of the RPC style (method calls into a remote interface).

Apparently, the behavior is already provided by invokeOneway().

4. Asynchronous raw send

      public void asynchSend(byte[] rawBytes) throws Throwable

This method call would put the bytes on the wire and the calling thread would return immediately, without blocking to wait for a reply. This method call will be used by Messaging for high-throughput asynchronous operations, for example individual acknowledgments. The method sends raw bytes, to avoid wasting time with unnecessary marshalling/unmarshalling, since both client and server know exactly what they send to each other. This will increase the overall throughput, as no unnecessary operations are performed.

ServerInvocationHandler API

Both synchronous and asynchronous invocations, as well as synchronously and asynchronously-sent byte arrays end up being delivered to the corresponding server-side ServerInvocationHandler implementation.

The ServerInvocationHandler interface must be amended to include the following three new methods, which will be added in top of already existing

      public Object invoke(InvocationRequest invocation) throws Throwable;

(to which all synchronous invocations are being delivered):

New methods:

      public byte[] handle(byte[] rawBytes) throws Throwable;

      public void asynchInvoke(InvocationRequest invocation);

      public void asynchHandle(byte[] rawBytes);

Server-Side Callback API

The server will use the existing callback mechanism to asynchronously send data back to the client. Currently, the callback mechanism only allows for synchronous callbacks. The API that defines the possibility of sending synchronous callbacks is, both on the client and the server side:

public interface InvokerCallbackHandler
{
     public void handleCallback(Callback callback) throws HandleCallbackException;
}

This allows sending fully-formed synchronous callbacks into the callback handler. The method will obviously stay, so all code that is using it will continue to work. The interface however needs to be amended to include:

      public void asynchHandle(byte[] rawBytes);

The reason for the presence of this method is to allow the server to send asynchronous high-throughput traffic into the client. Messaging will use it for receiving asynchronously messages for delivery on the client.  

A possible Remoting implementation could conceivably use NIO (nonblocking channels and readiness detection with selectors) underneath, so the reading/writing from a large number of sockets will be performed by a single thread (or a limited pool of threads), to avoid context switching inherent to a thread-per-connection approach. In this case, the Remoting implementation needs to make sure that asynchHandle() requests fielded by the server-side InvokerCallbackHandler implementation are queued to be processed by the same thread that detects readiness at selector level and performs the read/write operations.

So to summarize, this is how the client-side API should look like:

    Client client = new Client("bidirectional://jmsserverbox:7777", "EXAMPLE-SUBSYSTEM");
    client.connect()
    client.addHandler(new MessagingCallbackHandler());

    Object result = client.invoke(new MessagingInvocation());
    byte[] result = client.send(new byte[] { ... });
    client.asynchInvoke(new MessagingInvocation());
    client.asynchSend(new byte[] { ... });

The relevant fragment of the server-side ServerInvocationHandler interface is:

public interface ServerInvocationHandler
{
    ...

    public Object invoke(InvocationRequest invocation) throws Throwable;
    public byte[] handle(byte[] rawBytes) throws Throwable;
    public void asynchInvoke(InvocationRequest invocation);
    public void asynchHandle(byte[] rawBytes);
    
    ...
}

The relevant fragment of InvokerCallbackHandler is:

public interface InvokerCallbackHandler
{
    public void handleCallback(Callback callback) throws HandleCallbackException;
    public void asynchHandle(byte[] rawBytes);
}

NIO Implementation

The server-side implementation of the Remoting runtime that exposes the above API is currently based on a thread-per-connection model. The model is appropriate for usage patterns that involve a relatively low number of long lived clients. An alternate implementation based on NIO (nonblocking channels + readiness detection with selectors) should be provided. The above API does NOT preclude any of mentioned alternate implementation to be plugged in under it.

-

Referenced by: