6 Replies Latest reply on Oct 22, 2006 9:53 PM by ovidiu.feodorov

    Remoting - Multiplexed (Bidirectional) Transport

    ovidiu.feodorov

      We want to use the same physical TCP/IP connection to send data from client to server AND asynchronously, from server to client. Once this is available, the client won't need to create its physical callback server anymore. Callback servers cannot be used in firewalled environments, and even in common LAN environments they are liable to introduce complications.

      Tim wrote:

      Another issue I have is the remoting core abstraction being a unidirectional connection. This forces remoting to have to introduce the concept of callback servers to be able to handle communication from the server to the client.


      The "callback listener" model is a good abstraction to receive asynchronous notifications from the server, and it fits well with the whole remoting API. What I don't agree with is the need to have callback servers, as in pieces of code wrapped around server sockets on the client side. Once you have a TCP/IP connection from client to server, use that and don't open another one from server to client. Principle aside, there are also a lot of "practical" complications that arise when firewalls or funky host configurations come into play, and we had a good share of those reported by users.

      Tim wrote:

      The current multiplex transport if I remember correctly has some significant overhead as compared to the standard socket transport. It's my understanding that this is due to having to design this in terms of virtual sockets so it can work with the core remoting abstraction of a unidirectional connection.

      If remoting had used a bidirectional connection this would have been much simpler IMHO.

      With a birectional abstraction, multiplex connections are not a hard problem to solve. After all you just need to encapsulate each lump of bytes written in a packet with the id of the logical connection and the length of the packet and simply correlate them on receipt according to the id and send them off to logical connections. (Actually when we get to AMQP we _must_ implement multiplex this way - we have no options since it is a requirement of the specification)

      I can see how this would become more difficult to implement if you only have a unidirectional remoting abstraction to work with, and have to somehow unite the two remoting connections (one in one direction and one in another) so they actually use the same connection.


      I am sure this can be simplified, by leaving the current multiplex transport alone, and implementing a "light-multiplex" that doesn't try to offer anything like "virtual sockets", but it just carries bytes from client to server and vice-versa. I will start working with Ron or Tom on that.

      Tim wrote:

      The fact is, that we can't afford any overhead in the multiplex transport since we'll lose in benchmarks because of this.


      True. To me, this is one of the highest priority now.

      Tom Elrod wrote:

      There is currently a multiplex transport (as Ovidiu noted) and how it works under the hood is explained at http://labs.jboss.com/portal/jbossremoting/docs/multiplex/multiplex.html .
      Its performance is not great (see http://labs.jboss.com/portal/jbossremoting/docs/benchmark/performance_benchmark.html for comparison benchmarks). Ron can probably comment more on how much more performance we can expect to sqeeze out of it.


      Tim wrote:

      I suspect this would require changes to the remoting API - specifically in the area of bidirectional connections as I have discussed.


      Maybe not.

      Let's see what Ron has to say. Ron, you have the microphone.

        • 1. Re: Remoting - Multiplexed (Bidirectional) Transport

           


          What I don't agree with is the need to have callback servers, as in pieces of code wrapped around server sockets on the client side . Once you have a TCP/IP connection from client to server, use that and don't open another one from server to client.


          Although a callback server is needed on the client side, this does not necessarily mean a server socket is needed. If using the multiplex transport, it can use the same client socket to receive call back from the server (using virtual sockets). I have tested this through my firewall and know that the multiplex transport supports this. So think the functional requirement of multiplex is already met. Now would be a question of either performance not being good enough or programming model not fitting.

          I agree that we need Ron's input as to how much more performance gains can realistically be expected of the multiplex transport.









          • 2. Re: Remoting - Multiplexed (Bidirectional) Transport
            ron_sigal

            1. There is another solution to the firewall problem.


            What I don't agree with is the need to have callback servers, as in pieces of code wrapped around server sockets on the client side . Once you have a TCP/IP connection from client to server, use that and don't open another one from server to client


            Tom implemented a polling callback mechanism [I think it was requested by the Messaging group] which simulates the use of a callback server on the client side. I've used it on the Branch_HTTP_Experiment Messaging branch to avoid client side server sockets with the HTTP transport, but it would work with the socket transport as well. There's some latency according to the (configurable) polling frequency, but it also has the advantage of bundling up all pending callbacks in a single transmission.

            2. I'm not sure what a bidirectional channel would look like in Remoting, but there's a sense in which Remoting connections are close to a bidirectional channel. From client to server, there's the option of an invocation which is asynchronous at the application level, i.e., it returns immediately, and with the NIO transport it will be asynchronous at the transport level. From server to client, callbacks are not too conceptually different from asynchronous invocations. A payload is passed to Remoting and on the other side it lands on an application level handler. It may not be as elegant as a bidirectional channel, but I'd guess performance trumps elegance right now. And I'd say that Tim has done a pretty virtuoso job of working with the materials at hand.

            3. As for the performance of the current multiplex transport, the only honest answer I can give is that I don't know how much better it can be. I went through one round of optimization a few months ago, but in the absence of customers it wasn't clearly productive to keep squeezing. There is certainly overhead in lifecycle operations, but the real issue is pushing bytes on the network. I don't think the presence of virtual sockets is really a problem - there's a lot of code there that makes them look like real sockets but isn't in the line of fire most of the time. If there's room for improvement, I think it must be at the layer below virtual sockets. Right now a VirtualSocket.write() operation leaves some bytes on a queue, from which they are retrieved by another thread which writes them to the actual socket. The point was to allow the write() to return without waiting for the network i/o to finish. But in the optimization phase I switched to NIO sockets (except for SSL connections), so that issue is moot, and maybe there's some improvement to be gained by directly accessing the socket.

            It does seem to me that, in terms of i/o, virtual sockets shouldn't be significantly slower than real sockets. I'd be happy to have another go at it.

            • 3. Re: Remoting - Multiplexed (Bidirectional) Transport
              timfox

               

              "ron_sigal" wrote:

              2. I'm not sure what a bidirectional channel would look like in Remoting,


              I was thinking something vaguely similar to the Channel interface in Java NIO, except it would need to support both synchronous and asynchronous invocations.

              This could be the lowest level of the layered approach that Scott has suggested, and on top of that the rest of remoting could be layered.



              but there's a sense in which Remoting connections are close to a bidirectional channel. From client to server, there's the option of an invocation which is asynchronous at the application level,


              Does this mean that in the current implementation the invocation is synchronous at the transport level?

              Also, what does the current wire format of the multiplex transport look like? Is it based on frames? Or some other way of doing it?

              I'm thinking of how hard/easy it would be to get it to look similar(ish) to the AMQP wire format?

              • 4. Re: Remoting - Multiplexed (Bidirectional) Transport
                ovidiu.feodorov

                 

                Tom wrote:

                Although a callback server is needed on the client side, this does not necessarily mean a server socket is needed.


                Why would be a callback server exposed in API and required to be explicitely created on the client side?

                You only need to create a client for a specific server and then declare that you want to receive push callbacks from that server. All you need from an API point of view is to call

                client.setListener(CallbackListener)


                The server knows exactly what client added the callback listener, so if the connection happens to be over a bidirectional transport, the server knows how to get back to it. No need to expose the callback server in API in this case, it only adds extra complexity.

                For a unidirectional socket transport, yes, fine, but we want to improve the API, not stick to something that doesn't exactly fit our needs.

                Ron wrote:

                There is another solution to the firewall problem.
                Tom implemented a polling callback mechanism [I think it was requested by the Messaging group] which simulates the use of a callback server on the client side. I've used it on the Branch_HTTP_Experiment Messaging branch to avoid client side server sockets with the HTTP transport, but it would work with the socket transport as well.


                ... and it will be horribly slow.

                Why complicate things? Bidirectional TCP socket connection, it's all we need. And polling for HTTP, but there is no way around that there.

                Ron wrote:

                2. I'm not sure what a bidirectional channel would look like in Remoting, but there's a sense in which Remoting connections are close to a bidirectional channel. From client to server, there's the option of an invocation which is asynchronous at the application level, i.e., it returns immediately, and with the NIO transport it will be asynchronous at the transport level. From server to client, callbacks are not too conceptually different from asynchronous invocations. A payload is passed to Remoting and on the other side it lands on an application level handler. It may not be as elegant as a bidirectional channel,


                From an API perspective, client to server invocations (synchronous and asynchronous) and server to client callback (synchronous and asynchronous) is all we need. Remoting is very close to that, and I will soon publish a wiki document with suggested API amendments. But this is not the problem. The problem is that we don't need to mess around with virtual sockets to get bi-directionality. A TCP connection is inherently bi-directional. You write stuff at this end and it comes out at the other end. You write stuff at the other end and it comes out at this end. Remoting should take advantage of that in the simplest way possible. Write a different transport for that, call it "bidirectional" or "nebuchadnezzar", doesn't matter, and Messaging will use it.

                Tim wrote:

                Ron wrote:

                2. I'm not sure what a bidirectional channel would look like in Remoting,

                I was thinking something vaguely similar to the Channel interface in Java NIO, except it would need to support both synchronous and asynchronous invocations.


                I'll very soon publish some suggestions to amend the Remoting API. It only adds things to it, and keeps everything else pretty much as it is, so current application won't break. It'll be a wiki document, I'll link it here.

                Tim wrote:

                Also, what does the current wire format of the multiplex transport look like? Is it based on frames? Or some other way of doing it?


                I don't think we should even touch the current multiplex transport. It doesn't make sense to do that for AMQP. In AMQP we don't control the client, the concept of invocation is irrelevant, we just get a bunch of bytes on the wire, so we need to add a protocol handler on the server that "generates" invocations out of the protocol stream.



                • 5. Re: Remoting - Multiplexed (Bidirectional) Transport
                  timfox

                   

                  "ovidiu.feodorov@jboss.com" wrote:

                  From an API perspective, client to server invocations (synchronous and asynchronous) and server to client callback (synchronous and asynchronous) is all we need. Remoting is very close to that, and I will soon publish a wiki document with suggested API amendments. But this is not the problem. The problem is that we don't need to mess around with virtual sockets to get bi-directionality. A TCP connection is inherently bi-directional. You write stuff at this end and it comes out at the other end. You write stuff at the other end and it comes out at this end. Remoting should take advantage of that in the simplest way possible. Write a different transport for that, call it "bidirectional" or "nebuchadnezzar", doesn't matter, and Messaging will use it.


                  +1.

                  A bidirectional transport core abstraction is what I have been suggesting from the beginning.

                  It just make more sense, and prevent us having to jump though fiery hoops (callback servers, virtual sockets) to get a simple(ish) job done.


                  • 6. Re: Remoting - Multiplexed (Bidirectional) Transport
                    ovidiu.feodorov

                    This is the wiki document that contains proposed Remoting API extensions: http://wiki.jboss.org/wiki/Wiki.jsp?page=JBossMessagingRemotingAPIExtensions

                    Once materialized, it should isolate Messaging from future Remoting implementation changes.

                    Please comment.