14 Replies Latest reply on Oct 24, 2006 12:38 AM by ovidiu.feodorov

    Remoting - Client Asynchronous Calls

    ovidiu.feodorov

      Messaging clients should be able to send data on the wire without the unnecessarily added burden of waiting for a reply. Similarly, the server should be able to asynchronously send data to the client.

      Tom Elrod wrote:

      Have made the code changes (for jira issue http://jira.jboss.com/jira/browse/JBREM-548 ) so that if doing one-way invocations, only the request ever gets written to the wire and that thread on the client returns immediately after the write. On the server side, it never writes the response back to client.


      In what release is this available? I would like to play a little bit with it.

      Tom Elrod wrote:

      Probably good to have better definition of what asynchronous means. At a high level in the call stack, can just mean using a future model to make call and wait for a response to come back at later point in time (or just a callback on a different thread at later point in time). At lowest level, could be talking about only writing on the wire and not waiting for response off the wire. Lots of places term can be used in between the high and low level.


      In what I am concerned, asynchronous means the ability of using a client thread to put data on the wire and return immediately without waiting for any reply. "True" asynchronicity in this context means that no unnecessary work is done (like sending a reply but discarding it on the client side).

      Tim wrote:

      Here is a an example that came up in the forums the other day (there is also a support case related to this from another customer with JBossMQ which suffers from a similar problem):

      The customer has a fantastic high bandwidth network, but it has very high latency (large round trip time). They would love to use their network to it's capacity with our messaging product. Currently when we deliver messages (usually one by one) from the server to the client consumer they are delivered by a remoting RPC call. This writes to a socket then waits for a response. Therefore the minimum amount of time to send one message is 2 x latency. And this is for every message (!).
      The "correct" way to do this is to forget responses, just write to the socket and carry on. Flow control messages from the client to the server then prevent the client being overrun with messages (this is kind of analogous to how TCP flow control works - although there are some differences).


      Your "correct" approach raises the following issue:

      Assume that you want to write a client that needs guaranteed delivery on a queue. You write that client to create a persistent message and send the message by invoking a producer's send() method. As soon as your send() method returns without an exception, this is a hard guarantee to your client that the messaging broker accepted the message for further delivery. You cannot have that guarantee without you call actually reaching the server. At that point, the client can call System.exit() and go to the client's heaven, fully assured that his message is being taken care of. If you use the asynchronous approach you describe, the client will be never able to rely on this guarantee.

      Tim wrote:

      Moving on to serialization. JBoss Remoting is based around the idea of invocations that get serialized. We a) don't want invocations and b) In almost every case we don't want serialization either (there is one case we do want serialization) - this is because we know the types of the objects being transmitted at compile time so we can encode that information in a much more efficient way (in a byte) - the overhead of serialization is just to much for us.


      I disagree with a). An RPC model is very convenient when dealing with life cycle operations such as creating connections, sessions, producer, consumer, browsers, etc. It's a very convenient model for a high granularity reliable operations.

      The alternative would be to send an asynchronous invocation to say, create a Session, and then block on the client for an explicit acknowledgment from the server. Why would you want to do that when you have RPC for free?

      I agree with b), though. This is an overhead we can do without.



        • 1. Re: Remoting - Client Asynchronous Calls
          ovidiu.feodorov

           

          Tim wrote:

          In many cases with messaging we don't want RPC (in some cases we do).


          This is a table that contains ALL client-to-server interactions, represented as metod signatures.

           Method Name Return Value
          ConnectionFactoryEndpoint
           createConnectionDelegate() ConnectionDelegate
           getClientAOPConfig() byte[]
           getIdBlock() IdBlock
          ConnectionEndpoint
           createSessionDelegate() SessionDelegate
           getClientID() String
           getPreparedTransactions() Xid[]
           isClosed() boolean
           setClientID() -
           start() -
           stop() -
           sendTransaction() -
           close() -
           closing() -
          SessionEndpoint
           createConsumerDelegate() ConsumerDelegate
           createBrowserDelegate() BrowserDelegate
           createQueue() JBossQueue
           createTopic() JBossTopic
           isClosed() boolean
           acknowledgeBatch() -
           acknowledge() -
           addTemporaryDestination() -
           deleteTemporaryDestination()-
           unsubscribe() -
           send() -
           cancelDeliveries() -
           close() -
           closing() -
          ConsumerEndpoint
           isClosed() boolean
           more() -
           close() -
           closing() -
          BrowserEndpoint
           nextMessage() Message
           hasNextMessage() boolean
           nextMessageBlock() Message[]
          
          


          This table lists server-to-client interactions.

           Method Name Return Value
          
          MessageCallbackHandler handleMessage() HandleMessageResponse
          
          



          I don't think that methods that return a non-void result need arguing (with the exception of MessageCallbackHandler.handleMessage(), which I will talk about later). They represent round-trips to the server, and they are invoked because the client needs something from the server. The client cannot continue unless it has the reply it needs, so trying to suggest to replace these with asynchronous calls just for the sake of it seems to me at least questionable

          Now let's analyze the methods that don't return anything, but represent life-cycle operations, and rarely used mutators (ConnectionEndpoint's start(), stop(), close(), closing(), setClientID(), SessionEndpoint's addTemporaryDestination(), deleteTemporaryDestination(), unsubscribe(), close(), closing(), ConsumerEndpoint's close() and closing()). How many times a second do you start, stop or close connections? Or how many times a second do you add or delete temporary destinations. These are life cycle methods (we use DEBUG to log them), why would they need to be optimized for speed, when reliability and the convenience of getting the acknowledgment of their successful completion (or not) are much more important? Optimizing these operation do not improve message throughput, which is in the end all that matters.

          ConnectionEnpoint.sendTransaction(). When your transaction successfully completes on the server (or, more importantly, when it does not), your client wants to know. Using an RPC style invocation makes propagating exceptions back easy. The alternative would be to asynchronously send the "transaction" and then somehow listen for a confirmation (positive or negative). What would be the benefit of doing that?

          SessionEndpoint.send(). See the previous "correct approach" discussion. If you don't invoke this method synchronously, how does the client know that the message has been accepted by the broker?

          SessionEndpoint.cancelDeliveries(). This is used to cancel any undelivered messages left in the client buffer when the consumer is closed or to cancel any messages that couldn't be redelivered locally at session recovery. Both cases are extraordinary events that do not impact message throughput performance in any way.

          This leaves us SessionEndpoint.acknowledgeBatch(), SessionEnpoint.acknoweldge() and ConsumerEndpoint.more() that can arguably improve the throughput if they're made asynchronous. Also, sending the message to the client for delivery is a good candidate for asynchronicity.

          So, getting back to what you said, "In many cases with messaging we don't want RPC (in some cases we do).", I would actually say that in many cases with messaging we want RPC, and is some very special cases we don't.

          Remoting should support both modes, and we should choose carefully how we use them.


          • 2. Re: Remoting - Client Asynchronous Calls

             


            In what release is this available? I would like to play a little bit with it.


            It is in HEAD and targeted for 2.2.0.Beta1 (Bluto) (which is already past due and probably won?t be out for a while).


            • 3. Re: Remoting - Client Asynchronous Calls
              timfox

               

              "ovidiu.feodorov@jboss.com" wrote:

              Your "correct" approach raises the following issue:

              Assume that you want to write a client that needs guaranteed delivery on a queue. You write that client to create a persistent message and send the message by invoking a producer's send() method. As soon as your send() method returns without an exception, this is a hard guarantee to your client that the messaging broker accepted the message for further delivery. You cannot have that guarantee without you call actually reaching the server. At that point, the client can call System.exit() and go to the client's heaven, fully assured that his message is being taken care of. If you use the asynchronous approach you describe, the client will be never able to rely on this guarantee.


              When I was talking about flow control previously I was specifically talking about pushing messages from server -> consumer, not sending messages from client -> server.

              For pushing from server->client, for the reasons already stated (latency problems) an RPC model will ruin performance.

              For sending from client to server, yes you are correct in stating this must be RPC, for JMS at least, although AMQP is less strict on this and allows asynchronous send I believe.

              (Also bear in mind, that an absence of an exception does imply success, but the presence of an exception does not imply failure - the message could actually have been added to the queue - but this is not particularly relevant to the discussion).

              So, for JMS, to maximise sending throughput for a high latency network would be done on the application level by sending messages in transactions with multiple messages per transaction. This allows many messages to be sent in one RPC.


              I disagree with a). An RPC model is very convenient when dealing with life cycle operations such as creating connections, sessions, producer, consumer, browsers, etc. It's a very convenient model for a high granularity reliable operations.


              Let's be clear on this. I am not advocating an all or nothing asynchronous approach. Of course some operations need to be synchronous.



              • 4. Re: Remoting - Client Asynchronous Calls
                timfox

                 

                "ovidiu.feodorov@jboss.com" wrote:

                I don't think that methods that return a non-void result need arguing (with the exception of MessageCallbackHandler.handleMessage(), which I will talk about later). They represent round-trips to the server, and they are invoked because the client needs something from the server. The client cannot continue unless it has the reply it needs, so trying to suggest to replace these with asynchronous calls just for the sake of it seems to me at least questionable

                Now let's analyze the methods that don't return anything, but represent life-cycle operations, and rarely used mutators (ConnectionEndpoint's start(), stop(), close(), closing(), setClientID(), SessionEndpoint's addTemporaryDestination(), deleteTemporaryDestination(), unsubscribe(), close(), closing(), ConsumerEndpoint's close() and closing()). How many times a second do you start, stop or close connections? Or how many times a second do you add or delete temporary destinations. These are life cycle methods (we use DEBUG to log them), why would they need to be optimized for speed, when reliability and the convenience of getting the acknowledgment of their successful completion (or not) are much more important? Optimizing these operation do not improve message throughput, which is in the end all that matters.

                ConnectionEnpoint.sendTransaction(). When your transaction successfully completes on the server (or, more importantly, when it does not), your client wants to know. Using an RPC style invocation makes propagating exceptions back easy. The alternative would be to asynchronously send the "transaction" and then somehow listen for a confirmation (positive or negative). What would be the benefit of doing that?

                SessionEndpoint.send(). See the previous "correct approach" discussion. If you don't invoke this method synchronously, how does the client know that the message has been accepted by the broker?

                SessionEndpoint.cancelDeliveries(). This is used to cancel any undelivered messages left in the client buffer when the consumer is closed or to cancel any messages that couldn't be redelivered locally at session recovery. Both cases are extraordinary events that do not impact message throughput performance in any way.

                This leaves us SessionEndpoint.acknowledgeBatch(), SessionEnpoint.acknoweldge() and ConsumerEndpoint.more() that can arguably improve the throughput if they're made asynchronous. Also, sending the message to the client for delivery is a good candidate for asynchronicity.

                So, getting back to what you said, "In many cases with messaging we don't want RPC (in some cases we do).", I would actually say that in many cases with messaging we want RPC, and is some very special cases we don't.

                Remoting should support both modes, and we should choose carefully how we use them.


                To cut a long story short, of course many operations need to be synchronous, this is not in contention.

                However most of the operations (with the exception of send) on the primary execution paths can probably be done asynchronously.

                These are the ones that need to be fast, they include:

                Sending messages from server=>client

                Flow control messages (currently just more()) but we will need to extend this probably.

                Acknowledgements (I am pretty sure but not 100% about this one - it needs more thought)



                • 5. Re: Remoting - Client Asynchronous Calls
                  ovidiu.feodorov

                   

                  Tim wrote:

                  For pushing from server->client, for the reasons already stated (latency problems) an RPC model will ruin performance.


                  As far as I remember, the current code relies on the fact that it receives an acknowledgment (in the form of an exception NOT being thrown) for each server->client delivery.

                  Tim wrote:

                  For sending from client to server, yes you are correct in stating this must be RPC, for JMS at least, although AMQP is less strict on this and allows asynchronous send I believe.


                  Well, in this case AMQP has a problem if it doesn't also provide for synchronous client-to-server delivery. Need to hit the books, to make sure this is (or not) the case.

                  Tim wrote:

                  So, for JMS, to maximise sending throughput for a high latency network would be done on the application level by sending messages in transactions with multiple messages per transaction. This allows many messages to be sent in one RPC.


                  Right. So ConnectionEnpoint.sendTransaction() IS a RPC.

                  Tim wrote:

                  Let's be clear on this. I am not advocating an all or nothing asynchronous approach. Of course some operations need to be synchronous.


                  We're clear. We need both. And this needs to be reflected in the Remoting API. I'll soon add the document with the proposed changes.


                  Tim wrote:

                  However most of the operations (with the exception of send) on the primary execution paths can probably be done asynchronously.
                  These are the ones that need to be fast, they include:
                  Sending messages from server=>client
                  Flow control messages (currently just more()) but we will need to extend this probably.
                  Acknowledgements (I am pretty sure but not 100% about this one - it needs more thought)


                  This is exactly what I was saying. See above (or below):

                  Ovidiu wrote:

                  This leaves us SessionEndpoint.acknowledgeBatch(), SessionEnpoint.acknoweldge() and ConsumerEndpoint.more() that can arguably improve the throughput if they're made asynchronous. Also, sending the message to the client for delivery is a good candidate for asynchronicity.




                  • 6. Re: Remoting - Client Asynchronous Calls
                    timfox

                     

                    "ovidiu.feodorov@jboss.com" wrote:


                    As far as I remember, the current code relies on the fact that it receives an acknowledgment (in the form of an exception NOT being thrown) for each server->client delivery.



                    Sure, the current code uses RPC, we don't have a choice with the version of remoting we are using, that's the only reason it's that way.

                    But it is unnecessary.

                    Actually we can radically simplify the server consumer endpoint message dispatch if we go async - no need for the deliverer or any of that stuff any more - less context switches etc


                    Well, in this case AMQP has a problem if it doesn't also provide for synchronous client-to-server delivery. Need to hit the books, to make sure this is (or not) the case.


                    I think AMQP supports both sync AND async. I need to hit the books too :)


                    Right. So ConnectionEnpoint.sendTransaction() IS a RPC.


                    It is. Did I say otherwise?


                    We're clear. We need both. And this needs to be reflected in the Remoting API. I'll soon add the document with the proposed changes.


                    Cool


                    • 7. Re: Remoting - Client Asynchronous Calls
                      genman

                      AMQP is certainly meant to work async on the client side. Non-durable messages could be supported by using a memory buffer. (There would have to be some sort of "flow control" to avoid out of memory conditions.)

                      For durability, SonicMQ supports durability by using persistence on the client side; MessageProducer.send() persists messages on disk, then delivery happens asynchronously in another thread. Again, some sort of mechanism would provide flow control.

                      For "thin clients", you'd to have to provide some sort of simple client-side persistence model. A central database wouldn't make much sense. I'm guessing a "simple" transaction log might work. ("simple" rarely is though.) There'd have to be some sort of recovery mechanism and callbacks for error detection...

                      • 8. Re: Remoting - Client Asynchronous Calls
                        ovidiu.feodorov

                         

                        genman wrote:
                        For durability, SonicMQ supports durability by using persistence on the client side; MessageProducer.send() persists messages on disk, then delivery happens asynchronously in another thread. Again, some sort of mechanism would provide flow control.

                        For "thin clients", you'd to have to provide some sort of simple client-side persistence model. A central database wouldn't make much sense. I'm guessing a "simple" transaction log might work. ("simple" rarely is though.) There'd have to be some sort of recovery mechanism and callbacks for error detection...


                        Yes, this is the "point of sale" scenario. There is a JIRA task about this ...

                        • 9. Re: Remoting - Client Asynchronous Calls
                          timfox

                          We have to be very careful with our definition of "durable" when persisting on the client side.

                          It may well be a much weaker level of durability than persisting on the server.

                          Typically client machines might be basic desktop boxes with cheap hard disks not under the direct control of the IT department.

                          Server boxes may well be highly reliable RAID arrays.

                          We should therefore be very careful about saying we can guarantee reliable delivery just by persisting on the client side. Of course this depends on the individual setup but it's not always (and probably in most situations) is not an acceptable way to get durability.

                          In some situations it may well be sufficient e.g. POS. But we should use this as a general technique.

                          • 10. Re: Remoting - Client Asynchronous Calls
                            timfox

                            Error in last post: I mean we *shouldn't* use this as a general technique.

                            • 11. Re: Remoting - Client Asynchronous Calls
                              ovidiu.feodorov

                              Obviously, client-side storage won't provide the same reliability guarantees. However, if this is made clear when configuring the corresponding QoS level, the user will take the final decision on whether the risk is acceptable or not. In some cases, is much better than not providing the choice.


                              • 12. Re: Remoting - Client Asynchronous Calls
                                ovidiu.feodorov

                                This is the wiki document that contains proposed Remoting API extensions: http://wiki.jboss.org/wiki/Wiki.jsp?page=JBossMessagingRemotingAPIExtensions

                                Once materialized, it should isolate Messaging from future Remoting implementation changes.

                                Please comment.

                                • 13. Re: Remoting - Client Asynchronous Calls
                                  genman

                                  Having reviewed what you wrote and the remoting API, I don't quite understand why additional methods are necessary.

                                  "2. Synchronous raw send" - In general, I find APIs that use byte[] sort of cumbersome. Its brother "send(byte [], int off, int len)" is better, but still ugly. I would avoid that sort of thing. Using actual objects is better. I mean, if you really want to optimize things, have your objects implement Externalizable. I don't see how "instanceof" and a cast would be slow.

                                  "4. asynchSend" - I can't see how this is different than "invokeOneWay" I guess you say that, but still suggest adding it?

                                  • 14. Re: Remoting - Client Asynchronous Calls
                                    ovidiu.feodorov

                                     

                                    genman wrote:

                                    "2. Synchronous raw send" - In general, I find APIs that use byte[] sort of cumbersome. Its brother "send(byte [], int off, int len)" is better, but still ugly. I would avoid that sort of thing. Using actual objects is better. I mean, if you really want to optimize things, have your objects implement Externalizable. I don't see how "instanceof" and a cast would be slow.


                                    We most likely won't have any send(byte[]), but an send(ByteBuffer) or equivalent instead. There's a vigorous discussion on this subject going on here: http://www.jboss.org/index.html?module=bb&op=viewtopic&t=92869

                                    The point in contention is whether to offer this as a top level API call, or make it accessible via a "chained marshaller" mechanism Tom Elrod is experimenting with as we speak. We're supposed to have a conference to discuss this, some time mid week. If interested, I can send you the call info.

                                    genman wrote:

                                    "4. asynchSend" - I can't see how this is different than "invokeOneWay" I guess you say that, but still suggest adding it?


                                    I like symmetry. However, adding a new method name doesn't make too much sense if the current one provides the behavior we need, so I guess I won't make too much noise over that.

                                    The document is just a proposal, it's not etched in stone :).