2 Replies Latest reply on Jun 10, 2013 1:25 PM by paladin8

    Multiple sessions/producers for higher throughput?

    paladin8
    paladin8 Jun 7, 2013 9:44 PM

      Hi,

       

      We are using the core API to write messages to HornetQ. As I understand it, the ServerLocator and ClientSessionFactory should be initialized once and shared across all threads (i.e. they are thread-safe). On the other hand, the ClientSession and ClientProducer are not thread-safe, so it is necessary to either (1) create separate instances of them for each thread/use some sort of pool, or (2) synchronize access to single shared instances. Assuming we have 100-200 threads producing to HornetQ and the server/network are not bottlenecks, does (1) give better throughput or is there some kind of shared resource that would limit the throughput anyway? We currently do (2) for simplicity and are wondering if creating multiple sessions/producers will be beneficial.

       

      Thanks,

      Jeffrey

      • 213 Views
      • Tags:
        • 1. Re: Multiple sessions/producers for higher throughput?
          ataylor
          ataylor Jun 10, 2013 8:18 AM (in response to paladin8)

          option 1 will probably give you a better throughput but this all depends on your disc speed, network speed and whether or not the messages are persistent (as well as some other configuration, ack mode, producer send rate of your app etc). If your messages are persistent then the disc speed will ultimately be the bottle neck, other wise network speed will be. I would go for option 1 and pool your sessions/producers and load test with a pool of 1 (which is basically option 2) and then find the most performant pool size.

          1 of 1 people found this helpful
            • Actions
          • 2. Re: Multiple sessions/producers for higher throughput?
            paladin8
            paladin8 Jun 10, 2013 1:25 PM (in response to ataylor)

            Thanks Andy; we'll try that out, then. Am I correct in thinking that using multiple sessions/producers will open up multiple connections to the server? I recall reading somewhere in the documentation that a single ClientSessionFactory has a default of 8 connections.

             

            FWIW, we are using acks and persistent messages, so that certainly affects the throughput. The reason I brought this up is that we are observing many threads in our app blocked in ClientProducer code without obvious network/disk bottlenecks on the server side.

              • Actions