"bstansberry@jboss.com" wrote:
OK, good. That knocks off the trivial stuff. I'd still like to get Bela's input on the thread_pool.queue_enabled and thread_pool.queue_max_size, where I think the values you guys had are better.
That leaves enable_bundling, where I want to understand your usage so I can pick the best default.
How data intensive and latency sensitive is the jbm-control channel? And is it used for request/response type messages (i.e. thread on node A sends a message, blocks waiting for a response from the other nodes)? Or just for async (thread on A sends and returns; doesn't block for response)?
The enable_bundling value impacts that. When it is true, there can be an up to 30 ms delay before a message is sent. Adds latency to when the message arrives, which may or may not matter to JBM. That delay doesn't cause the thread that told JGroups to send the message to block -- unless the thread is blocking waiting for a response. If it's waiting for a response, that thread might wait 30 ms for the msg to be sent, plus another 30 ms for the response to be sent.
The offset to that increased latency is higher throughput under heavy load w/ enable_bundling=true. The "udp" stack will be used for session replication, which *might* involve heavy load, so we've optimized for that. IMHO that was a marginal decision though, so I want to understand JBM's usage.
Regarding bundling: if you send many (larger) messages and fill up the buffer (>= max_bundle_size), then there won't be any latency. Say, we're filling up 60'000 bytes in 2ms, then the message will get sent in 2ms. However (unless you send large return values), the responses will usually run into the max_bundle_time timeout (default: 30ms).
You can tweak the max_bundle_size or max_bundle_timeout to a lower value, and/or change the enable_unicast_bundling flag.
Regarding the max thread pool size: it doesn't hurt to make this value big, but make sure you also set keep_alive_time to a reasonable value, so that threads are placed back into the pool after that time elapsed, so they are available to other service.
