I'm quite dissapointed because performance test are not as good as I expected. I'm changing some thread parameters (core and max threads) in config files but I think smx thread behaviour is not that one I have set. Per example, If I set core thread on 32, and max thread on 64, pool never create more than 32 thread per component (camel or cxf-bc), even If I lunch more than 32 concurrent request ( It will cause smx colapse threads waiting for exchange and no more request will be processed ).
Setted example configuration in "/org.apache.servicemix.jbi.cfg":
I would appreciate an answer.
If I don't find out a way to increase performance up to 100 user (threads) I will think in another possible solution to my scenario because now I'm using very simple tests (consumer -> provider) that should run better and faster than a real test.
Seems your threadpool configuration doesn't get picked up. Where you put org.apache.servicemix.components.cxfbc.cfg in? It should be $SMX_HOME/etc/ folder.
Also, could you change
to see if you get better performance?
Btw, you should use org.apache.servicemix.nmr.cfg but not org.apache.servicemix.jbi.cfg to configure the NMR threadpool.
Created a new config file $SMX_HOME/etc/org.apache.servicemix.nmr.cfg and setted:
But behaviour of smx didn't change. ThreadPool never creates more threads than the value of "corePoolSize", in this case 64.
camel threads => 64
cxf-bc threads =>64
More than aprox. 60 users (client threads) will bring to a standstill to smx. Threads appear to be "waiting for exchange XXXXXXXXXXXXXXX".
I can increase corePoolSize, of course, but that's not the case. I would know if maximumPoolSize works and if it really set up the top amount of threads that will be created if there are more requests than value of corePoolSize threads...
Did you ever put queueSize(default is 1024 iirc) in org.apache.servicemix.nmr.cfg,
Here is the rule about how *PoolSize works
if the number of threads is less than corePoolSize, the executor will create a new thread to handle the job
if the number of queued jobs is less than queueSize, the job is queued
if the queue is full and the number of threads is less than maximumPoolSize, a new thread is created to handle the job
else, the current thread will handle the job
so the job is queued until queue is full and you can't see a new thread created until queue is full. If you want to see create more thread than corePoolSize immediately, you need set queueSize=0
Btw, could you add synchronous="false" to your cxf bc consumer endpoint configuration, this will get better performance.
I hadn't put queueSize parametger in org.apache.servicemix.nmr.cfg but I did it and changed also queueSize = 0 in every configuration file (org.apache.servicemix.nmr.cfg, org.apache.servicemix.jbi.cfg and org.apache.servicemix.components.cxfbc.cfg). And since them I can see how there are some variations in the amount of threads that before I couldn't see. But even setting queueSize = 0 and maximumPoolSize = 1024, smx never reaches the maximum value or exceeds the corePoolSize.
Performance tests with more than corePoolSize value users usually brings to a standstill Smx and I have to restart it. Perhaps I am not understanding very well how thread pool works, but I think that It should queue a request (or 1024 request by default) and process them when there is a free thread left, don't it?
I also tried to increase jvm heap and permSize memory, just in case, but there weren't any memory problems.
The next I will check out is to add synchronous="false" to my cxf bc consumer endpoint configuration...
I don't know what the solution is, but I'd like to say that I'm experiencing the exact same problem.
I usually resolve it by stopping/starting the CXF - BC/SE via the smx console..
Add synchronous="false" attribute for your cxf bc consumer endpoint should resolve it. More details please see my blog "Why ESB server hang after runing for a while?" from http://freemanfang.blogspot.com
Should I also add synchronous="false" attribute for my cxf bc provider endpoint? From time to time server hang, not as frequently as before but It still happens!
After reading your blog post I think I did not configure my working flow correctly because smx do not create more than "corePoolSize value" threads even though I setted queueSize = 0. I'm using:
cxfbc-consumer ( synchronous="false" )
camel-enricher (bean implementing enrich method)
camel-recipientList (bean implementing getTargets())
camel-transformer (xslt file)
First test (call 1): 200 users
Second test (call 2): 300 users
Third test (call 3): 400 users -> smx hang (threads waiting for exchange).
I atach a jconsole screenshot.
- thread configuraction
creating and setting "org.apache.servicemix.nmr.cfg" has no effect in my case.
Edited by: fsalceda on Oct 6, 2010 1:48 PM
threads_waiting_for_exchange.jpg 257.7 KB