Scaling UP tests
clebert.suconic May 23, 2008 5:26 PMI have been trying to scale up few tests with nonTransactionSends/Persistent (that means.. every send requires transaction), and this scaled pretty nicely with AIO.
With NIO I've got about 350 sends / second with a single perfSender, because of the syncs on every send.
With AIO I've got 1200 sends /s econd with a single perfSender.
I've also tested with 5 perfSender, and I've got the same rate on every perfSender. That means.. it scaled without any problem.
*However* when I added a listener, things were much different.
The rate on the send was terrible, and I got a bunch of threads waiting on this condition:
[java] "pool-7-thread-4" prio=1 tid=0x00007fb8e96d56c0 nid=0x3d44 waiting for monitor entry [0x000000004396f000..0x000000004396fb20] [java] at org.jboss.messaging.core.server.impl.QueueImpl.addLast(QueueImpl.java:159) [java] - waiting to lock <0x00007fb8f0a62968> (a org.jboss.messaging.core.server.impl.QueueImpl) [java] at org.jboss.messaging.core.server.impl.ServerSessionImpl.send(ServerSessionImpl.java:321) [java] at org.jboss.messaging.core.server.impl.ServerProducerImpl.send(ServerProducerImpl.java:103) [java] at org.jboss.messaging.core.server.impl.ServerProducerPacketHandler.doHandle(ServerProducerPacketHandler.java:61) [java] at org.jboss.messaging.core.server.impl.ServerPacketHandlerSupport.handle(ServerPacketHandlerSupport.java:48) [java] at org.jboss.messaging.core.remoting.impl.PacketDispatcherImpl.dispatch(PacketDispatcherImpl.java:139) [java] at org.jboss.messaging.core.remoting.impl.mina.MinaHandler.messageReceivedInternal(MinaHandler.java:283) [java] at org.jboss.messaging.core.remoting.impl.mina.MinaHandler.access$000(MinaHandler.java:32) [java] at org.jboss.messaging.core.remoting.impl.mina.MinaHandler$1.run(MinaHandler.java:165) [java] at org.jboss.messaging.util.OrderedExecutorFactory$ChildExecutor.run(OrderedExecutorFactory.java:70) [java] at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:650) [java] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:675) [java] at java.lang.Thread.run(Thread.java:595) [java]
While this thread was holding the lock:
[java] "pool-7-thread-1" prio=1 tid=0x00007fb8e9870bc0 nid=0x3d33 in Object.wait() [0x00000000413a4000..0x00000000413a4da0] [java] at java.lang.Object.wait(Native Method) [java] - waiting on <0x00007fb8f09b0528> (a org.jboss.messaging.core.remoting.impl.mina.MinaHandler) [java] at org.jboss.messaging.core.remoting.impl.mina.MinaHandler.checkWrite(MinaHandler.java:210) [java] - locked <0x00007fb8f09b0528> (a org.jboss.messaging.core.remoting.impl.mina.MinaHandler) [java] at org.jboss.messaging.core.remoting.impl.mina.MinaHandler$2.send(MinaHandler.java:251) [java] at org.jboss.messaging.core.server.impl.DeliveryImpl.deliver(DeliveryImpl.java:80) [java] at org.jboss.messaging.core.server.impl.ServerSessionImpl.handleDelivery(ServerSessionImpl.java:222) [java] at org.jboss.messaging.core.server.impl.ServerConsumerImpl.handle(ServerConsumerImpl.java:217) [java] - locked <0x00007fb8f0a62a90> (a java.lang.Object) [java] at org.jboss.messaging.core.server.impl.QueueImpl.deliver(QueueImpl.java:631) [java] at org.jboss.messaging.core.server.impl.QueueImpl.add(QueueImpl.java:528) [java] at org.jboss.messaging.core.server.impl.QueueImpl.addLast(QueueImpl.java:159) [java] - locked <0x00007fb8f0a62968> (a org.jboss.messaging.core.server.impl.QueueImpl) [java] at org.jboss.messaging.core.server.impl.ServerSessionImpl.send(ServerSessionImpl.java:321) [java] at org.jboss.messaging.core.server.impl.ServerProducerImpl.send(ServerProducerImpl.java:103) [java] at org.jboss.messaging.core.server.impl.ServerProducerPacketHandler.doHandle(ServerProducerPacketHandler.java:61) [java] at org.jboss.messaging.core.server.impl.ServerPacketHandlerSupport.handle(ServerPacketHandlerSupport.java:48) [java] at org.jboss.messaging.core.remoting.impl.PacketDispatcherImpl.dispatch(PacketDispatcherImpl.java:139) [java] at org.jboss.messaging.core.remoting.impl.mina.MinaHandler.messageReceivedInternal(MinaHandler.java:283) [java] at org.jboss.messaging.core.remoting.impl.mina.MinaHandler.access$000(MinaHandler.java:32) [java] at org.jboss.messaging.core.remoting.impl.mina.MinaHandler$1.run(MinaHandler.java:165) [java] at org.jboss.messaging.util.OrderedExecutorFactory$ChildExecutor.run(OrderedExecutorFactory.java:70) [java] at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:650) [java] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:675) [java] at java.lang.Thread.run(Thread.java:595) [java]
I haven't taken any conclusions about what's wrong as I haven't looked on the code yet. These are just figures from my test.