Help: Problems with asynchronous parallel processing - can J

[ivanvaghi]

I hope someone in the forum can give us a hand on this.

We have been using jboss for some time in a project, mainly as a JMS server.

However we have now hit a performance problem that we cannot solve and that does not appear on other application servers.

Our architecture has a producer of "jobs" that are put on a queue. Several consumers, from the same or different PCs can get jobs from the queue and carry them out. This architecture was designed to allow the plug-in of processing power on demand.

However, as we started adding consumers, processing times started to increase linearly with the number of consumers rather than decreasing!

I am talking about the receiving time here... first we fill up the queue, then we run a number of JMS clients both on a multiprocessor machine and other PCs on the LAN. The clients dequeue the messages, do some JDBS accesses and then process the messages.

The processing time is negligible compared to the DB accesses and the JMS connection.

If it takes me 100 seconds to empty a queue using one client, it takes me about 200 seconds when I use two clients, and about 300 seconds with 3 clients.

This defies the purpose of putting the messages on a single queue for distributed consumption... we solved the problem splitting the queue in many smaller queues, and getting different clients on different queues, but this approach is inflexible and not very scaleable, requiring manual fine-tuning.

This has put us in a position where either we solve the problem or we have to switch to a different application server... which we don't really want to do.

Do you guys have any advice on how to solve this?

Thanks in advance for any help!

Ivan

[craigday]

If you are just using the JMS component and you dont need clustering then I can reccommend SUN IMQ Message Queueing server. Its free in non-clustered mode, stable and very reliable, its not the fastest, but it doesnt suffer from the kind of performance degradation you are talking about, the admin console is sweet,...

Dont get me wrong, I love JBoss and we use it extensively, just not the JMS part. Your integration tier and services always need to be rock-solid.

cheers
craig

[craigday]

Sorry,
My mistake, you are doing multi-consumer queue type work, thats not an available queue delivery policy in the free version of Suns IMQ.

sorry
craig