How to deal with exceptional throughput
Hi all,
I hope this doesn't sound very stupid, but I'm going ask anyway just in case ;-)
I am configuring a JBoss ESB 4.9 standalone system. A freestanding client using the ServiceInvoker will send one or more Messages into the bus one immediately after the other via a transacted JMS provider. These messages can contain a byte[] payload of indeterminate length. Naturally, flags go up here; there must be enough memory available to handle the Messages. This is more true on the server than the client, since the client only needs enough memory to handle the largest single Message, yet the bus will be handling some combination of one or more at the same time.
The first question I have is, does anyone have any suggestions for maximizing the server's use of the memory that is available? We'll have to establish some kind of memory requirements, but it would be nice to use this memory as efficiently as possible.
For example, each Message can be processed independently of all the others, so we could add some small delay on the client's side between each Message. However, we feel this will only have minimal impact. Processing each message involves an FTP action to a remote server, and the larger files start to take whole seconds to complete. The client itself is spawned by a GUI with a (possibly impatient) user attached; the longer he waits, the more likely he'll just interrupt the whole process and try again.
The second question I have involves what to do when (because inevitably it will) things go wrong; the server starts throwing OutOfMemoryErrors and the Messages after redelivery eventually end up on the DLQ.
What I've noticed is that if we restart the server, the DLQ service subsequently fails with:
2011-04-11 16:12:23,896 ERROR [org.jboss.messaging.util.ExceptionUtil] Queue[null, name=DLQ] startService
java.io.EOFException
at java.io.DataInputStream.readFully(DataInputStream.java:178)
at java.io.DataInputStream.readUTF(DataInputStream.java:565)
at java.io.DataInputStream.readUTF(DataInputStream.java:522)
at org.jboss.messaging.util.StreamUtils.readMap(StreamUtils.java:313)
at org.jboss.messaging.core.impl.JDBCPersistenceManager.bytesToMap(JDBCPersistenceManager.java:2331)
at org.jboss.messaging.core.impl.JDBCPersistenceManager$1GetMessageListTX.doTransaction(JDBCPersistenceManager.java:578)
at org.jboss.messaging.core.impl.JDBCPersistenceManager$1GetMessageListTX.doTransaction(JDBCPersistenceManager.java:498)
at org.jboss.messaging.core.impl.JDBCSupport$JDBCTxRunner2.execute(JDBCSupport.java:474)
at org.jboss.messaging.core.impl.JDBCSupport$JDBCTxRunner2.executeWithRetry(JDBCSupport.java:512)
at org.jboss.messaging.core.impl.JDBCPersistenceManager.getMessages(JDBCPersistenceManager.java:615)
at org.jboss.messaging.core.impl.PagingChannelSupport.processReferences(PagingChannelSupport.java:574)
at org.jboss.messaging.core.impl.PagingChannelSupport.doLoad(PagingChannelSupport.java:506)
at org.jboss.messaging.core.impl.PagingChannelSupport.load(PagingChannelSupport.java:213)
at org.jboss.jms.server.destination.QueueService.startService(QueueService.java:105)
(etc)
My assumption is that the server on startup tries to read the messages on the DLQ and encounters problems because there still isn't enough memory available. I doubt this assumption, however, because I can't imagine why anything on the DLQ would be read. However the only way to solve this problem is to manually delete the messages from the database and they all have channel_id 0 (DLQ).
Any advice or insight would be appreciated.
Thanks
Adam