Large Messages and the JMS Bridge

[clebert.suconic]

This is a discussion for https://jira.jboss.org/jira/browse/JBMESSAGING-1601

Once you received a LargeMessage, as soon as you consume another message you can't move back to an already received message.

This is explained at 22.6 on the user' s doc:

http://labs.jboss.com/file-access/default/members/jbossmessaging/freezone/docs/usermanual-2.0.0.beta3/html/large-messages.html#d0e3733

But it happens that the JMS Bridge is caching messages, and it needs to eventually go back and reprocess messages.

I was looking around the code, and I can think of two options to fix it:

- Ressurrect the option for FileConsumers.
(This would be optional. and all could be controlled within the LargeMessageBuffer).
This would be a good option for users caching messages in other scenarios. And this is my preferred option.

- Make the JMSBridge aware of LargeMessages. I could have some JMS property identifying a message as LargeMessage, and the JMSBridge won't cache those messages.

- Make the user to turn of caching on the bridge. (I don't like this option. I would prefer the Bridge dealing with it automatically).

[clebert.suconic]

I will choose option I - allowing Flie Consumers as an option.

It will be different than what I had before there. Before you would require it in order to receive large messages without blowing the memory. But now this would be optional.

[clebert.suconic]

There is a field directory on ClientConsumerImpl that will be used for this.


I told Tim Fox the other day to get rid of it, but I will find use for it now.

[clebert.suconic]

We had a relevant conversation about this today on the IRC, and I wanted to keep a record of it here:

"Clebert, Andy and Jeff on IRC" wrote:

(10:15:18 AM) clebert: Cache on LargeMessage
(10:15:24 AM) clebert: the way I implemented is...
(10:15:34 AM) clebert: I added a new attribute on the SessionFactory and ConnectionFactory
(10:15:42 AM) clebert: cacheLargeMessageClient
(10:15:45 AM) clebert: with default = false
(10:16:07 AM) clebert: if that is set, the LargeMessageBuffer will use a temporary file (on the TempDir) to hold on the large message content
(10:16:19 AM) clebert: tempDir = File.createTempFile()
(10:16:29 AM) clebert: the file is marked to be deleted after exit
(10:16:36 AM) clebert: and finalize will also delete the temp file
(10:16:48 AM) clebert: that will be probably used only on the bridge... or if the user needs it
(10:16:58 AM) clebert: I only need to document it now
(10:17:07 AM) AndyTaylor: clebert: so the message will stay in temp for the user to obtain
(10:17:30 AM) clebert: AndyTaylor: LargeMessages are usually a stream of packets...
(10:17:35 AM) clebert: if that property is true...
(10:17:39 AM) clebert: there will be a file on the client...
(10:17:48 AM) clebert: so if the user moves the buffer back, it will recover the content from the file
(10:18:05 AM) clebert: otherwise it' s not possible to move a largemessage buffer back, since it would be a streaming otherwise
(10:18:12 AM) AndyTaylor: clebert: ok i see.
(10:18:22 AM) jmesnil: clebert: do you think it is worth to have a example with a cached large message?
(10:19:05 AM) clebert: jmesnil: I' m not sure... I would prefer users not using that feature... only letting it for cases where you need to cache the message (i.e., the Bridge)
(10:19:16 AM) AndyTaylor: clebert: +1, i agree
(10:19:17 AM) clebert: jmesnil: I' m not 100% sure
(10:19:25 AM) AndyTaylor: clebert: internal use only
(10:19:26 AM) jmesnil: clebert: ok, it's more an "internal" feature
(10:19:54 AM) clebert: jmesnil: allthough I would need to set the attribute for the Bridge
(10:19:59 AM) clebert: on the CF used by the Bridge
(10:20:10 AM) clebert: maybe I would only set it on the JMSBridge
(10:20:38 AM) jmesnil: clebert: on the core bridge too?
(10:21:05 AM) clebert: jmesnil: no... the core bridge deals with ServerMessage which has the File on it already
(10:21:17 AM) jmesnil: clebert: ah, ok
(10:21:26 AM) clebert: jmesnil: I will double check it though

[timfox]

Can you add some tests to make make sure this works with core bridges too?

[timfox]

Did you add the tests to make sure this works with core bridges too?

[clebert.suconic]

org.jboss.messaging.tests.integration.cluster.bridge.BridgeTest already test for LargeMessages.

Also, LargeMessage on core bridge is treated on the server's side using ServerMessage. Hence it wasn't needed to change anything.