-
1. Re: JBM2 Beta5 with JBAS5.1 - Memory leak SessionContinuatio
clebert.suconic Aug 20, 2009 1:52 PM (in response to simon_temple)Did you make any changes on the windowSize?
You should have up to your WindowSize messages in memory. (We control that thorugh flow control).
http://www.jboss.org/file-access/default/members/jbossmessaging/freezone/docs/usermanual-2.0.0.beta4/html/flow-control.html#flow-control.core.api
If you disable flow-control.. you will have messages piling up on the memory.
if that's not the case, I would need to take a look at your test. -
2. Re: JBM2 Beta5 with JBAS5.1 - Memory leak SessionContinuatio
simon_temple Aug 21, 2009 5:37 AM (in response to simon_temple)The loop in ClientProducerImpl seems to be responsible:
while (!lastChunk) { byte[] bytesRead = new byte[minLargeMessageSize]; int numberOfBytesRead; try { numberOfBytesRead = input.read(bytesRead); } catch (IOException e) { throw new MessagingException(MessagingException.LARGE_MESSAGE_ERROR_BODY, "Error reading the LargeMessageBody", e); } if (numberOfBytesRead < 0) { numberOfBytesRead = 0; lastChunk = true; } final SessionSendContinuationMessage chunk = new SessionSendContinuationMessage(bytesRead, numberOfBytesRead, !lastChunk, lastChunk && sendBlocking); if (sendBlocking && lastChunk) { // When sending it blocking, only the last chunk will be blocking. channel.sendBlocking(chunk); } else { channel.send(chunk); } }
By configuring my connection factory I have minLargeMessageSize=131072 (128K)
My input stream is a buffered input stream returning 1024 bytes at a time.
The stream source is 512K in length.
Therefore, each time around the loop I will create a new 128K byte[] with 1K of data read into it. I then pass a reference to the array to new SessionSendContinuationMessage.
With 512 loop interations required to read and send the stream content I consume 64M of heap space in byte arrays.
wdyt? -
3. Re: JBM2 Beta5 with JBAS5.1 - Memory leak SessionContinuatio
clebert.suconic Aug 21, 2009 10:11 AM (in response to simon_temple)What is your windowSize?
I will make a few tests.
With LargeMessage, you should have a bunch of pending packets waiting to be delivered on the NettyQueue, but you shouldn't have more than what's configured on the windowSize. Otherwise you would get out of memory easily.
I would need to look at your test to know what' s happening better.
Perhaps you're opening multiple producers? -
4. Re: JBM2 Beta5 with JBAS5.1 - Memory leak SessionContinuatio
simon_temple Aug 21, 2009 10:26 AM (in response to simon_temple)From my jbm-jms.xml
<connection-factory name="SocketConnectionFactory"> <connector-ref connector-name="netty"/> <entries> <entry name="ConnectionFactory"/> <entry name="XAConnectionFactory"/> </entries> <!-- 128K chunk size for large messages --> <min-large-message-size>131072</min-large-message-size> <!-- Set the consumer window size to 512K to limit the size of the buffer on the client side --> <consumer-window-size>524288</consumer-window-size> </connection-factory>
I'm creating a bytes message, adding my buffered stream to it:
bytesMessage.setObjectProperty( "JMS_JBM_InputStream", pin );
then calling:
producer.send( bytesMessage, deliveryMode, priority, timeToLive );
Many Thanks -
5. Re: JBM2 Beta5 with JBAS5.1 - Memory leak SessionContinuatio
clebert.suconic Aug 21, 2009 10:32 AM (in response to simon_temple)A single producer? Or you're opening several Producers? How many threads doing this.. just one?
-
6. Re: JBM2 Beta5 with JBAS5.1 - Memory leak SessionContinuatio
simon_temple Aug 21, 2009 10:35 AM (in response to simon_temple)Yes a single producer on one thread from a junit test case.
-
7. Re: JBM2 Beta5 with JBAS5.1 - Memory leak SessionContinuatio
clebert.suconic Aug 21, 2009 10:37 AM (in response to simon_temple)Can you send me a sample test showing this happening?
clebert at redhat dot com
or clebert dot suconic at jboss dot com -
8. Re: JBM2 Beta5 with JBAS5.1 - Memory leak SessionContinuatio
timfox Aug 22, 2009 8:07 AM (in response to simon_temple)Simon- you're absolutely right.
It's clear from looking at the code that it's highly inefficient to allocate a large buffer when perhaps as little as one byte is read from the stream.
I have refactored the code, so this is done more sensibly!
The fix will be in the next release (on Monday)