JMS Bridge doesn't work with large messages
rocklu Jul 21, 2014 8:22 AMWe seem to hit an issue with large messages on JMS Bridges. A non-functional requirement suggests to transfer Large Messages with a size of up to 10GB.
1. Testing the Large Message Example under ../hornetq-2.3.20.Final/examples/jms and the JMS Bridge Example under ../hornetq-2.3.20.Final/examples/javaee.
Both work fine with JBoss EAP 6.3.0.Beta. For this, I had to modify the Javaee POM (see attach). We are using the HornetQ Download for 2.3.20.Final for this test setup.
In order to show the issue, we have extended the existing HornetQ Examples using our exact scenario.
2. the Large Message Example (with Core Bridge) with Files up to 10 GB are working fine.
3. the JMS Bridge Example is modified to use Large Messages and it works with files of 1 GB successfully.
If I send a file with the size of 2 GB, I get an error:
Creating a file to send of size 2147483648 bytes. This may take a little while... If this is too big for your disk you can easily change the FILE_SIZE in the example.
File created.
Sending the huge message.
Large Message sent
Receiving message.
10:37:39,857 ERROR [stderr] (pool-3-thread-2) Exception in thread "pool-3-thread-2" java.lang.OutOfMemoryError: Requested array size exceeds VM limit
10:37:39,859 ERROR [stderr] (pool-3-thread-2) at org.jboss.netty.buffer.HeapChannelBuffer.<init>(HeapChannelBuffer.java:42)
10:37:39,860 ERROR [stderr] (pool-3-thread-2) at org.jboss.netty.buffer.BigEndianHeapChannelBuffer.<init>(BigEndianHeapChannelBuffer.java:34)
10:37:39,861 ERROR [stderr] (pool-3-thread-2) at org.jboss.netty.buffer.ChannelBuffers.buffer(ChannelBuffers.java:134)
10:37:39,862 ERROR [stderr] (pool-3-thread-2) at org.jboss.netty.buffer.HeapChannelBufferFactory.getBuffer(HeapChannelBufferFactory.java:68)
10:37:39,863 ERROR [stderr] (pool-3-thread-2) at org.jboss.netty.buffer.DynamicChannelBuffer.<init>(DynamicChannelBuffer.java:58)
10:37:39,864 ERROR [stderr] (pool-3-thread-2) at org.jboss.netty.buffer.DynamicChannelBuffer.<init>(DynamicChannelBuffer.java:43)
10:37:39,865 ERROR [stderr] (pool-3-thread-2) at org.jboss.netty.buffer.ChannelBuffers.dynamicBuffer(ChannelBuffers.java:207)
10:37:39,866 ERROR [stderr] (pool-3-thread-2) at org.jboss.netty.buffer.ChannelBuffers.dynamicBuffer(ChannelBuffers.java:197)
10:37:39,867 ERROR [stderr] (pool-3-thread-2) at org.hornetq.api.core.HornetQBuffers.dynamicBuffer(HornetQBuffers.java:36)
10:37:39,868 ERROR [stderr] (pool-3-thread-2) at org.hornetq.core.message.impl.MessageImpl.createBody(MessageImpl.java:961)
10:37:39,869 ERROR [stderr] (pool-3-thread-2) at org.hornetq.core.client.impl.ClientLargeMessageImpl.checkBuffer(ClientLargeMessageImpl.java:183)
10:37:39,871 ERROR [stderr] (pool-3-thread-2) at org.hornetq.core.client.impl.ClientLargeMessageImpl.getBodyBuffer(ClientLargeMessageImpl.java:99)
10:37:39,872 ERROR [stderr] (pool-3-thread-2) at org.hornetq.jms.client.HornetQMessage.checkBuffer(HornetQMessage.java:888)
10:37:39,872 ERROR [stderr] (pool-3-thread-2) at org.hornetq.jms.bridge.impl.JMSBridgeImpl$SourceReceiver.run(JMSBridgeImpl.java:1770)
10:37:39,873 ERROR [stderr] (pool-3-thread-2) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
10:37:39,873 ERROR [stderr] (pool-3-thread-2) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
10:37:39,873 ERROR [stderr] (pool-3-thread-2) at java.lang.Thread.run(Thread.java:744)
10:39:39,468 INFO [org.jboss.as.naming] (Remoting "celsius-h710" task-4) JBAS011806: Channel-Endbenachrichtigung erhalten, schließe Channel Channel ID 6af9fc0e (inbound) of Remoting connection 25869a4b to /127.0.0.1:55506
Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 155.502 sec <<< FAILURE!
Results :
Tests in error:
runExample(org.hornetq.javaee.examples.JMSBridgeRunnerTest)
Tests run: 1, Failures: 0, Errors: 1, Skipped: 0
[ERROR] There are test failures.
Our expectation:
The "JMS Bridge Example" is working until the large message size hits the available memory.
The "JMS Bridge Example" shows an unexpected OutOfMemory Exception. The issue is not around the 2GB of size.
Why does the JMS Bridge try to read the content of the Stream into the memory? It should simply stream the file content to the disk.
We have followed the HornetQ documenation and need help to understand the issue.
Can you explain the OutOfMemory exception?
What can we change to successfully send large messages where the file size exceeds the memory size?
-
JMSBridgeExample.java 7.9 KB
-
LogOutput.log.zip 4.9 KB
-
javaee_pom.xml 5.9 KB