-
1. Re: Limiting Physical Disk Space Usage
saltnlight5 May 12, 2010 11:39 PM (in response to tootedom)Have you tried the User Manual, section 19.2.1.3. Blocking producer window based flow control?
-
2. Re: Limiting Physical Disk Space Usage
tootedom May 13, 2010 5:33 AM (in response to saltnlight5)Yup, have looked at the manual and Producer Flow Control, and this seems to indicate that is in based on blocking the producer based on Memory Consumption, not based on Disk Space Usage:
"
For example, if I have a JMS queue called "myqueue", I could set the maximum memory size to 10MiB, and the the server will control the number of credits sent to any producers which are sending any messages to myqueue such that the total messages in the queue never exceeds 10MiB.
When the address gets full, producers will block on the client side until more space frees up on the address, i.e. until messages are consumed from the queue thus freeing up space for more messages to be sent.
"
It says it is an alternative to Paging, but doesn't indicate if both can be used. I.e. If memory usage for an address grows above X, then page. Then is the paged to store has reached X size, block.
Does Producer Flow Control act just based on JVM Memory Usage, or is it based on a combination of JVM Mem + Disk Storge Size. Docs, seem to tend towards JVM, but it's not fully clear.
cheers,
/dom
-
3. Re: Limiting Physical Disk Space Usage
timfox May 13, 2010 5:41 AM (in response to tootedom)If you limit the amount of memory used, that will limit the amount of disk space used too.
-
4. Re: Limiting Physical Disk Space Usage
tootedom May 13, 2010 5:51 AM (in response to timfox)Cheers Tim,
Thank you very much for clearing that up. I'll give it all a go.
I'm trying to get HornetQ up and running next to ActiveMQ, rather than have all my eggs in the ActiveMQ basket.
/dom
-
5. Re: Limiting Physical Disk Space Usage
tootedom May 15, 2010 9:00 AM (in response to tootedom)Hi again,
Sorry to bother you again with this. I have had a play around with various broker configurations, but I'm still unable to figure out how to:
- Page Messages to disk, when X amount of memory (jvm) is used and then
- Block (or send and exception to) the producer when X amount of physical disk space has been consumed
I'm able to:
- Page messages to disk (The amount of disk space consumed, grows and grows as messages are produced)
- Block the producer when X memory (jvm) has been consumed
But I'm trying to:
- Page messages to disk, and limit the amount of Disk Space Consumed when paging occurs. When the Disk Space limit is reached, the producer can be either blocked or sent a message say there is no space left (configurable)
What am I missing/mis-reading? Is what I am trying to achieve possible currently?
The following shows the various configurations attempted, all with the same producer code and a durable queue:
(HornetQ Server version 2.0.0.GA)
ClientProducer producer = session.createProducer("VirtualTopic.CWWTF");
session.start();ClientMessage message = session.createMessage(true);
message.putStringProperty("myprop", "Hello sent at " + new Date());
producer.send(message);
<queues>
<queue name="VirtualTopic.${SERVER_DATACENTER:CWWTF}">
<address>VirtualTopic.${SERVER_DATACENTER:CWWTF}</address>
</queue>
</queues>Paging ( but disk space growing - not limited to 10Mb):
<address-settings>
<!--default for catch all-->
<address-setting match="VirtualTopic.CWWTF">
<max-size-bytes>1048576</max-size-bytes>
<page-size-bytes>51200</page-size-bytes>
<address-full-policy>PAGE</address-full-policy>
</address-setting>
<address-setting match="#">
<max-size-bytes>1048576</max-size-bytes>
<address-full-policy>BLOCK</address-full-policy>
</address-setting>
</address-settings>
dominic-tootells-macbook-pro-2:data dominict$ du -sh *
2.0M bindings
20M journal
0B large-messages
48M pagingdominic-tootells-macbook-pro-2:data dominict$ ls -lh paging/0040df17-5ffd-11df-808a-005056c00008/
total 98328
-rw-r--r-- 1 dominict staff 50K 15 May 09:36 000000001.page
-rw-r--r-- 1 dominict staff 50K 15 May 09:36 000000002.page
....Blocking Producer (but no paging occuring)
<address-settings>
<address-setting match="VirtualTopic.CWWTF">
<max-size-bytes>1048576</max-size-bytes>
<address-full-policy>BLOCK</address-full-policy>
</address-setting>
<address-setting match="#">
<max-size-bytes>1048576</max-size-bytes>
<page-size-bytes>51200</page-size-bytes>
<address-full-policy>PAGE</address-full-policy>
</address-setting>
</address-settings>
dominic-tootells-macbook-pro-2:data dominict$ du -sh *
2.0M bindings
20M journal
0B large-messagesBlock Producer, (but Set Max Size larger than broker memory - Broker Crashes, due to low memory: 256Mb broker mem, 400mb max size):
<address-settings>
<address-setting match="VirtualTopic.CWWTF">
<max-size-bytes>419430400</max-size-bytes>
<address-full-policy>BLOCK</address-full-policy>
</address-setting>
Heap
PSYoungGen total 62912K, used 37358K [0x0000000115fc0000, 0x000000011b510000, 0x000000011b510000)
eden space 37376K, 99% used [0x0000000115fc0000,0x000000011843b8d0,0x0000000118440000)
from space 25536K, 0% used [0x0000000119c20000,0x0000000119c20000,0x000000011b510000)
to space 24448K, 0% used [0x0000000118440000,0x0000000118440000,0x0000000119c20000)
PSOldGen total 174784K, used 174783K [0x000000010b510000, 0x0000000115fc0000, 0x0000000115fc0000)
object space 174784K, 99% used [0x000000010b510000,0x0000000115fbfff8,0x0000000115fc0000)
PSPermGen total 21504K, used 21075K [0x0000000106110000, 0x0000000107610000, 0x000000010b510000)
object space 21504K, 98% used [0x0000000106110000,0x00000001075a4f28,0x0000000107610000)
On trying to start the broker (after hang).
DEPLOYMENTS IN ERROR:
Deployment "JMSServerManager" is in error due to: java.lang.OutOfMemoryError: Java heap spaceat org.jboss.kernel.plugins.deployment.AbstractKernelDeployer.internalValidate(AbstractKernelDeployer.java:278)
at org.jboss.kernel.plugins.deployment.AbstractKernelDeployer.validate(AbstractKernelDeployer.java:174)Attempt with a Wild Card Address (paging disk space consumed just grows):
<address-settings>
<address-setting match="VirtualTopic.#">
<max-size-bytes>10485760</max-size-bytes>
<page-size-bytes>1048576</page-size-bytes>
<address-full-policy>PAGE</address-full-policy>
</address-setting>
<address-settings>dominic-tootells-macbook-pro-2:data dominict$ du -sh *
2.0M bindings
20M journal
0B large-messages
208M pagingdominic-tootells-macbook-pro-2:data dominict$ ls -lh paging/019b5f3d-601a-11df-b330-005056c00008/
total 433232-rw-r--r-- 1 dominict staff 1.0M 15 May 13:04 000000001.page
....Thanks in advance, apologies if I'm just not reading something correctly in the docs.
/dom
- Page Messages to disk, when X amount of memory (jvm) is used and then
-
6. Re: Limiting Physical Disk Space Usage
timfox May 15, 2010 3:10 PM (in response to tootedom)Dominic Tootell wrote:
No, it's not currently supported.