-
1. Re: Paging file of topic messages is not removed after subscribe to the corresponding message.
clebert.suconic Jun 27, 2011 11:09 AM (in response to miyasama)Are you sure you don't have any hidden subscription?
-
2. Re: Paging file of topic messages is not removed after subscribe to the corresponding message.
clebert.suconic Jun 27, 2011 4:14 PM (in response to clebert.suconic)If you could provide a test please, with steps to reproduce it.
-
3. Re: Paging file of topic messages is not removed after subscribe to the corresponding message.
clebert.suconic Jun 27, 2011 7:37 PM (in response to miyasama)Here are steps I have done:
1. Publish some messages to topic A until new message is paging out.
(Confirm that a paging file is created and some messages stored in the file.)
What subscriptions did you have prior to this step?
3. Subscribe to all the messages from the topic(topic A). (including the paged out messages.)
Did you create a new subscription, or that's the same subscriptions you had before?
-
4. Re: Paging file of topic messages is not removed after subscribe to the corresponding message.
clebert.suconic Jun 27, 2011 7:38 PM (in response to clebert.suconic)If you provide a test, these questions will be answered anyway. Can you provide one?
-
5. Re: Paging file of topic messages is not removed after subscribe to the corresponding message.
miyasama Jun 27, 2011 10:44 PM (in response to clebert.suconic)Hi, Clebert.
Thank you for your comment.
Are you sure you don't have any hidden subscription?
No, I didn't have any hidden subscription.
1. Publish some messages to topic A until new message is paging out.
(Confirm that a paging file is created and some messages stored in the file.)
What subscriptions did you have prior to this step?
I created two subscription for topic A prior to this step.
3. Subscribe to all the messages from the topic(topic A). (including the paged out messages.)
Did you create a new subscription, or that's the same subscriptions you had before?
It's the latter. I was using the same subscriptions.
If you provide a test, these questions will be answered anyway. Can you provide one?
I was testing manually. So I try to make the test. if I create the test, how can I provide it to you?
-
6. Re: Paging file of topic messages is not removed after subscribe to the corresponding message.
clebert.suconic Jun 28, 2011 12:59 AM (in response to miyasama)I was testing manually. So I try to make the test. if I create the test, how can I provide it to you?
If you switch the forum using advanced editor, you can attach any files.
-
7. Re: Paging file of topic messages is not removed after subscribe to the corresponding message.
jeremystone Jun 30, 2011 10:02 AM (in response to clebert.suconic)Not wishing to hijack the topic - but we have seen this behaviour also. A simple test containing a producer and a consumer class is attached. The require a locally running HornetQ server with a topic A configured for paging.
After starting the producer and the consumer with the SLEEP_BETWEEN_MESSAGES_MILLIS fields set to 5 and 500 respectively, paging starts quickly (provided that consumer window size is 0 and max-size-bytes is not too large e.g. 1MB).
We see the following behaviour:
- If the consumer is stopped cleanly (by hitting enter in the console window) the page files are cleaned up (all bar a single empty page). However on repeating this, page files are not deleted the second time the consumer is stopped cleanly.
- If the consumer is stopped by killing it, page files continue to be added. After about a couple of minutes page files stop being created (the connection-ttl comes into play, I guess, an the server can infer that the consumer has gone away) and all page files (bar one) are deleted as above.
- If after (1) the producer and consumer are restarted but with SLEEP_BETWEEN_MESSAGES_MILLIS switched - i.e. a fast consumer, slow producer, then page files continue to be created. (Although none would be had the paging not started previously.)
Hope you can shed some light on this, Clebert - is it related to http://community.jboss.org/thread/163371 ?
-
hornetq-test.zip 5.7 KB
-
8. Re: Paging file of topic messages is not removed after subscribe to the corresponding message.
jeremystone Jun 30, 2011 10:37 AM (in response to jeremystone)BTW - this is HQ 2.2.5-Final, running on Windows
-
9. Re: Paging file of topic messages is not removed after subscribe to the corresponding message.
clebert.suconic Jun 30, 2011 11:43 AM (in response to jeremystone)@Jremy: It doesn't seem related to paging at all... if you have a consumer (I assume you have a subscription), messages will build up accordingly to how you consume it and ack it.
How you wanted it differently.. if you are not consuming.. you are not consuming and messages will build up...
if you kill the client, the client will be removed after ttl,and if you have a temp queue (or non durable subscription) the messages should be gone when TTL kicks in.
-
10. Re: Paging file of topic messages is not removed after subscribe to the corresponding message.
jeremystone Jun 30, 2011 12:20 PM (in response to clebert.suconic)Ok. Yes, when there are no subscriptions to the topic the messages do not build up (provided the ttl has expired if I kill the consumer). And yes messages will build up if I consume more slowly than messages are produced. So far so good.
However my points are that:
(a) page files stick around after the only (non-durable) subscriber is closed down for a second or subsequent time (my point 1, above) (and even if the producer is closed down also).
and
(b) if I start both the producer and consumer when it is in this state (i.e. has been paging) with the consumer configured to be much faster than the producer, then new page files still get written to the paging directory. I.e. once it has started paging, it seems to continue to want to page, even though it doesn't need to in this case.
-
11. Re: Paging file of topic messages is not removed after subscribe to the corresponding message.
clebert.suconic Jun 30, 2011 12:51 PM (in response to jeremystone)The system will page as long as there's till a subscription (or internal queue) live and messages being sent.
If you don't want the messages to build up, you can just configure blocking and provide enough memory and configure it accordingly.
-
12. Re: Paging file of topic messages is not removed after subscribe to the corresponding message.
jeremystone Jul 1, 2011 4:55 AM (in response to clebert.suconic)The system will page as long as there's till a subscription (or internal queue) live and messages being sent.
But the address-full-policy should come into play only when max-size-bytes is exceeded for the address.
We have a situation where we have one producer and one consumer capable of consuming messages from a topic as quickly as they are produced.
I would not expect paging to occur at all in this scenario and normally it does not. However...
...if we force it into a situation where it starts to page (by running a slow consumer), then closedown this consumer (so the old subscription/internal queue should go away) and start a fast consumer it continues to page and does not delete old page files.
This does not seem right. Although this is a topic, the symptoms are similar to those referred to in http://community.jboss.org/thread/163371.
If you don't want the messages to build up, you can just configure blocking and provide enough memory and configure it accordingly.
Unfortunately, configuring blocking is not really an option for us - we are using paging to act as a buffer between two processes (the first of which must complete in a timely manner) to cater for peak loads. Our problem is that paging never stops when the load reduces which eventually causes other performance issues.
The test classes attached above should enable you to readily reproduce this behaviour.
-
13. Re: Paging file of topic messages is not removed after subscribe to the corresponding message.
clebert.suconic Jul 1, 2011 11:04 AM (in response to jeremystone)If you have a slow consumer, disable caching... otherwise there will always be a message being held by the slow consumer.
-
14. Re: Paging file of topic messages is not removed after subscribe to the corresponding message.
jeremystone Jul 1, 2011 11:13 AM (in response to clebert.suconic)But the slow consumer has been shut down.
Which cache setting do you mean?