-
1. Re: Journal compaction
timfox Mar 23, 2010 3:20 AM (in response to kreide)No-one is going to be able to help you unless you actually post your client code. We are not psychic
Probably you are not acknowledging your messages.
-
2. Re: Journal compaction
kreide Mar 23, 2010 12:13 PM (in response to timfox)Sure, thanks for responding. My code is very similar to an example I found in the docs, "8.2. A simple example of using Core", but I did add the acknowledge() call which (surprisingly) was not part of that example. I also send the messages in a transaction to improve performance (I found it was very slow sending the messages one-by-one). Here is my code:
ClientSessionFactory sessionFactory = ...
while (true) {
ClientSession session = sessionFactory.createTransactedSession();
ClientProducer producer = session.createProducer("bar");
for (int i = 0; i < 1000; i++) {
ClientMessage message = session.createMessage(true);
message.getBodyBuffer().writeString("Hello");
producer.send(message);
}
session.commit();
session.close();
session = sessionFactory.createSession();
session.start();
ClientConsumer consumer = session.createConsumer("bar");
for (int i = 0; i < 1000; i++) {
ClientMessage msgReceived = consumer.receive();
msgReceived.acknowledge();
}
session.close();
}
My queue is defined like this:
<queues>
<queue name="bar"><address>bar</address><durable>true</durable></queue></queues>I hope something obvious jumps out. Thanks!
-
3. Re: Journal compaction
timfox Mar 23, 2010 1:21 PM (in response to kreide)What happens if you use a non transacted session?
-
4. Re: Journal compaction
kreide Mar 23, 2010 2:40 PM (in response to timfox)The same thing happens actually, but sending the messages is on the order of 100 times slower. It is somewhat interesting that despite not using transactions for the receiving part that is still very fast. Without transactions my send loop (1000 messages) takes 3-5 seconds, while the receive loop takes only 10 milliseconds. With transactions in the send loop it takes on the order of 20 milliseconds. Using transactions in the receive loop does not seem to impact performance significantly. Perhaps this is a clue that the acknowledge is not working properly?
-
5. Re: Journal compaction
timfox Mar 23, 2010 2:55 PM (in response to kreide)Kristian Eide wrote:
The same thing happens actually, but sending the messages is on the order of 100 times slower. It is somewhat interesting that despite not using transactions for the receiving part that is still very fast. Without transactions my send loop (1000 messages) takes 3-5 seconds, while the receive loop takes only 10 milliseconds. With transactions in the send loop it takes on the order of 20 milliseconds. Using transactions in the receive loop does not seem to impact performance significantly. Perhaps this is a clue that the acknowledge is not working properly?
That's to be expected.
When sending non transacted, you're probably sending persistent messages, and you're probably sending them blocking (see user manual), so you'll have a) a full network round trip + b) each message will be persisted individually and the response won't be sent back until the message has been synced to disk.
An average disk can do around 100-200 syncs per sec. Network round trip time is dependent on your network, but latencies are often around the order of 0.1 ms. Both of these are limits of the hardware.
When you send transacted, it doesn't need a round trip or a sync per message, only one for the commit.
-
6. Re: Journal compaction
timfox Mar 23, 2010 2:56 PM (in response to timfox)BTW this has been discussed many times on this forum. It might be worth a search.
-
7. Re: Journal compaction
timfox Mar 23, 2010 3:01 PM (in response to timfox)Now.. looking at your code.
I don't understand what you're doing here.
You send all your messages before you consume any of them, so clearly if you set policy to BLOCK, without consuming any, the producer is going to block.
-
8. Re: Journal compaction
kreide Mar 23, 2010 3:10 PM (in response to timfox)I send a batch of 1000 messages, then I receive (consume) a batch of 1000 and acknowledge them as well. I would expect that at this point the server would not retain those 1000 messages since they have already been acknowledged. Is this incorrect? Is there anything more I need to do to get the server to "forget" about these messages and reclaim the disk space the next time it does compaction? I guess this is essentially what I am asking.
-
9. Re: Journal compaction
kreide Mar 23, 2010 3:19 PM (in response to kreide)I should add that I am able to send and receive over 300.000 messages before the server blocks and I can see 15 journal files on the server.
-
10. Re: Journal compaction
timfox Mar 23, 2010 3:20 PM (in response to kreide)If the code you pasted is not the actual code you are using to demonstrate the issue, you need to post an example that does demonstrate the issue, otherwise we can't tell what you're doing.
-
11. Re: Journal compaction
kreide Mar 23, 2010 3:37 PM (in response to timfox)The code I posted is in fact exactly the code I am using. Are you saying the same code does not produce a similar issue for you? Could there be something in my server configuration that needs to be changed? I am basically using the configuration file that came with HornetQ 2.0 GA; the only changes I made in addition to defining the queue are:
<max-size-bytes>302428800</max-size-bytes>
<address-full-policy>BLOCK</address-full-policy>
Basically I want to limit the resources used by the server to make sure consumed messages are being cleaned up properly, which is what it not happening. Can you tell me how I can get the server to compact the journal files? What needs to happen for that to occur?
Thanks!
-
12. Re: Journal compaction
timfox Mar 23, 2010 3:41 PM (in response to kreide)Ok sorry.
I'll try your test program tomorrow and try and figure out what is going on.
-
13. Re: Journal compaction
kreide Mar 23, 2010 3:44 PM (in response to timfox)Thanks. I also read in the doc that async acknowledgement of messages is the default, so that explains why this part is fast even without transactions. I am also using the NIO journal if that makes a difference.
-
14. Re: Journal compaction
mcleanl Mar 23, 2010 3:54 PM (in response to kreide)Sorry to butt-in however I noticed a difference between the example and your actual code. In the example I didn't see any session.commit(); or session.close(); on the producer nor a second session being created. I don't know if that was a typo in the docs or intended.
i.e. instead of....
ClientSessionFactory sessionFactory = ...
while (true) {
ClientSession session = sessionFactory.createTransactedSession();
ClientProducer producer = session.createProducer("bar");
for (int i = 0; i < 1000; i++) {
ClientMessage message = session.createMessage(true);
message.getBodyBuffer().writeString("Hello");
producer.send(message);
}
session.commit();
session.close();
session = sessionFactory.createSession();
session.start();
ClientConsumer consumer = session.createConsumer("bar");
for (int i = 0; i < 1000; i++) {
ClientMessage msgReceived = consumer.receive();
msgReceived.acknowledge();
}
session.close();
}
have you tried with the ones session like...
ClientSessionFactory sessionFactory = ...
while (true) {
ClientSession session = sessionFactory.createTransactedSession();
ClientProducer producer = session.createProducer("bar");
for (int i = 0; i < 1000; i++) {
ClientMessage message = session.createMessage(true);
message.getBodyBuffer().writeString("Hello");
producer.send(message);
}
//session.commit();
//session.close();
//session = sessionFactory.createSession();
session.start();
ClientConsumer consumer = session.createConsumer("bar");
for (int i = 0; i < 1000; i++) {
ClientMessage msgReceived = consumer.receive();
msgReceived.acknowledge();
}
session.close();
}
or even changing
ClientSession session = sessionFactory.createTransactedSession();
to
ClientSession session = sessionFactory.createSession();
and using the one session for the entire transaction.