-
1. Re: Hornetq poor throughput
gaohoward Apr 20, 2010 12:58 AM (in response to pnranjith)Can I ask how did you measure the throughput? Was it at the sending end or receiving end?
-
2. Re: Hornetq poor throughput
pnranjith Apr 20, 2010 1:10 AM (in response to gaohoward)Both at sending and recieving end it is giving almost same throughput. We measure it using jconsole and also we have a database in which the messege status are written. So we use it to query the details.
-
3. Re: Hornetq poor throughput
gaohoward Apr 20, 2010 1:39 AM (in response to pnranjith)Did you set any flow control parameters such as consumer-window-size or producer-window-size?
-
4. Re: Hornetq poor throughput
pnranjith Apr 20, 2010 1:54 AM (in response to gaohoward)No. I did not set any window size. not at producer side nor at consumer.
-
5. Re: Hornetq poor throughput
ataylor Apr 20, 2010 4:33 AM (in response to pnranjith)This is probably a config issue somewhere but without any context its hard for us to advise you. are you using default config, persistent messages, what size messages etc etc. post some config and some code and maybe we can help you.
-
6. Re: Hornetq poor throughput
timfox Apr 20, 2010 4:59 AM (in response to pnranjith)Wow, that question is almost impossible to answer since you provide almost zero information.
No config, no test program etc.
-
7. Re: Hornetq poor throughput
timfox Apr 20, 2010 5:00 AM (in response to timfox)What you need to do is create a simple test program we can run against both us and ActiveMQ, so we can see this difference.
-
8. Re: Hornetq poor throughput
timfox Apr 20, 2010 5:22 AM (in response to timfox)Please read this as well: http://community.jboss.org/wiki/Howtoreportabugissue
When posting it's vital you provide as much information as possible, and this includes a test case.
Otherwise it's a bit like walking into a doctor, saying your friend is sick, giving no information and expecting the doctor to diagnose the problem.
-
9. Re: Hornetq poor throughput
timfox Apr 20, 2010 5:26 AM (in response to timfox)"also we have a database in which the messege status"
Well, the db is going to be the weakest link for a start
-
10. Re: Hornetq poor throughput
pnranjith Apr 20, 2010 8:20 AM (in response to timfox)So, getting deeper with the debugging.
Here I measured the time taken and its around 30ms sometime:
This is the writer part:
objMessage is an instance of javax.jms.ObjectMessage
msg is an instance of message
long startTime = 0L ;
ObjectMessage objMessage;
try {
objMessage = session.createObjectMessage();
objMessage.setObject(msg);
startTime = System.currentTimeMillis() ;
producer.send(objMessage);
AppLogger.logger.info("Time in Writer.write "+(System.currentTimeMillis() - startTime));}
And I am creating the producer with the following code:
connection = connectionFactory.createConnection();
connection.start();
session = connection.createSession(false, Session.AUTO_ACKNOWLEDGE);Queue queue = createQueue(subject) ;
producer = session.createProducer(queue);
producer.setDeliveryMode(DeliveryMode.PERSISTENT); -
11. Re: Hornetq poor throughput
timfox Apr 20, 2010 8:31 AM (in response to pnranjith)This is not a test program, it's just a code snippet.
We would need to see a working test program, that we can actually run and actually produces some result we can look at.
Also, you didn't say what versions of HornetQ or ActiveMQ you are using, nor did you provide your config.
Also, sending just one message means absolutely nothing!
You need to
a) send many thousands of messages, and average over a long time in order to minimise indeterminacy due to gc and thread scheduling
b) allow enough time for JIT to kick in
This has been discussed on several threads before. I'm getting a bit tired having to continually repeat myself on this (please search).
Also bear in mind that microbenchmarks using a single sender/consumers are rarely very interesting as they say nothing about how the system scales with many senders/consumers.
Also, did you make sure you have turned ActiveMQ syncing on?
http://activemq.apache.org/amq-message-store.html
Actually you didn't say either what ActiveMQ store you're using, so I'll assume it's AMQ message store.
Remember, that ActiveMQ "cheats" and does not sync to disk in it's out of the box config.
Set syncOnWrite to true
-
12. Re: Hornetq poor throughput
pnranjith Apr 20, 2010 9:09 AM (in response to timfox)But with a test program I wrote, it is working fine. That is why I didn't send the test program. The issue comes when I connect my already live application to the hornetq. One more thing is when I run the queue seperately in another system with 2GB RAM, it is giving me 200 messages per second. Same program connected with the hornetq running on the same machine with 1GB RAM is giving only 5-10 message per second. Both systems with activemq is working with the same throughput which is much better than with the hornet.
version I use is: hornetq-2.0.0.GA
amq version is : fuse amq 5.0.0.29
I am sending with 5 producer and 5 consumer with 250 million messages .
The results are taken after 6 minutes.
I have 5 queues which are created dynamically, viz. response queue, request queue, alertq, db queue and reprocess queue whose name implies its functionalities.
One more doubt is, Can I creae a cluster to improve the performance
Thanks a lot
-
13. Re: Hornetq poor throughput
timfox Apr 20, 2010 3:12 PM (in response to pnranjith)So you need to create a test program that does exhibit a problem. This will help you track down the issue, which is almost certainly a problem in your config, or in ActiveMQ config.
Like I said before, activemq doesn't sync to disk by default.
-
14. Re: Hornetq poor throughput
timfox Apr 28, 2010 5:00 AM (in response to timfox)A few points:
By default HornetQ will force a sync to the disk to ensure the data has really got there. (Other systems e.g. ActiveMQ ***DO NOT** sync by defaut - on other words, they cheat and you could lose data with them).
An average disk on a laptop/desktop can do maybe 100 to 200 syncs per sec. THIS IS A HARDWARE LIMIT OF THE DISK, it is not a limit of HornetQ! This would explain a 100 to 200 message rate.
If you really want persistent messaging you can either:
a) Buy an expensive SCSI disk with battery backed cache - you might be able to get a couple of thousand syncs per sec
b) Buy solid state disk (e.g. FusionIO)
c) Send many messages in a single transaction
d) Use async send acknowledgements (see user manual) and don't block on sends.
e) Don't use persistent messages
f) Probably some other stuff I can't think of right now
Like I say, pretty much all this is discussed in the user manual. Please take some time to read it.