-
1. Re: using hornetq as a log server
timfox Jun 5, 2010 5:46 AM (in response to augustsimonelli)I don't see why you can't use HornetQ in this way, although I've never seen it before.
Tell us how it goes
-
2. Re: using hornetq as a log server
leosbitto Jun 5, 2010 6:57 PM (in response to augustsimonelli)1 of 1 people found this helpfulI think that studying Log4J's JDBCAppender should give you the idea how to implement JMSAppender which would send the data to be logged to any JMS destination. Be prepared that it is going to be much slower than logging to the local file, though. Anyway, you could configre Log4J to send only some selected messages to your new JMSAppender, so the lower speed might be acceptable...
-
3. Re: using hornetq as a log server
augustsimonelli Jun 6, 2010 1:54 AM (in response to leosbitto)Leos -
Good idea ... i'll check out http://logging.apache.org/log4j/1.2/apidocs/org/apache/log4j/net/JMSAppender.html and see what i learn.
As far as speed, i was hoping this could be faster ... because (naiive sysadmin hat goes on) if i use the async appender to handle the JMSappender output i could batch it there and then send on to hornetq. That way we are quickly done as the async appender has kicked in. Then i thought JMS would be faster (ie geting the message and sending to DB) than syslog (in my case rsyslog or syslog-ng) writing to a file. But maybe not. And perhaps the java devs would rather a good grep and awk fest than having their logs dumped in a DB.
Still, it seems a good challenge; i think it may not be as "revolutionary" as i may have been thinking :-)
I'll report how i go!
August
-
4. Re: using hornetq as a log server
timfox Jun 6, 2010 5:07 AM (in response to augustsimonelli)1 of 1 people found this helpfulHornetQ internally can write to it's files much faster than you can write to a local log file, since we do all sorts of tricks to optimise this, use AIO etc.
If you're consuming from a queue and then logging to a file, or inserting in a DB, that's going to be the limiting factor, not HornetQ.
Make sure you send and consume messages async for best perf.
-
5. Re: using hornetq as a log server
augustsimonelli Jun 6, 2010 5:44 PM (in response to timfox)thanks everyone! i can see that this is more an interesting project for me to learn about hornetq, jms and log4j - and may not be a huge "improvement" over my current set up. but as a nice bonding exercise between dev and ops i'm gonna go for it.
thanks for the details and i'll report back any interesting results!
love the way i get such quick and useful AND friendly responses form y'all ... this is what it's all about!
- August