5 Replies Latest reply on Jun 6, 2010 5:44 PM by August Simonelli

    using hornetq as a log server

    August Simonelli Newbie

      Hi All,

       

      Disclaimer: I'm a sysadmin and not a developer. So i'm kinda slow with this stuff :-)

       

      I have tomcat logging nicely with log4j. Our developers also use log4j. Right now, the logging goes to a file for quick review and syslog for long term storage. This is nice, but i thought it'd be neat to try and log to hornetq. Well, actually, this (http://www.ibm.com/developerworks/websphere/library/techarticles/0207_barcia/barcia.html) got me thinking that way.

       

      So, i'm embarking on this little journey and thought i'd see if anyone else has forged ahead and had some pointers etc? Or if anyone else was interested in sharing recipes/ideas/failures/success/etc!

       

      August

        • 1. Re: using hornetq as a log server
          Tim Fox Master

          I don't see why you can't use HornetQ in this way, although I've never seen it before.

           

          Tell us how it goes

          • 2. Re: using hornetq as a log server
            Leos Bitto Novice

            I think that studying Log4J's JDBCAppender should give you the idea how to implement JMSAppender which would send the data to be logged to any JMS destination. Be prepared that it is going to be much slower than logging to the local file, though. Anyway, you could configre Log4J to send only some selected messages to your new JMSAppender, so the lower speed might be acceptable...

            1 of 1 people found this helpful
            • 3. Re: using hornetq as a log server
              August Simonelli Newbie

              Leos -

               

              Good idea ... i'll check out http://logging.apache.org/log4j/1.2/apidocs/org/apache/log4j/net/JMSAppender.html and see what i learn.

               

              As far as speed, i was hoping this could be faster ... because (naiive sysadmin hat goes on) if i use the async appender to handle the JMSappender output i could batch it there and then send on to hornetq. That way we are quickly done as the async appender has kicked in. Then i thought JMS would be faster (ie geting the message and sending to DB) than syslog (in my case rsyslog or syslog-ng) writing to a file. But maybe not. And perhaps the java devs would rather a good grep and awk fest than having their logs dumped in a DB.

               

              Still, it seems a good challenge; i think it may not be as "revolutionary" as i may have been thinking :-)

               

              I'll report how i go!

               

              August

              • 4. Re: using hornetq as a log server
                Tim Fox Master

                HornetQ internally can write to it's files much faster than you can write to a local log file, since we do all sorts of tricks to optimise this, use AIO etc.

                 

                If you're consuming from a queue and then logging to a file, or inserting in a DB, that's going to be the limiting factor, not HornetQ.

                 

                Make sure you send and consume messages async for best perf.

                1 of 1 people found this helpful
                • 5. Re: using hornetq as a log server
                  August Simonelli Newbie

                  thanks everyone! i can see that this is more an interesting project for me to learn about hornetq, jms and log4j - and may not be a huge "improvement" over my current set up. but as a nice bonding exercise between dev and ops i'm gonna go for it.

                   

                  thanks for the details and i'll report back any interesting results!

                   

                  love the way i get such quick and useful AND friendly responses form y'all ... this is what it's all about!

                   

                  - August