6 Replies Latest reply on Mar 11, 2010 12:40 PM by edx

    Asynchronous logging?

    massios

      Dear ESBers,

       

      In our action pipelines we have at several "checkpoint" actions that perform logging at an oracle database.

       

      So far we are not in production so our the logging tables are rather small and the logging times are negligible.

       

      However we are considering making the logging actions asynchronous because of worries about future performance.

       

      One possibility I can think of for doing that is putting for logging purposes a message to a queue that is read by an MDB and let the MDB put the logging information into the database.

       

      Any comments? Has anyone tried something to improve logging performance?

       

      Nikos.

        • 1. Re: Asynchronous logging?
          massios

          Better yet, instead of using an MDB, since we are talking about an esb, sending a message to a logging service.

           

          Nikos.

          • 2. Re: Asynchronous logging?
            edx

            I have developed the appender that takes the messages from JBoss logging subsystem (log4j) and then writes them to database via configured pool of DB workers. If you are interested in such solution I can share it here.

             

            Edward

            • 3. Re: Asynchronous logging?
              h.wolffenbuttel

              Hi Edward,

               

              Please do share

               

              Regards,

               

              Hans

              • 4. Re: Asynchronous logging?
                edx

                Hello

                The attached tar gzipped file (DbLogger.tar.gz) is a complete Eclipse (JBossTools 3.1) project directory

                that you can extract somwhere and import to your projects workspace.

                 

                The project creates the Service Archive (.sar) that deploys two things:

                1. (dblogger-ds.xml) Data Source that handles the conenctions to the instance of logging database

                2. DbLoggingService as MBean in JBoss microcontainer.

                 

                The attached project configures Data Source to use mySQL database, other types of DB engines can be used (In the past I used PostgreSQL without problems).

                 

                The setup of the logging database is defined in files $ROOT/db-info/ directory,

                where there is a sql database init script that creates logentries table

                and there is also the project file dblogger.mwb for MySQL Workbench tool (DB model).

                 

                DbLoggingService is deployed as MBean for which the configuration is defined

                in the file $ROOT/src/META-INF/jboss-service.xml which contains the JNDI name

                definition that references the DataSource and also the dependency is defined for that datasource.

                 

                The DbLoggingService when started does also reconfiguration of JBoss log4j service

                by adding the reference to the new Appender - EdxLogAppender.

                 

                The configuration of the new appender is in the file $ROOT/src/META-INF/log4j.xml

                where three important startup parameters are set:

                Timeout - value in miliseconds for waiting on new message in an internal Vector<>.

                               After that time the worker thread validates the connection to database.

                DbWorkerNum - number of concurrent threads that do stroing log messages to database

                MsgQueueDepth - the size of the message queue (Vector<>) used as FIFO for

                                              feeding the workers.

                 

                The log4j.xml also defines the root of Category tree this appender is assigned.

                The project defines (thanks to JBossTools plugin) the final product the deployment archive

                DbLogger.sar that you can simply drop into JBossAS deploy directory.

                 

                In your ESB developments you can use the new logging facility by defining in your classes:

                 

                Logger log = Logger.getLogger( ctgry );

                ....

                log.info("message");

                 

                where ctgry has to be started with root category defined in log4j.xml file.

                 

                Enjoy my work !

                 

                regards,

                Edward

                • 5. Re: Asynchronous logging?
                  massios
                  • 6. Re: Asynchronous logging?
                    edx

                    Hello,

                    even in the std log4j you have the possibility of logging directly to database using JdbcAppender that is included in the base log4j package.

                    However the std. appender logs messages as text only (you have to provide the layout or the output message).

                    My appender logs messages into table and in separate columns the log level and source category is written.

                    This provides further possibility of intelligent analysing of the log data by searching for log entries of the particular importance (log level) and of the particular origin (category). Additionally it is deployed as the MBean service that you can access from any application from the JBoss runtime environment.

                    Regards,

                    Edward