MDC filter with subsystem logging module

[mantonec]

Hi,

regarding logging module, is it possible to add a filter on MDC parameters ?

In logging class i have something like this:

Logger logger = Logger.getLogger( LoggerMessageListener.class );

MDC.put( Constants.MESSAGE_TYPE, msg.getStringProperty( Constants.MESSAGE_TYPE ));

logger.log( level, msg.getText() );

Where Constants.MESSAGE_TYPE = "messageType" and msg.getStringProperty( Constants.MESSAGE_TYPE ) is filled at runtime with a text that identify my business activity (i.e. Operation1, Operation2 ...)

With log4j i was able to split all logs belongs Operation1 into Operation1.log, all logs belongs Operation2 into Operation2.log and so on.

Follows log4j configuration file:

<appender name="Operation1Appender" class="org.apache.log4j.DailyRollingFileAppender">
          <param name="File" value="<logs>/Operation1.log" />
...
          <filter class="com.mycompany.common.log.MDCMatchFilter">
                    <param name="KeyToMatch" value="messageType" />
                    <param name="ValueToMatch" value="Operation1" />
                    <param name="ExactMatch" value="true" />
          </filter>
</appender>

With Jboss AS7 instead i have, in my standalone.xml, something like this:

<periodic-rotating-file-handler name="Operation1" autoflush="true">
      <level name="DEBUG"/>
      <filter>
            <match pattern="Operation1"/>
            <not>
                <match pattern="Operation10|Operation11"/>
            </not>
      </filter>
      <formatter>
            ...
      </formatter>
      <file relative-to="jboss.server.log.dir" path="Operation1.log"/>
      ...
</periodic-rotating-file-handler>

This method of filtering is not feasible because filtering is not made on a field (messageType) but on the content of the entire log row and can cause unexpected behavior (for example with Operation1 and Operation11)

Moreover, introducing a new Operation (let me say Operation12) i'm constrained to change the configuration of other handlers (in this case Operation1)

So, what i want is to be able to define a filter on a specific MDC field, this grants that the filter is made only on the content of this unique field.

Thanks a lot,

Michele.

[nickarls]

One would think you could use a customer appender with filter implementation for that(?)

[mantonec]

Thanks for your response.

Please could you be more clear ?

Are you talking about to use log4j or create a custom handler in AS7 ? In such case where i can find the documentation in order to build a custom handler ?

[nickarls]

A custom logger (have a look at https://community.jboss.org/wiki/CustomLogHandlersOn701, hope it's not too obsolete).

But in your case you might get away by extending the org.jboss.logmanager.handlers.PeriodicRotatingFileHandler and overriding

isLoggable(LogRecord record) (or the method in the actual filter)

James will wake up soon and fill in the details, I'm going home ;-)

[mantonec]

Thank you very much.

With your information I was able to create the CustomPeriodicFileHandler as follows and deploy it as JBoss Module.

1. is it the correct way to do it ?
2. I haven't achieve the goal yet because i'm facing with the following issue:
   In logger category i have declared the package com.mycompany.logger and the handler Test.
   This solution works if I define always the MDC  messageType and configure the corresponding custom-handler .
   Is it possible to define a DEFAULT custom handler that collects only the logs that doesn't match any other custom-handler?
   I've found a workaround adding a new customHandler implementation that manage only logs that doesn't match the list of given parameters.
   But in this case
   • increasing the messageType list the Default handler becomes cumbersome (list can growth up to 100 tokens)
     
   • adding a new messageType i must add a messageType in the exclusion list
     
     

public class CustomPeriodicFileHandler extends PeriodicRotatingFileHandler {


          private String messageType;
          private String filePath;
          private String outFile;
          private String suffixOutFile;
  
          @Override
          public boolean isLoggable( LogRecord arg0 ) {
                    if( messageType.equals( MDC.get( "messageType" ) )) {
                              try {
                                        String logFile = filePath + File.separator + outFile + "_" + MDC.get( "engineName" ) + suffixOutFile;
  
                                        super.setFileName( logFile );
                                        super.setFile( new File( logFile ) );
                                        super.setAutoFlush( true );
                                        super.setAppend( true );
                                        super.setSuffix(".yyyy-MM-dd");
                              } catch (FileNotFoundException e) {
                                        e.printStackTrace();
                              }
                              return true;
                    }
                    else {
                              return false;
                    }
          }

     // follows setter and getter

}

(some properties (i.e.  AutoFlush, Append, and Suffix) shold be mapped as configurable properties but it is a prototype still)

the in configuration file i've add the following configuration

<custom-handler name="Test" class="com.mycompany.log.handler.CustomPeriodicFileHandler" module="com.mycompany.dffp.common">
    <level name="DEBUG"/>
    <formatter>
          <pattern-formatter pattern="..."/>
    </formatter>
    <properties>
          <property name="messageType" value="Operation1"/>
          <property name="filePath" value="PATH_TO_LOG"/>
          <property name="outFile" value="Test"/>
          <property name="suffixOutFile" value=".log"/>
    </properties>
</custom-handler>

....

<logger category="com.mycompany.logger" use-parent-handlers="false">
    <level name="DEBUG"/>
    <handlers>
          <handler name="Test"/>
         <handler name="Default"/>
    </handlers>
</logger>

follows the workaround for default handler

<custom-handler name="Default" class="com.mycompany.dffp.log.handler.CustomDefaultPeriodicFileHandler" module="com.mycompany.dffp.common">
    <level name="DEBUG"/>
    <formatter>
          <pattern-formatter pattern="..."/>
    </formatter>
    <properties>
          <property name="messageType" value="Operation1|Operation2|...|Operation20"/>
          <property name="filePath" value="PATH_TO_LOG"/>
          <property name="outFile" value="Default"/>
          <property name="suffixOutFile" value=".log"/>
    </properties>
</custom-handler>

[nickarls]

Hmm. Not sure if it's threadsafe to set the file in isLoggable (it might be called again before the actual logging is done).

One option would be to skip filtering altogether, and do the file selection in the actual logging method. Perhaps keep a Map<String, File> in the logger where you would lookup the logging file to look for which MDC value.

You would get one logger that would cover all cases.

[mantonec]

I've tested with 20 threads and seems to works fine.
It is an empirical test but i think that for this topic should exists a solution anyhow.I will investigate further.

The idea to skip filtering altogether is an option but in this case the capability in tuning the severity over single functionality will be lost.

[nickarls]

It might be that they are both called within a synchronized by the logging subsystem.

I haven't checked if log level is examined even before calling the logger but if the logger can determine it in the isLoggable then one option could be to have a logOverrides property that

could have the form (operation:level,)* like "add:DEBUG,list:ERROR,remove:INFO".