1 Reply Latest reply on Sep 23, 2019 11:43 AM by Drew Nutter

    Safe and efficient log filtering - Multiple handlers logging to the same file

    Drew Nutter Newbie

      What is the safest and most efficient way to apply filter expressions to log messages that always occur with the same category?

       

      I have over 100 applications logging to the same file. The messages I want to handle are very specific. Applying a single complex filter-spec to every message seems like a lot of overhead, so I decided create individual loggers for each category with a message I want to filter. Each logger gets its own handler with its own filter-spec. That way I would only apply the filtering logic to the smallest number of log messages possible.

       

      Here are the loggers in standalone.xml:

      <root-logger>
          <level name="INFO"/>
          <handlers>
              <handler name="CONSOLE"/>
              <handler name="FILE"/>
          </handlers>
      </root-logger>
      <logger category="org.hibernate.engine.jdbc.spi.SqlExceptionHelper" use-parent-handlers="false">
          <level name="INFO"/>
          <handlers>
              <handler name="CONSOLE"/>
              <handler name="SQL_EXECUTION_HELPER_FILE"/>
          </handlers>
      </logger>
      <logger category="org.jboss.as.ejb3.invocation" use-parent-handlers="false">
          <level name="INFO"/>
          <handlers>
              <handler name="CONSOLE"/>
              <handler name="EJB3_INVOCATION_FILE"/>
          </handlers>
      </logger>
      

       

      Here are the file handlers:

      <periodic-rotating-file-handler name="FILE" autoflush="true">
          <formatter>
              <named-formatter name="PATTERN"/>
          </formatter>
          <file relative-to="jboss.server.log.dir" path="server.log"/>
          <suffix value=".yyyy-MM-dd"/>
          <append value="true"/>
      </periodic-rotating-file-handler>
      <periodic-rotating-file-handler name="SQL_EXECUTION_HELPER_FILE">
          <filter-spec value="any(  not(match(&quot;ERROR: duplicate key value violates unique constraint \&quot;option_option_expiry_id_option_type_id_strike_price_key\&quot;&quot;)),  all(  match(&quot;ERROR: duplicate key value violates unique constraint \&quot;option_option_expiry_id_option_type_id_strike_price_key\&quot;&quot;),  levelChange(WARN)  )  )"/>
          <formatter>
              <named-formatter name="PATTERN"/>
          </formatter>
          <file relative-to="jboss.server.log.dir" path="server.log"/>
          <suffix value=".yyyy-MM-dd"/>
          <append value="true"/>
      </periodic-rotating-file-handler>
      <periodic-rotating-file-handler name="EJB3_INVOCATION_FILE">
          <filter-spec value="any(  not(match(&quot;EJB Invocation failed on component OptionProductManagementDataService for method public void com.nodalexchange.optionproductmanagement.OptionProductManagementDataService.insert&quot;)),  all(  match(&quot;EJB Invocation failed on component OptionProductManagementDataService for method public void com.nodalexchange.optionproductmanagement.OptionProductManagementDataService.insert&quot;),  levelChange(WARN)  )  )"/>
          <formatter>
              <named-formatter name="PATTERN"/>
          </formatter>
          <file relative-to="jboss.server.log.dir" path="server.log"/>
          <suffix value=".yyyy-MM-dd"/>
          <append value="true"/>
      </periodic-rotating-file-handler>
      

       

      I don't understand how these file handlers work under the hood, for example whether they share a file descriptor or each open their own. This has always worked fine for me except there was one incident where I was only getting the messages from a single handler and nothing else, which leads me to believe it is unsafe to use multiple handlers, but I have not been able to reproduce this.

       

      1. Is there a risk of the files becoming corrupted with many handlers writing to the same file?

      2. Is there a better way?