1 Reply Latest reply on Sep 23, 2019 11:43 AM by nodal

    Safe and efficient log filtering - Multiple handlers logging to the same file

    nodal

      What is the safest and most efficient way to apply filter expressions to log messages that always occur with the same category?

       

      I have over 100 applications logging to the same file. The messages I want to handle are very specific. Applying a single complex filter-spec to every message seems like a lot of overhead, so I decided create individual loggers for each category with a message I want to filter. Each logger gets its own handler with its own filter-spec. That way I would only apply the filtering logic to the smallest number of log messages possible.

       

      Here are the loggers in standalone.xml:

      <root-logger>
          <level name="INFO"/>
          <handlers>
              <handler name="CONSOLE"/>
              <handler name="FILE"/>
          </handlers>
      </root-logger>
      <logger category="org.hibernate.engine.jdbc.spi.SqlExceptionHelper" use-parent-handlers="false">
          <level name="INFO"/>
          <handlers>
              <handler name="CONSOLE"/>
              <handler name="SQL_EXECUTION_HELPER_FILE"/>
          </handlers>
      </logger>
      <logger category="org.jboss.as.ejb3.invocation" use-parent-handlers="false">
          <level name="INFO"/>
          <handlers>
              <handler name="CONSOLE"/>
              <handler name="EJB3_INVOCATION_FILE"/>
          </handlers>
      </logger>
      

       

      Here are the file handlers:

      <periodic-rotating-file-handler name="FILE" autoflush="true">
          <formatter>
              <named-formatter name="PATTERN"/>
          </formatter>
          <file relative-to="jboss.server.log.dir" path="server.log"/>
          <suffix value=".yyyy-MM-dd"/>
          <append value="true"/>
      </periodic-rotating-file-handler>
      <periodic-rotating-file-handler name="SQL_EXECUTION_HELPER_FILE">
          <filter-spec value="any(  not(match(&quot;ERROR: duplicate key value violates unique constraint \&quot;option_option_expiry_id_option_type_id_strike_price_key\&quot;&quot;)),  all(  match(&quot;ERROR: duplicate key value violates unique constraint \&quot;option_option_expiry_id_option_type_id_strike_price_key\&quot;&quot;),  levelChange(WARN)  )  )"/>
          <formatter>
              <named-formatter name="PATTERN"/>
          </formatter>
          <file relative-to="jboss.server.log.dir" path="server.log"/>
          <suffix value=".yyyy-MM-dd"/>
          <append value="true"/>
      </periodic-rotating-file-handler>
      <periodic-rotating-file-handler name="EJB3_INVOCATION_FILE">
          <filter-spec value="any(  not(match(&quot;EJB Invocation failed on component OptionProductManagementDataService for method public void com.nodalexchange.optionproductmanagement.OptionProductManagementDataService.insert&quot;)),  all(  match(&quot;EJB Invocation failed on component OptionProductManagementDataService for method public void com.nodalexchange.optionproductmanagement.OptionProductManagementDataService.insert&quot;),  levelChange(WARN)  )  )"/>
          <formatter>
              <named-formatter name="PATTERN"/>
          </formatter>
          <file relative-to="jboss.server.log.dir" path="server.log"/>
          <suffix value=".yyyy-MM-dd"/>
          <append value="true"/>
      </periodic-rotating-file-handler>
      

       

      I don't understand how these file handlers work under the hood, for example whether they share a file descriptor or each open their own. This has always worked fine for me except there was one incident where I was only getting the messages from a single handler and nothing else, which leads me to believe it is unsafe to use multiple handlers, but I have not been able to reproduce this.

       

      1. Is there a risk of the files becoming corrupted with many handlers writing to the same file?

      2. Is there a better way?