Safe and efficient log filtering - Multiple handlers logging to the same file
nodal Sep 19, 2019 2:26 PMWhat is the safest and most efficient way to apply filter expressions to log messages that always occur with the same category?
I have over 100 applications logging to the same file. The messages I want to handle are very specific. Applying a single complex filter-spec to every message seems like a lot of overhead, so I decided create individual loggers for each category with a message I want to filter. Each logger gets its own handler with its own filter-spec. That way I would only apply the filtering logic to the smallest number of log messages possible.
Here are the loggers in standalone.xml:
<root-logger> <level name="INFO"/> <handlers> <handler name="CONSOLE"/> <handler name="FILE"/> </handlers> </root-logger> <logger category="org.hibernate.engine.jdbc.spi.SqlExceptionHelper" use-parent-handlers="false"> <level name="INFO"/> <handlers> <handler name="CONSOLE"/> <handler name="SQL_EXECUTION_HELPER_FILE"/> </handlers> </logger> <logger category="org.jboss.as.ejb3.invocation" use-parent-handlers="false"> <level name="INFO"/> <handlers> <handler name="CONSOLE"/> <handler name="EJB3_INVOCATION_FILE"/> </handlers> </logger>
Here are the file handlers:
<periodic-rotating-file-handler name="FILE" autoflush="true"> <formatter> <named-formatter name="PATTERN"/> </formatter> <file relative-to="jboss.server.log.dir" path="server.log"/> <suffix value=".yyyy-MM-dd"/> <append value="true"/> </periodic-rotating-file-handler> <periodic-rotating-file-handler name="SQL_EXECUTION_HELPER_FILE"> <filter-spec value="any( not(match("ERROR: duplicate key value violates unique constraint \"option_option_expiry_id_option_type_id_strike_price_key\"")), all( match("ERROR: duplicate key value violates unique constraint \"option_option_expiry_id_option_type_id_strike_price_key\""), levelChange(WARN) ) )"/> <formatter> <named-formatter name="PATTERN"/> </formatter> <file relative-to="jboss.server.log.dir" path="server.log"/> <suffix value=".yyyy-MM-dd"/> <append value="true"/> </periodic-rotating-file-handler> <periodic-rotating-file-handler name="EJB3_INVOCATION_FILE"> <filter-spec value="any( not(match("EJB Invocation failed on component OptionProductManagementDataService for method public void com.nodalexchange.optionproductmanagement.OptionProductManagementDataService.insert")), all( match("EJB Invocation failed on component OptionProductManagementDataService for method public void com.nodalexchange.optionproductmanagement.OptionProductManagementDataService.insert"), levelChange(WARN) ) )"/> <formatter> <named-formatter name="PATTERN"/> </formatter> <file relative-to="jboss.server.log.dir" path="server.log"/> <suffix value=".yyyy-MM-dd"/> <append value="true"/> </periodic-rotating-file-handler>
I don't understand how these file handlers work under the hood, for example whether they share a file descriptor or each open their own. This has always worked fine for me except there was one incident where I was only getting the messages from a single handler and nothing else, which leads me to believe it is unsafe to use multiple handlers, but I have not been able to reproduce this.
1. Is there a risk of the files becoming corrupted with many handlers writing to the same file?
2. Is there a better way?