5 Replies Latest reply on Jul 16, 2012 11:57 AM by thelinker

    log4j SMTP appender sending errors one line stacktrace per mail

    simoncigoj

      I've configured SMTP error reporting via log4j SMPT appender for jboss5.1 but I'm sometimes reciving one line of stack trace error in one mail, so for one error I receive 100 or more email-s.

       

      How can I avoid this type of behaveour. When I restart the server it works for some time then on irregualr basis it starst again with one line emails.

       

      Any hints?

       

      I'm using

       

      jboss 5.1

      Windosw server 2008 R2 64 bit

        • 1. Re: jog4j SMTP appender sendin errors one line stachtrace per mail
          peterj

          I suspect that you will have to debug Log4J's SMTP appender to answer this.

           

          But I did have a thought - when you receive multiple emails for a stack trace, what does that same stack trace look like in the server.log? Is that stack trace, by any chance, being written to stdout/stderr and intercepted by log4j? If so, then it is one line per stack trace entry. If that is the case, you will have to either fix the app that is writing to stdout/stderr, or write a custom SMTP appender that handles this situation.

          • 2. Re: jog4j SMTP appender sendin errors one line stachtrace per mail
            simoncigoj

            Hmm I have set the File appender and the SMPT appender, so the log is outputed to a file and to a SMTP, I have commented out the console appender in jboss-log4j.xml. After the inspection off server.log files I can approve that also there there are errors written by lines, could be some sort of buffer setting error? ... the stack trace of an error is writen line by line for example like this :

             

             

            13:24:59,911 ERROR [STDERR] [] - [root] (ajp-127.0.0.1-8009-4) javax.imageio.IIOException: Can't get input stream from URL!

            13:24:59,926 ERROR [STDERR] [] - [root] (ajp-127.0.0.1-8009-4) at javax.imageio.ImageIO.read(Unknown Source)

            13:24:59,942 ERROR [STDERR] [] - [root] (ajp-127.0.0.1-8009-4) at si.arctur.fourPM.components.gant.ProjectGantExport.getImage(ProjectGantExport.java:176)

            13:24:59,958 ERROR [STDERR] [] - [root] (ajp-127.0.0.1-8009-4) at si.arctur.fourPM.components.gant.ProjectGantExport.drawItemIcons(ProjectGantExport.java:343)

            13:24:59,973 ERROR [STDERR] [] - [root] (ajp-127.0.0.1-8009-4) at si.arctur.fourPM.components.gant.ProjectGantExport.drawItems(ProjectGantExport.java:199)

            13:24:59,989 ERROR [STDERR] [] - [root] (ajp-127.0.0.1-8009-4) at si.arctur.fourPM.components.gant.ProjectGantExport.generateImage(ProjectGantExport.java:129)

            13:25:00,005 ERROR [STDERR] [] - [root] (ajp-127.0.0.1-8009-4) at si.arctur.fourPM.components.gant.ProjectGantExport.exportStructure(ProjectGantExport.java:92)

            13:25:00,020 ERROR [STDERR] [] - [root] (ajp-127.0.0.1-8009-4) at si.arctur.fourPM.components.gant.ProjectGantExport.generate(ProjectGantExport.java:75)

            13:25:00,036 ERROR [STDERR] [] - [root] (ajp-127.0.0.1-8009-4) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

            13:25:00,051 ERROR [STDERR] [] - [root] (ajp-127.0.0.1-8009-4) at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)

            13:25:00,083 ERROR [STDERR] [] - [root] (ajp-127.0.0.1-8009-4) at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)

            ......

             

             

            But it happens on irregular bases, I cann not reproduce this type of behaviour, so it is quite difficult to debug. After restarting the server it usulay works for a while.

             

            My log4j appender conf settings:

             

             

            <!-- A time/date based rolling appender -->

               <appender name="FILE" class="org.jboss.logging.appender.DailyRollingFileAppender">

                  <errorHandler class="org.jboss.logging.util.OnlyOnceErrorHandler"/>

                  <param name="File" value="D:\\Logs\\jboss\\server.log"/>

                        <!--<param name="ImmediateFlush" value="false" />

                        <param name="BufferedIO" value="true" />-->

                       

                  <param name="Append" value="true"/>

                  <!-- In AS 5.0.x the server log threshold was set by a system

                       property. In 5.1 and later we are instead using the system

                       property to set the priority on the root logger (see <root/> below)

                  <param name="Threshold" value="${jboss.server.log.threshold}"/>

                  -->

                 

                  <!-- Rollover at midnight each day -->

                  <param name="DatePattern" value="'.'yyyy-MM-dd"/>

             

             

                  <!-- Rollover at the top of each hour

                  <param name="DatePattern" value="'.'yyyy-MM-dd-HH"/>

                  -->

             

             

                  <layout class="org.apache.log4j.PatternLayout">

                     <!-- The default pattern: Date Priority [Category] (Thread) Message\n -->

                     <param name="ConversionPattern" value="%d{ABSOLUTE} %-5p [%c] [%X{url}] - [%X{username}] (%t) %m%n"/>

             

             

                     <!-- The full pattern: Date MS Priority [Category] (Thread:NDC) Message\n

                     <param name="ConversionPattern" value="%d %-5r %-5p [%c] (%t:%x) %m%n"/>

                      -->

                  </layout>

               </appender>

             

             

            <appender name="SMTP" class="org.apache.log4j.net.SMTPAppender">

                 <errorHandler class="org.jboss.logging.util.OnlyOnceErrorHandler"/>

                 <param name="Threshold" value="ERROR"/>

                 <param name="To" value="emailAccount"/>

                 <param name="From" value="emailAccount"/>

                 <param name="Subject" value="Subject"/>

                 <param name="SMTPHost" value="smtp host"/>

                 <param name="BufferSize" value="20"/>

                 <layout class="org.apache.log4j.PatternLayout">

                   <param name="ConversionPattern" value="[%d{ABSOLUTE},%c] [%X{url}] [%X{username}]  %m%n"/>

                 </layout>

               </appender>

            • 3. Re: jog4j SMTP appender sendin errors one line stachtrace per mail
              alesj

              Something is writting to StdErr.

               

              ----

               

              dmlloyd: ever seen this?

               

               

               

              10:32

              dmlloyd

              10:32

              yeah

              10:32

              that'll happen when stack traces are written to stderr instead of logged

              10:32

              there's no way the stdio interceptor can tell what lines are part of a message

              10:32

              a single message

               

               

               

              10:33

              alesj

              10:33

              why would suddenly things be written to stderr and not log?

              10:33

              dmlloyd: ^

               

               

               

              10:34

              dmlloyd

              10:34

              they're not suddenly being written to stderr

              10:34

              it's this particular one which is

               

               

               

              10:35

              alesj

              10:35

              so, you're saying the somewhere in the stack, something is writting to stderr … some user app code?

               

               

               

              10:35

              dmlloyd

              10:35

              the exception is thrown from line 1395 of javax/imageio/ImageIO.java

              10:35

              yeah probably

              10:36

              whoever is calling that, probably si.arctur.fourPM.components.gant.ProjectGantExport#getImage() if I had to guess

              10:36

              if they don't want that behavior, they should edit their logging config and set the STDERR category to not use parent handlers

              10:37

              then manually add CONSOLE and FILE back in

              10:37

              to jsut that category

              • 4. Re: jog4j SMTP appender sendin errors one line stachtrace per mail
                simoncigoj

                Tnx for the detailed answer...indead I checked our code and in some places we use printStackTrace() which is by default writing to STDERR, I will try to log those exceptions better.


                I' will also try your suggestion, and observer the behaveour.

                 

                Tnx again for the quick answer.

                p.s.

                I've also found some suggestions to solve this problem here:
                http://i-proving.ca/space/Technologies/JBoss/JBoss+and+log4j

                and here
                http://community.jboss.org/thread/160630

                • 5. Re: jog4j SMTP appender sendin errors one line stachtrace per mail
                  thelinker