Hi,

I was just putting our application in production on jbossas 6.

Everything works as expected but after a while server stops responding at all...

• cpu usage is nothing
• memmory consumption is normal
• there are no new log messages in any of log files
• server does not respond anymore (not even a simple http request)

Restart of the server helps but just for a little while then story repeats.

I did thread dump and have discovered that there are "gazillions" of threads in BLOCKED state mostly like this:

"pool-21-thread-1049" prio=10 tid=0x0b6d8000 nid=0x821 waiting for monitor entry [0x32988000]

   java.lang.Thread.State: BLOCKED (on object monitor)

        at java.lang.Throwable.printStackTrace(Throwable.java:461)

        - waiting to lock <0x4a4f0690> (a org.jboss.stdio.StdioContext$2)

        at java.lang.ThreadGroup.uncaughtException(ThreadGroup.java:985)

        at java.lang.ThreadGroup.uncaughtException(ThreadGroup.java:976)

        at java.lang.ThreadGroup.uncaughtException(ThreadGroup.java:976)

        at java.lang.Thread.dispatchUncaughtException(Thread.java:1874)

and:

"pool-21-thread-4040" prio=10 tid=0x0db4e000 nid=0x13f8 waiting for monitor entry [0x012cc000]

   java.lang.Thread.State: BLOCKED (on object monitor)

        at java.io.PrintStream.write(PrintStream.java:474)

        - waiting to lock <0x4a51d648> (a java.io.PrintStream)

        at java.io.PrintStream.print(PrintStream.java:619)

        at org.jboss.stdio.StdioContext$DelegatingPrintStream.print(StdioContext.java:257)

        at java.lang.ThreadGroup.uncaughtException(ThreadGroup.java:983)

        at java.lang.ThreadGroup.uncaughtException(ThreadGroup.java:976)

        at java.lang.ThreadGroup.uncaughtException(ThreadGroup.java:976)

After seeing this I have reconfigured jboss-logging to not use async logger and everything works as it should.

It looks like async logger just uses up all threads in thread pools and after that non of the services work anymore(jboss-web,jms,..)

To repeat this, you can do a simple POC, configure async logger and configuire him to log to "FILE" appender then

just try to log a few hundered log statements after the server is started. you could just do:

Logger log = LoggerFactory.getLogger("Jboss.crasher");

for (int i=0;i<10000;i++){

   log.info("logging "+i);

}

and server will just stop responding somewhere inbetween.

cheers,

tomaz