2 Replies Latest reply on Oct 20, 2010 7:39 AM by Ivan Rododendro

    ThreadPoolFull Exception

    Ivan Rododendro Newbie

      Hi,

      I,'ve two JBoss 4.2.2 instances (jdk 1.6) in a production environment, a JMS provider (MQ) and a topic consumer.

       

      After about 32K messages consumed I get this error on the consumer :

       

      2010-10-14 15:00:20,116 ERROR [Connection Consumer for dest Subscription[subId=-2147483648connection=ConnectionToken:JBEX81/41dc33536d876be333c811381549540f destination=TOPIC.requestTopic messageSelector=(applicationName = 'references') OR (applicationName = 'ocea') OR (applicationName = 'tarifC5') Local Create] id=3] [org.jboss.resource.adapter.jms.inflow.JmsServerSession] Unable to schedule work
      javax.resource.spi.work.WorkRejectedException: org.jboss.util.threadpool.ThreadPoolFullException: java.lang.OutOfMemoryError: unable to create new native thread
      at org.jboss.resource.work.WorkWrapper.rejected(WorkWrapper.java:250)
      at org.jboss.util.threadpool.BasicTaskWrapper.taskRejected(BasicTaskWrapper.java:345)
      at org.jboss.util.threadpool.BasicTaskWrapper.rejectTask(BasicTaskWrapper.java:164)
      at org.jboss.util.threadpool.BasicThreadPool.execute(BasicThreadPool.java:417)
      at org.jboss.util.threadpool.BasicThreadPool.runTaskWrapper(BasicThreadPool.java:192)
      at org.jboss.resource.work.JBossWorkManager.executeWork(JBossWorkManager.java:205)
      at org.jboss.resource.work.JBossWorkManager.scheduleWork(JBossWorkManager.java:144)
      at org.jboss.resource.adapter.jms.inflow.JmsServerSession.start(JmsServerSession.java:212)
      at org.jboss.mq.SpyConnectionConsumer.run(SpyConnectionConsumer.java:353)
      at java.lang.Thread.run(Thread.java:619)
      Caused by: org.jboss.util.threadpool.ThreadPoolFullException: java.lang.OutOfMemoryError: unable to create new native thread
      ... 7 more

      2010-10-14 15:00:20,116 ERROR [Connection Consumer for dest Subscription[subId=-2147483648connection=ConnectionToken:JBEX81/41dc33536d876be333c811381549540f destination=TOPIC.requestTopic messageSelector=(applicationName = 'references') OR (applicationName = 'ocea') OR (applicationName = 'tarifC5') Local Create] id=3] [org.jboss.resource.adapter.jms.inflow.JmsServerSession] Unable to schedule work

      javax.resource.spi.work.WorkRejectedException: org.jboss.util.threadpool.ThreadPoolFullException: java.lang.OutOfMemoryError: unable to create new native thread

           at org.jboss.resource.work.WorkWrapper.rejected(WorkWrapper.java:250)

           at org.jboss.util.threadpool.BasicTaskWrapper.taskRejected(BasicTaskWrapper.java:345)

           at org.jboss.util.threadpool.BasicTaskWrapper.rejectTask(BasicTaskWrapper.java:164)

           at org.jboss.util.threadpool.BasicThreadPool.execute(BasicThreadPool.java:417)

           at org.jboss.util.threadpool.BasicThreadPool.runTaskWrapper(BasicThreadPool.java:192)

           at org.jboss.resource.work.JBossWorkManager.executeWork(JBossWorkManager.java:205)

           at org.jboss.resource.work.JBossWorkManager.scheduleWork(JBossWorkManager.java:144)

           at org.jboss.resource.adapter.jms.inflow.JmsServerSession.start(JmsServerSession.java:212)

           at org.jboss.mq.SpyConnectionConsumer.run(SpyConnectionConsumer.java:353)

           at java.lang.Thread.run(Thread.java:619)

      Caused by: org.jboss.util.threadpool.ThreadPoolFullException: java.lang.OutOfMemoryError: unable to create new native thread

           ... 7 more

       

      After this no more message is consumed, and the subscription in non more visible on jmx-console.

       

      I've checked JVM heap and permgen Memory.

       

      I've checked OS memory and there's 1GB of free memory.

       

      Talking about threads I can see that every time a message is consumed a new thread is created, no thread is reused in MDB pool. Nevertheless every thread of this pool is destroyed after 60s. So when the exception occurs I can see about 32K threads total created in VisualVM. Live threads are about 180.

       

      If I add physical memory to the consumer server the MDB is able to consume more messages.

       

      Goolgling around it seems to be a problem of memory fragmentation, but I was'nt able to verify this.

       

      Does anyone can help?

       

      Thanks

      Ivan