2 Replies Latest reply on May 6, 2006 2:41 PM by chancks

    Increasing Thread Problem leading to Out Of Memory

    chancks
    chancks May 3, 2006 10:18 PM

      Hi,

      We are using JBoss AS 4.0.3SP1. Was wondering if anyone out here faced a similar situation of getting an increasing number of threads created while using JBossMQ on Solaris.

      We used the javax.naming and javax.jms packages.

      To send out a message, we passed in the queue name to get the QueueConnectionFactory. After that we use the QueueConnectionFactory to create a new QueueConnection & QueueSession. Lastly, we will use the QueueSession to create a new QueueSender to send the message.

      To receive a message, we passed in the queue name to get the QueueConnectionFactory. After that we use the QueueConnectionFactory to create a new QueueConnection & QueueSession. Lastly will use the QueueSession to create a new QueueReceiver and set a listener to this QueueConnection to listen for the incoming message.

      As the application runs, the number of threads tied to the JBoss process will keep increasing until it hits the 3000+ range and the following out of memory error will occur:

      2006-04-16 04:15:40,876 DEBUG [org.jboss.mq.il.uil2.SocketManager] Exiting on unexpected error in read task

      java.lang.OutOfMemoryError: unable to create new native thread

      at java.lang.Thread.start(Native Method)

      at EDU.oswego.cs.dl.util.concurrent.PooledExecutor.addThread(PooledExecutor.java:520)

      at EDU.oswego.cs.dl.util.concurrent.PooledExecutor.execute(PooledExecutor.java:880)

      at org.jboss.mq.il.uil2.SocketManager$ReadTask.run(SocketManager.java:309)

      at java.lang.Thread.run(Thread.java:534)

      We ran the same code on JBoss 4.0.3SP1 on Windows and did a low load test. We found that while the number of threads increased during the load, it fell back to the same level after the load test. So this shows that the application isn't the culprit.

      I also read from http://wiki.jboss.org/wiki/Wiki.jsp?page=ConfigJBossMQDB that there are thread and socket leaks with hsqldb that comes default with JBossMQ. We are using this. Could this be the reason? Is there a patch for this problem?

      Would hope that some kind soul can assist here... thanks.

      - Stephen

      • 2455 Views
      • Tags:
        • 1. Re: Increasing Thread Problem leading to Out Of Memory
          genman
          genman May 4, 2006 4:17 PM (in response to chancks)


          If you're accessing queues locally, use the pooled connection provider java:/JmsXA ... remotely (on a client), use a single cached connection for all your messaging uses.

            • Actions
          • 2. Re: Increasing Thread Problem leading to Out Of Memory
            chancks
            chancks May 6, 2006 2:41 PM (in response to chancks)

            Hi,

            On my earlier mentioned problem, I've captured a thread dump after running a few transactions and it seems that most the following threads shown below should have exited but they haven't.

            Hope that someone can enlighten me on what's happening. Thanks.


            - Stephen
            ==========
            "UIL2(SocketManager.MsgPool@14649ed client=172.31.11.33:56225)#4" daemon prio=5 tid=0x0045f810 nid=0xa3 in Object.wait() [eab41000..eab419c0]
            at java.lang.Object.wait(Native Method)
            - waiting on <0xed8fb868> (a EDU.oswego.cs.dl.util.concurrent.LinkedNode)
            at EDU.oswego.cs.dl.util.concurrent.SynchronousChannel.poll(SynchronousChannel.java:353)
            - locked <0xed8fb868> (a EDU.oswego.cs.dl.util.concurrent.LinkedNode)
            at EDU.oswego.cs.dl.util.concurrent.PooledExecutor.getTask(PooledExecutor.java:723)
            at EDU.oswego.cs.dl.util.concurrent.PooledExecutor$Worker.run(PooledExecutor.java:747)
            at java.lang.Thread.run(Thread.java:534)

            "UIL2(SocketManager.MsgPool@188a387 client=172.31.11.33:56222)#3" daemon prio=5 tid=0x005bf138 nid=0x9c in Object.wait() [eab71000..eab719c0]
            at java.lang.Object.wait(Native Method)
            - waiting on <0xed809a48> (a EDU.oswego.cs.dl.util.concurrent.LinkedNode)
            at EDU.oswego.cs.dl.util.concurrent.SynchronousChannel.poll(SynchronousChannel.java:353)
            - locked <0xed809a48> (a EDU.oswego.cs.dl.util.concurrent.LinkedNode)
            at EDU.oswego.cs.dl.util.concurrent.PooledExecutor.getTask(PooledExecutor.java:723)
            at EDU.oswego.cs.dl.util.concurrent.PooledExecutor$Worker.run(PooledExecutor.java:747)
            at java.lang.Thread.run(Thread.java:534)

            "UIL2(SocketManager.MsgPool@169c6f2 client=172.31.11.33:56223)#3" daemon prio=5 tid=0x00759b20 nid=0x9a in Object.wait() [eaba1000..eaba19c0]
            at java.lang.Object.wait(Native Method)
            - waiting on <0xed7384b0> (a EDU.oswego.cs.dl.util.concurrent.LinkedNode)
            at EDU.oswego.cs.dl.util.concurrent.SynchronousChannel.poll(SynchronousChannel.java:353)
            - locked <0xed7384b0> (a EDU.oswego.cs.dl.util.concurrent.LinkedNode)
            at EDU.oswego.cs.dl.util.concurrent.PooledExecutor.getTask(PooledExecutor.java:723)
            at EDU.oswego.cs.dl.util.concurrent.PooledExecutor$Worker.run(PooledExecutor.java:747)
            at java.lang.Thread.run(Thread.java:534)

            "UIL2.SocketManager.WriteTask#40 client=172.31.11.33:56418" daemon prio=5 tid=0x00f2e5a8 nid=0x93 in Object.wait() [ea881000..ea8819c0]
            at java.lang.Object.wait(Native Method)
            - waiting on <0xed42a620> (a java.lang.Object)
            at java.lang.Object.wait(Object.java:429)
            at EDU.oswego.cs.dl.util.concurrent.LinkedQueue.take(LinkedQueue.java:122)
            - locked <0xed42a620> (a java.lang.Object)
            at org.jboss.mq.il.uil2.SocketManager$WriteTask.run(SocketManager.java:492)
            at java.lang.Thread.run(Thread.java:534)

            "UIL2(SocketManager.MsgPool@1565e9d client=172.31.11.33:56220)#4" daemon prio=5 tid=0x01093ec8 nid=0x8e in Object.wait() [eac31000..eac319c0]
            at java.lang.Object.wait(Native Method)
            - waiting on <0xed4ccfb0> (a EDU.oswego.cs.dl.util.concurrent.LinkedNode)
            at EDU.oswego.cs.dl.util.concurrent.SynchronousChannel.poll(SynchronousChannel.java:353)
            - locked <0xed4ccfb0> (a EDU.oswego.cs.dl.util.concurrent.LinkedNode)
            at EDU.oswego.cs.dl.util.concurrent.PooledExecutor.getTask(PooledExecutor.java:723)
            at EDU.oswego.cs.dl.util.concurrent.PooledExecutor$Worker.run(PooledExecutor.java:747)
            at java.lang.Thread.run(Thread.java:534)

            "UIL2.SocketManager.WriteTask#34 client=172.31.11.33:56412" daemon prio=5 tid=0x01412f88 nid=0x88 in Object.wait() [ea911000..ea9119c0]
            at java.lang.Object.wait(Native Method)
            - waiting on <0xed2190b0> (a java.lang.Object)
            at java.lang.Object.wait(Object.java:429)
            at EDU.oswego.cs.dl.util.concurrent.LinkedQueue.take(LinkedQueue.java:122)
            - locked <0xed2190b0> (a java.lang.Object)
            at org.jboss.mq.il.uil2.SocketManager$WriteTask.run(SocketManager.java:492)
            at java.lang.Thread.run(Thread.java:534)

            "UIL2(SocketManager.MsgPool@1497b1 client=172.31.11.33:56218)#4" daemon prio=5 tid=0x014125e8 nid=0x86 in Object.wait() [eac91000..eac919c0]
            at java.lang.Object.wait(Native Method)
            - waiting on <0xed362e20> (a EDU.oswego.cs.dl.util.concurrent.LinkedNode)
            at EDU.oswego.cs.dl.util.concurrent.SynchronousChannel.poll(SynchronousChannel.java:353)
            - locked <0xed362e20> (a EDU.oswego.cs.dl.util.concurrent.LinkedNode)
            at EDU.oswego.cs.dl.util.concurrent.PooledExecutor.getTask(PooledExecutor.java:723)
            at EDU.oswego.cs.dl.util.concurrent.PooledExecutor$Worker.run(PooledExecutor.java:747)
            at java.lang.Thread.run(Thread.java:534)

            "UIL2.SocketManager.WriteTask#32 client=172.31.11.33:56409" daemon prio=5 tid=0x01107cc8 nid=0x84 in Object.wait() [ea9a1000..ea9a19c0]
            at java.lang.Object.wait(Native Method)
            - waiting on <0xed2141a0> (a java.lang.Object)
            at java.lang.Object.wait(Object.java:429)
            at EDU.oswego.cs.dl.util.concurrent.LinkedQueue.take(LinkedQueue.java:122)
            - locked <0xed2141a0> (a java.lang.Object)
            at org.jboss.mq.il.uil2.SocketManager$WriteTask.run(SocketManager.java:492)
            at java.lang.Thread.run(Thread.java:534)

            "UIL2(SocketManager.MsgPool@1735f84 client=172.31.11.33:56216)#3" daemon prio=5 tid=0x00d87a90 nid=0x7f in Object.wait() [fbff1000..fbff19c0]
            at java.lang.Object.wait(Native Method)
            - waiting on <0xed1dd300> (a EDU.oswego.cs.dl.util.concurrent.LinkedNode)
            at EDU.oswego.cs.dl.util.concurrent.SynchronousChannel.poll(SynchronousChannel.java:353)
            - locked <0xed1dd300> (a EDU.oswego.cs.dl.util.concurrent.LinkedNode)
            at EDU.oswego.cs.dl.util.concurrent.PooledExecutor.getTask(PooledExecutor.java:723)
            at EDU.oswego.cs.dl.util.concurrent.PooledExecutor$Worker.run(PooledExecutor.java:747)
            at java.lang.Thread.run(Thread.java:534)

            "JBoss System Threads(1)-12" daemon prio=5 tid=0x011086d8 nid=0x74 in Object.wait() [eaa61000..eaa619c0]
            at java.lang.Object.wait(Native Method)
            - waiting on <0xefc4dc80> (a java.lang.Object)
            at EDU.oswego.cs.dl.util.concurrent.BoundedLinkedQueue.poll(BoundedLinkedQueue.java:253)
            - locked <0xefc4dc80> (a java.lang.Object)
            at EDU.oswego.cs.dl.util.concurrent.PooledExecutor.getTask(PooledExecutor.java:723)
            at org.jboss.util.threadpool.MinPooledExecutor.getTask(MinPooledExecutor.java:91)
            at EDU.oswego.cs.dl.util.concurrent.PooledExecutor$Worker.run(PooledExecutor.java:747)
            at java.lang.Thread.run(Thread.java:534)

            "Message Pushers-1" daemon prio=5 tid=0x010020e0 nid=0x6f in Object.wait() [ea9d1000..ea9d19c0]
            at java.lang.Object.wait(Native Method)
            - waiting on <0xef961608> (a java.util.LinkedList)
            at java.lang.Object.wait(Object.java:429)
            at org.jboss.mq.threadpool.ThreadPool$WorkerThread.idle(ThreadPool.java:205)
            at org.jboss.mq.threadpool.ThreadPool$WorkerThread.run(ThreadPool.java:224)
            - locked <0xef961608> (a java.util.LinkedList)

            "JBoss System Threads(1)-11" daemon prio=5 tid=0x009b2e00 nid=0x6a in Object.wait() [eab11000..eab119c0]
            at java.lang.Object.wait(Native Method)
            - waiting on <0xefc4dc80> (a java.lang.Object)
            at EDU.oswego.cs.dl.util.concurrent.BoundedLinkedQueue.poll(BoundedLinkedQueue.java:253)
            - locked <0xefc4dc80> (a java.lang.Object)
            at EDU.oswego.cs.dl.util.concurrent.PooledExecutor.getTask(PooledExecutor.java:723)
            at org.jboss.util.threadpool.MinPooledExecutor.getTask(MinPooledExecutor.java:91)
            at EDU.oswego.cs.dl.util.concurrent.PooledExecutor$Worker.run(PooledExecutor.java:747)
            at java.lang.Thread.run(Thread.java:534)

            "UIL2(SocketManager.MsgPool@1843ca4 client=172.31.11.33:56224)#2" daemon prio=5 tid=0x005be4a8 nid=0x66 in Object.wait() [eabd1000..eabd19c0]
            at java.lang.Object.wait(Native Method)
            - waiting on <0xed6cb8b8> (a EDU.oswego.cs.dl.util.concurrent.LinkedNode)
            at EDU.oswego.cs.dl.util.concurrent.SynchronousChannel.poll(SynchronousChannel.java:353)
            - locked <0xed6cb8b8> (a EDU.oswego.cs.dl.util.concurrent.LinkedNode)
            at EDU.oswego.cs.dl.util.concurrent.PooledExecutor.getTask(PooledExecutor.java:723)
            at EDU.oswego.cs.dl.util.concurrent.PooledExecutor$Worker.run(PooledExecutor.java:747)
            at java.lang.Thread.run(Thread.java:534)

            "UIL2(SocketManager.MsgPool@11f2158 client=172.31.11.33:56221)#2" daemon prio=5 tid=0x01279480 nid=0x65 in Object.wait() [eac01000..eac019c0]
            at java.lang.Object.wait(Native Method)
            - waiting on <0xeda0bfb0> (a EDU.oswego.cs.dl.util.concurrent.LinkedNode)
            at EDU.oswego.cs.dl.util.concurrent.SynchronousChannel.poll(SynchronousChannel.java:353)
            - locked <0xeda0bfb0> (a EDU.oswego.cs.dl.util.concurrent.LinkedNode)
            at EDU.oswego.cs.dl.util.concurrent.PooledExecutor.getTask(PooledExecutor.java:723)
            at EDU.oswego.cs.dl.util.concurrent.PooledExecutor$Worker.run(PooledExecutor.java:747)
            at java.lang.Thread.run(Thread.java:534)

            "UIL2(SocketManager.MsgPool@ac576f client=172.31.11.33:56219)#2" daemon prio=5 tid=0x0072bc70 nid=0x63 in Object.wait() [eac61000..eac619c0]
            at java.lang.Object.wait(Native Method)
            - waiting on <0xed7f26e0> (a EDU.oswego.cs.dl.util.concurrent.LinkedNode)
            at EDU.oswego.cs.dl.util.concurrent.SynchronousChannel.poll(SynchronousChannel.java:353)
            - locked <0xed7f26e0> (a EDU.oswego.cs.dl.util.concurrent.LinkedNode)
            at EDU.oswego.cs.dl.util.concurrent.PooledExecutor.getTask(PooledExecutor.java:723)
            at EDU.oswego.cs.dl.util.concurrent.PooledExecutor$Worker.run(PooledExecutor.java:747)
            at java.lang.Thread.run(Thread.java:534)

            "UIL2.SocketManager.WriteTask#18 client=172.31.11.33:56225" daemon prio=5 tid=0x009fc768 nid=0x56 in Object.wait() [ea431000..ea4319c0]
            at java.lang.Object.wait(Native Method)
            - waiting on <0xef9626a8> (a java.lang.Object)
            at java.lang.Object.wait(Object.java:429)
            at EDU.oswego.cs.dl.util.concurrent.LinkedQueue.take(LinkedQueue.java:122)
            - locked <0xef9626a8> (a java.lang.Object)
            at org.jboss.mq.il.uil2.SocketManager$WriteTask.run(SocketManager.java:492)
            at java.lang.Thread.run(Thread.java:534)

            "UIL2.SocketManager.ReadTask#17 client=172.31.11.33:56225" daemon prio=5 tid=0x006c6a60 nid=0x55 runnable [ea461000..ea4619c0]
            at java.net.SocketInputStream.socketRead0(Native Method)
            at java.net.SocketInputStream.read(SocketInputStream.java:129)
            at java.io.BufferedInputStream.fill(BufferedInputStream.java:183)
            at java.io.BufferedInputStream.read(BufferedInputStream.java:201)
            - locked <0xef962f60> (a org.jboss.util.stream.NotifyingBufferedInputStream)
            at org.jboss.util.stream.NotifyingBufferedInputStream.read(NotifyingBufferedInputStream.java:67)
            at java.io.ObjectInputStream$PeekInputStream.peek(ObjectInputStream.java:2133)
            at java.io.ObjectInputStream$BlockDataInputStream.readBlockHeader(ObjectInputStream.java:2313)
            at java.io.ObjectInputStream$BlockDataInputStream.refill(ObjectInputStream.java:2380)
            at java.io.ObjectInputStream$BlockDataInputStream.read(ObjectInputStream.java:2452)
            at java.io.ObjectInputStream$BlockDataInputStream.readByte(ObjectInputStream.java:2601)
            at java.io.ObjectInputStream.readByte(ObjectInputStream.java:845)
            at org.jboss.mq.il.uil2.SocketManager$ReadTask.run(SocketManager.java:290)
            at java.lang.Thread.run(Thread.java:534)

            "UIL2.SocketManager.WriteTask#16 client=172.31.11.33:56224" daemon prio=5 tid=0x005bd638 nid=0x51 in Object.wait() [ea521000..ea5219c0]
            at java.lang.Object.wait(Native Method)
            - waiting on <0xef963058> (a java.lang.Object)
            at java.lang.Object.wait(Object.java:429)
            at EDU.oswego.cs.dl.util.concurrent.LinkedQueue.take(LinkedQueue.java:122)
            - locked <0xef963058> (a java.lang.Object)
            at org.jboss.mq.il.uil2.SocketManager$WriteTask.run(SocketManager.java:492)
            at java.lang.Thread.run(Thread.java:534)

            "UIL2.SocketManager.ReadTask#15 client=172.31.11.33:56224" daemon prio=5 tid=0x005bcff0 nid=0x50 runnable [ea551000..ea5519c0]
            at java.net.SocketInputStream.socketRead0(Native Method)
            at java.net.SocketInputStream.read(SocketInputStream.java:129)
            at java.io.BufferedInputStream.fill(BufferedInputStream.java:183)
            at java.io.BufferedInputStream.read(BufferedInputStream.java:201)
            - locked <0xef963910> (a org.jboss.util.stream.NotifyingBufferedInputStream)
            at org.jboss.util.stream.NotifyingBufferedInputStream.read(NotifyingBufferedInputStream.java:67)
            at java.io.ObjectInputStream$PeekInputStream.peek(ObjectInputStream.java:2133)
            at java.io.ObjectInputStream$BlockDataInputStream.readBlockHeader(ObjectInputStream.java:2313)
            at java.io.ObjectInputStream$BlockDataInputStream.refill(ObjectInputStream.java:2380)
            at java.io.ObjectInputStream$BlockDataInputStream.read(ObjectInputStream.java:2452)
            at java.io.ObjectInputStream$BlockDataInputStream.readByte(ObjectInputStream.java:2601)
            at java.io.ObjectInputStream.readByte(ObjectInputStream.java:845)
            at org.jboss.mq.il.uil2.SocketManager$ReadTask.run(SocketManager.java:290)
            at java.lang.Thread.run(Thread.java:534)

            "UIL2.SocketManager.WriteTask#14 client=172.31.11.33:56223" daemon prio=5 tid=0x00946be8 nid=0x4f in Object.wait() [ea581000..ea5819c0]
            at java.lang.Object.wait(Native Method)
            - waiting on <0xef963a08> (a java.lang.Object)
            at java.lang.Object.wait(Object.java:429)
            at EDU.oswego.cs.dl.util.concurrent.LinkedQueue.take(LinkedQueue.java:122)
            - locked <0xef963a08> (a java.lang.Object)
            at org.jboss.mq.il.uil2.SocketManager$WriteTask.run(SocketManager.java:492)
            at java.lang.Thread.run(Thread.java:534)

            "UIL2.SocketManager.ReadTask#13 client=172.31.11.33:56223" daemon prio=5 tid=0x00946938 nid=0x4e runnable [ea5b1000..ea5b19c0]
            at java.net.SocketInputStream.socketRead0(Native Method)
            at java.net.SocketInputStream.read(SocketInputStream.java:129)
            at java.io.BufferedInputStream.fill(BufferedInputStream.java:183)
            at java.io.BufferedInputStream.read(BufferedInputStream.java:201)
            - locked <0xef9642c0> (a org.jboss.util.stream.NotifyingBufferedInputStream)
            at org.jboss.util.stream.NotifyingBufferedInputStream.read(NotifyingBufferedInputStream.java:67)
            at java.io.ObjectInputStream$PeekInputStream.peek(ObjectInputStream.java:2133)
            at java.io.ObjectInputStream$BlockDataInputStream.readBlockHeader(ObjectInputStream.java:2313)
            at java.io.ObjectInputStream$BlockDataInputStream.refill(ObjectInputStream.java:2380)
            at java.io.ObjectInputStream$BlockDataInputStream.read(ObjectInputStream.java:2452)
            at java.io.ObjectInputStream$BlockDataInputStream.readByte(ObjectInputStream.java:2601)
            at java.io.ObjectInputStream.readByte(ObjectInputStream.java:845)
            at org.jboss.mq.il.uil2.SocketManager$ReadTask.run(SocketManager.java:290)
            at java.lang.Thread.run(Thread.java:534)

            "UIL2.SocketManager.WriteTask#12 client=172.31.11.33:56222" daemon prio=5 tid=0x009974d0 nid=0x4c in Object.wait() [ea611000..ea6119c0]
            at java.lang.Object.wait(Native Method)
            - waiting on <0xef9643b8> (a java.lang.Object)
            at java.lang.Object.wait(Object.java:429)
            at EDU.oswego.cs.dl.util.concurrent.LinkedQueue.take(LinkedQueue.java:122)
            - locked <0xef9643b8> (a java.lang.Object)
            at org.jboss.mq.il.uil2.SocketManager$WriteTask.run(SocketManager.java:492)
            at java.lang.Thread.run(Thread.java:534)

            "UIL2.SocketManager.ReadTask#11 client=172.31.11.33:56222" daemon prio=5 tid=0x00996530 nid=0x4b runnable [ea641000..ea6419c0]
            at java.net.SocketInputStream.socketRead0(Native Method)
            at java.net.SocketInputStream.read(SocketInputStream.java:129)
            at java.io.BufferedInputStream.fill(BufferedInputStream.java:183)
            at java.io.BufferedInputStream.read(BufferedInputStream.java:201)
            - locked <0xef964c70> (a org.jboss.util.stream.NotifyingBufferedInputStream)
            at org.jboss.util.stream.NotifyingBufferedInputStream.read(NotifyingBufferedInputStream.java:67)
            at java.io.ObjectInputStream$PeekInputStream.peek(ObjectInputStream.java:2133)
            at java.io.ObjectInputStream$BlockDataInputStream.readBlockHeader(ObjectInputStream.java:2313)
            at java.io.ObjectInputStream$BlockDataInputStream.refill(ObjectInputStream.java:2380)
            at java.io.ObjectInputStream$BlockDataInputStream.read(ObjectInputStream.java:2452)
            at java.io.ObjectInputStream$BlockDataInputStream.readByte(ObjectInputStream.java:2601)
            at java.io.ObjectInputStream.readByte(ObjectInputStream.java:845)
            at org.jboss.mq.il.uil2.SocketManager$ReadTask.run(SocketManager.java:290)
            at java.lang.Thread.run(Thread.java:534)

            "UIL2.SocketManager.WriteTask#10 client=172.31.11.33:56221" daemon prio=5 tid=0x00a84738 nid=0x46 in Object.wait() [ea731000..ea7319c0]
            at java.lang.Object.wait(Native Method)
            - waiting on <0xef964d68> (a java.lang.Object)
            at java.lang.Object.wait(Object.java:429)
            at EDU.oswego.cs.dl.util.concurrent.LinkedQueue.take(LinkedQueue.java:122)
            - locked <0xef964d68> (a java.lang.Object)
            at org.jboss.mq.il.uil2.SocketManager$WriteTask.run(SocketManager.java:492)
            at java.lang.Thread.run(Thread.java:534)

            "UIL2.SocketManager.ReadTask#9 client=172.31.11.33:56221" daemon prio=5 tid=0x00996b38 nid=0x45 runnable [ea761000..ea7619c0]
            at java.net.SocketInputStream.socketRead0(Native Method)
            at java.net.SocketInputStream.read(SocketInputStream.java:129)
            at java.io.BufferedInputStream.fill(BufferedInputStream.java:183)
            at java.io.BufferedInputStream.read(BufferedInputStream.java:201)
            - locked <0xef965620> (a org.jboss.util.stream.NotifyingBufferedInputStream)
            at org.jboss.util.stream.NotifyingBufferedInputStream.read(NotifyingBufferedInputStream.java:67)
            at java.io.ObjectInputStream$PeekInputStream.peek(ObjectInputStream.java:2133)
            at java.io.ObjectInputStream$BlockDataInputStream.readBlockHeader(ObjectInputStream.java:2313)
            at java.io.ObjectInputStream$BlockDataInputStream.refill(ObjectInputStream.java:2380)
            at java.io.ObjectInputStream$BlockDataInputStream.read(ObjectInputStream.java:2452)
            at java.io.ObjectInputStream$BlockDataInputStream.readByte(ObjectInputStream.java:2601)
            at java.io.ObjectInputStream.readByte(ObjectInputStream.java:845)
            at org.jboss.mq.il.uil2.SocketManager$ReadTask.run(SocketManager.java:290)
            at java.lang.Thread.run(Thread.java:534)

            "UIL2.SocketManager.WriteTask#8 client=172.31.11.33:56220" daemon prio=5 tid=0x00a85f80 nid=0x43 in Object.wait() [ea7c1000..ea7c19c0]
            at java.lang.Object.wait(Native Method)
            - waiting on <0xef965718> (a java.lang.Object)
            at java.lang.Object.wait(Object.java:429)
            at EDU.oswego.cs.dl.util.concurrent.LinkedQueue.take(LinkedQueue.java:122)
            - locked <0xef965718> (a java.lang.Object)
            at org.jboss.mq.il.uil2.SocketManager$WriteTask.run(SocketManager.java:492)
            at java.lang.Thread.run(Thread.java:534)

            "UIL2.SocketManager.ReadTask#7 client=172.31.11.33:56220" daemon prio=5 tid=0x0075a3e0 nid=0x42 runnable [ea7f1000..ea7f19c0]
            at java.net.SocketInputStream.socketRead0(Native Method)
            at java.net.SocketInputStream.read(SocketInputStream.java:129)
            at java.io.BufferedInputStream.fill(BufferedInputStream.java:183)
            at java.io.BufferedInputStream.read(BufferedInputStream.java:201)
            - locked <0xef965fd0> (a org.jboss.util.stream.NotifyingBufferedInputStream)
            at org.jboss.util.stream.NotifyingBufferedInputStream.read(NotifyingBufferedInputStream.java:67)
            at java.io.ObjectInputStream$PeekInputStream.peek(ObjectInputStream.java:2133)
            at java.io.ObjectInputStream$BlockDataInputStream.readBlockHeader(ObjectInputStream.java:2313)
            at java.io.ObjectInputStream$BlockDataInputStream.refill(ObjectInputStream.java:2380)
            at java.io.ObjectInputStream$BlockDataInputStream.read(ObjectInputStream.java:2452)
            at java.io.ObjectInputStream$BlockDataInputStream.readByte(ObjectInputStream.java:2601)
            at java.io.ObjectInputStream.readByte(ObjectInputStream.java:845)
            at org.jboss.mq.il.uil2.SocketManager$ReadTask.run(SocketManager.java:290)
            at java.lang.Thread.run(Thread.java:534)

            "UIL2.SocketManager.WriteTask#6 client=172.31.11.33:56219" daemon prio=5 tid=0x00a856a0 nid=0x41 in Object.wait() [ea821000..ea8219c0]
            at java.lang.Object.wait(Native Method)
            - waiting on <0xef9660c8> (a java.lang.Object)
            at java.lang.Object.wait(Object.java:429)
            at EDU.oswego.cs.dl.util.concurrent.LinkedQueue.take(LinkedQueue.java:122)
            - locked <0xef9660c8> (a java.lang.Object)
            at org.jboss.mq.il.uil2.SocketManager$WriteTask.run(SocketManager.java:492)
            at java.lang.Thread.run(Thread.java:534)

            "UIL2.SocketManager.ReadTask#5 client=172.31.11.33:56219" daemon prio=5 tid=0x00a859e0 nid=0x40 runnable [ea851000..ea8519c0]
            at java.net.SocketInputStream.socketRead0(Native Method)
            at java.net.SocketInputStream.read(SocketInputStream.java:129)
            at java.io.BufferedInputStream.fill(BufferedInputStream.java:183)
            at java.io.BufferedInputStream.read(BufferedInputStream.java:201)
            - locked <0xef966980> (a org.jboss.util.stream.NotifyingBufferedInputStream)
            at org.jboss.util.stream.NotifyingBufferedInputStream.read(NotifyingBufferedInputStream.java:67)
            at java.io.ObjectInputStream$PeekInputStream.peek(ObjectInputStream.java:2133)
            at java.io.ObjectInputStream$BlockDataInputStream.readBlockHeader(ObjectInputStream.java:2313)
            at java.io.ObjectInputStream$BlockDataInputStream.refill(ObjectInputStream.java:2380)
            at java.io.ObjectInputStream$BlockDataInputStream.read(ObjectInputStream.java:2452)
            at java.io.ObjectInputStream$BlockDataInputStream.readByte(ObjectInputStream.java:2601)
            at java.io.ObjectInputStream.readByte(ObjectInputStream.java:845)
            at org.jboss.mq.il.uil2.SocketManager$ReadTask.run(SocketManager.java:290)
            at java.lang.Thread.run(Thread.java:534)

            "UIL2.SocketManager.WriteTask#4 client=172.31.11.33:56218" daemon prio=5 tid=0x00bc0b98 nid=0x3e in Object.wait() [ea8b1000..ea8b19c0]
            at java.lang.Object.wait(Native Method)
            - waiting on <0xef966a78> (a java.lang.Object)
            at java.lang.Object.wait(Object.java:429)
            at EDU.oswego.cs.dl.util.concurrent.LinkedQueue.take(LinkedQueue.java:122)
            - locked <0xef966a78> (a java.lang.Object)
            at org.jboss.mq.il.uil2.SocketManager$WriteTask.run(SocketManager.java:492)
            at java.lang.Thread.run(Thread.java:534)

            "UIL2.SocketManager.ReadTask#3 client=172.31.11.33:56218" daemon prio=5 tid=0x00bc09f8 nid=0x3d runnable [ea8e1000..ea8e19c0]
            at java.net.SocketInputStream.socketRead0(Native Method)
            at java.net.SocketInputStream.read(SocketInputStream.java:129)
            at java.io.BufferedInputStream.fill(BufferedInputStream.java:183)
            at java.io.BufferedInputStream.read(BufferedInputStream.java:201)
            - locked <0xef967330> (a org.jboss.util.stream.NotifyingBufferedInputStream)
            at org.jboss.util.stream.NotifyingBufferedInputStream.read(NotifyingBufferedInputStream.java:67)
            at java.io.ObjectInputStream$PeekInputStream.peek(ObjectInputStream.java:2133)
            at java.io.ObjectInputStream$BlockDataInputStream.readBlockHeader(ObjectInputStream.java:2313)
            at java.io.ObjectInputStream$BlockDataInputStream.refill(ObjectInputStream.java:2380)
            at java.io.ObjectInputStream$BlockDataInputStream.read(ObjectInputStream.java:2452)
            at java.io.ObjectInputStream$BlockDataInputStream.readByte(ObjectInputStream.java:2601)
            at java.io.ObjectInputStream.readByte(ObjectInputStream.java:845)
            at org.jboss.mq.il.uil2.SocketManager$ReadTask.run(SocketManager.java:290)
            at java.lang.Thread.run(Thread.java:534)

            "UIL2.SocketManager.WriteTask#2 client=172.31.11.33:56216" daemon prio=5 tid=0x012792e0 nid=0x3b in Object.wait() [ea941000..ea9419c0]
            at java.lang.Object.wait(Native Method)
            - waiting on <0xef967428> (a java.lang.Object)
            at java.lang.Object.wait(Object.java:429)
            at EDU.oswego.cs.dl.util.concurrent.LinkedQueue.take(LinkedQueue.java:122)
            - locked <0xef967428> (a java.lang.Object)
            at org.jboss.mq.il.uil2.SocketManager$WriteTask.run(SocketManager.java:492)
            at java.lang.Thread.run(Thread.java:534)

            "UIL2.SocketManager.ReadTask#1 client=172.31.11.33:56216" daemon prio=5 tid=0x00bc0de0 nid=0x3a runnable [ea971000..ea9719c0]
            at java.net.SocketInputStream.socketRead0(Native Method)
            at java.net.SocketInputStream.read(SocketInputStream.java:129)
            at java.io.BufferedInputStream.fill(BufferedInputStream.java:183)
            at java.io.BufferedInputStream.read(BufferedInputStream.java:201)
            - locked <0xef967cd0> (a org.jboss.util.stream.NotifyingBufferedInputStream)
            at org.jboss.util.stream.NotifyingBufferedInputStream.read(NotifyingBufferedInputStream.java:67)
            at java.io.ObjectInputStream$PeekInputStream.peek(ObjectInputStream.java:2133)
            at java.io.ObjectInputStream$BlockDataInputStream.readBlockHeader(ObjectInputStream.java:2313)
            at java.io.ObjectInputStream$BlockDataInputStream.refill(ObjectInputStream.java:2380)
            at java.io.ObjectInputStream$BlockDataInputStream.read(ObjectInputStream.java:2452)
            at java.io.ObjectInputStream$BlockDataInputStream.readByte(ObjectInputStream.java:2601)
            at java.io.ObjectInputStream.readByte(ObjectInputStream.java:845)
            at org.jboss.mq.il.uil2.SocketManager$ReadTask.run(SocketManager.java:290)
            at java.lang.Thread.run(Thread.java:534)

            "RMI LeaseChecker" daemon prio=5 tid=0x00ea1b70 nid=0x33 waiting on condition [eaac1000..eaac19c0]
            at java.lang.Thread.sleep(Native Method)
            at sun.rmi.transport.DGCImpl$LeaseChecker.run(DGCImpl.java:307)
            at java.lang.Thread.run(Thread.java:534)            ==============

              • Actions