3 Replies Latest reply on Feb 24, 2008 3:05 PM by memema

    JMS provider failure

    frenchyboss

      Hi !
      Running JBoss 4.0.4 GA with JBoss MQ on Sun JVM 1.4, Win2K3, in a clustered configuration

      After a variable period of activity (from 2 hours to a week), I get exception from the MDB Container:

      WARN [org.jboss.ejb.plugins.jms.JMSContainerInvoker] JMS provider failure detected for xxxMDB
      org.jboss.mq.SpyJMSException: Exiting on IOE; - nested throwable: (java.net.SocketTimeoutException: Read timed out)
       at org.jboss.mq.SpyJMSException.getAsJMSException(SpyJMSException.java:78)
       at org.jboss.mq.Connection.asynchFailure(Connection.java:420)
       at org.jboss.mq.il.uil2.UILClientILService.asynchFailure(UILClientILService.java:174)
       at org.jboss.mq.il.uil2.SocketManager$ReadTask.handleStop(SocketManager.java:440)
       at org.jboss.mq.il.uil2.SocketManager$ReadTask.run(SocketManager.java:372)
       at java.lang.Thread.run(Thread.java:534)
      Caused by: java.net.SocketTimeoutException: Read timed out
       at java.net.SocketInputStream.socketRead0(Native Method)
       at java.net.SocketInputStream.read(SocketInputStream.java:129)
       at java.io.BufferedInputStream.fill(BufferedInputStream.java:183)
       at java.io.BufferedInputStream.read(BufferedInputStream.java:201)
       at org.jboss.util.stream.NotifyingBufferedInputStream.read(NotifyingBufferedInputStream.java:79)
       at java.io.ObjectInputStream$PeekInputStream.peek(ObjectInputStream.java:2133)
       at java.io.ObjectInputStream$BlockDataInputStream.readBlockHeader(ObjectInputStream.java:2313)
       at java.io.ObjectInputStream$BlockDataInputStream.refill(ObjectInputStream.java:2380)
       at java.io.ObjectInputStream$BlockDataInputStream.read(ObjectInputStream.java:2452)
       at java.io.ObjectInputStream$BlockDataInputStream.readByte(ObjectInputStream.java:2601)
       at java.io.ObjectInputStream.readByte(ObjectInputStream.java:845)
       at org.jboss.mq.il.uil2.SocketManager$ReadTask.run(SocketManager.java:317)


      Then the container reconnects to JMS after the 10seconds wait and fails again 120 seconds later.

      12:11:34,100 INFO [org.jboss.ejb.plugins.jms.JMSContainerInvoker] Waiting for reconnect internal 10000ms for xxxMDB
      12:11:44,147 INFO [org.jboss.ejb.plugins.jms.JMSContainerInvoker] Trying to reconnect to JMS provider for xxxMDB
      12:11:44,178 INFO [org.jboss.ejb.plugins.jms.JMSContainerInvoker] Reconnected to JMS provider for xxxMDB
      12:13:44,694 WARN [org.jboss.ejb.plugins.jms.JMSContainerInvoker] JMS provider failure detected for xxxMDB

      The whole is ending up in an out of memory exception a few hours later.
      Any ideas ?
      Thanks in advance,
      François


        • 1. Re: JMS provider failure
          roschi

          Hi.

          Have the same problem.
          JBoss 4.0.5.GA, jdk 1.5.0_10 on Solaris 10.

          On load get the exception 5 minutes after start for each running bean.
          The error occurs on MDB, where I don't have to set an ExceptionListener().

          2007-01-09 13:43:21,040 WARN [org.jboss.mq.Connection] Connection failure, use javax.jms.Connection.setExceptionListener() to handle this error and reconnect
          org.jboss.mq.SpyJMSException: Exiting on IOE; - nested throwable: (java.net.SocketTimeoutException: Read timed out)
           at org.jboss.mq.SpyJMSException.getAsJMSException(SpyJMSException.java:72)
           at org.jboss.mq.Connection.asynchFailure(Connection.java:421)
           at org.jboss.mq.il.uil2.UILClientILService.asynchFailure(UILClientILService.java:174)
           at org.jboss.mq.il.uil2.SocketManager$ReadTask.handleStop(SocketManager.java:439)
           at org.jboss.mq.il.uil2.SocketManager$ReadTask.run(SocketManager.java:371)
           at java.lang.Thread.run(Thread.java:595)
          Caused by: java.net.SocketTimeoutException: Read timed out
           at java.net.SocketInputStream.socketRead0(Native Method)
           at java.net.SocketInputStream.read(SocketInputStream.java:129)
           at java.io.BufferedInputStream.fill(BufferedInputStream.java:218)
           at java.io.BufferedInputStream.read(BufferedInputStream.java:235)
           at org.jboss.util.stream.NotifyingBufferedInputStream.read(NotifyingBufferedInputStream.java:79)
           at java.io.ObjectInputStream$PeekInputStream.peek(ObjectInputStream.java:2196)
           at java.io.ObjectInputStream$BlockDataInputStream.readBlockHeader(ObjectInputStream.java:2376)
           at java.io.ObjectInputStream$BlockDataInputStream.refill(ObjectInputStream.java:2443)
           at java.io.ObjectInputStream$BlockDataInputStream.read(ObjectInputStream.java:2515)
           at java.io.ObjectInputStream$BlockDataInputStream.readByte(ObjectInputStream.java:2664)
           at java.io.ObjectInputStream.readByte(ObjectInputStream.java:875)
           at org.jboss.mq.il.uil2.SocketManager$ReadTask.run(SocketManager.java:316)
           ... 1 more
          


          After a while the retry for each bean:
          2007-01-09 13:43:37,390 INFO [org.jboss.ejb.plugins.jms.JMSContainerInvoker] Trying to reconnect to JMS provider for ProcessorMessageBean
          


          Any help / guess welcome!
          Thom

          • 2. Re: JMS provider failure
            frenchyboss

            Hi Thom,
            I've found that we had a leak with some QueueConnection / QueueSession in CLIENT_ACKNOWLEDGE mode : some messages where retrieved but not acknowledged. (Symptom : ClientCount in DestinationManager is always growing). Thoses messages were consumed from an another session.
            I guess that this leak generated some ressource shortage for JMS and explains the provider failure.
            François

            • 3. Re: JMS provider failure
              memema

              Hi Francois, Hi Thom
              I have the same identical problem with JBoss 4.0.4 GA in cluster and JDK 1.5.07 on HP-Unix.

              It seems a JBoss bug... :-(

              Did you have found a solution to that problem?

              Thanks for any suggestion,
              Claudio