5 Replies Latest reply on Aug 29, 2007 12:45 PM by nanjunda

    JBossMQ Remote Client receives java.io.EOFException

    alchemista

      Hello,

      I have a pretty high traffic JBossMQ connection between an app server (JBoss 4.0.3 on Win2k3) and a logging server (JBoss 4.0.2 on RHEL). By high traffic I'm referring to 1-25MB per minute.

      I occasionally have connection problems between the Win2k3 machine and RHEL machine, but most often this problem is on the JBossMQ connection.

      On the client side, the error I receive is:

      org.jboss.mq.SpyJMSException: Exiting on IOE; - nested throwable: (java.io.EOFException)
       at org.jboss.mq.SpyJMSException.getAsJMSException(SpyJMSException.java:66)
       at org.jboss.mq.Connection.asynchFailure(Connection.java:437)
       at org.jboss.mq.il.uil2.UILClientILService.asynchFailure(UILClientILService.java:156)
       at org.jboss.mq.il.uil2.SocketManager$ReadTask.handleStop(SocketManager.java:413)
       at org.jboss.mq.il.uil2.SocketManager$ReadTask.run(SocketManager.java:345)
       at java.lang.Thread.run(Thread.java:595)
      Caused by: java.io.EOFException
       at java.io.ObjectInputStream$BlockDataInputStream.readByte(ObjectInputStream.java:2670)
       at java.io.ObjectInputStream.readByte(ObjectInputStream.java:864)
       at org.jboss.mq.il.uil2.SocketManager$ReadTask.run(SocketManager.java:290)
       ... 1 more
      


      On the server, I normally receive DEBUG-level messages like:

      java.net.SocketTimeoutException: Read timed out
       at java.net.SocketInputStream.socketRead0(Native Method)
       at java.net.SocketInputStream.read(SocketInputStream.java:129)
       at java.io.BufferedInputStream.fill(BufferedInputStream.java:218)
       at java.io.BufferedInputStream.read(BufferedInputStream.java:235)
       at org.jboss.util.stream.NotifyingBufferedInputStream.read(NotifyingBufferedInputStream.java:67)
       at java.io.ObjectInputStream$PeekInputStream.peek(ObjectInputStream.java:2205)
       at java.io.ObjectInputStream$BlockDataInputStream.readBlockHeader(ObjectInputStream.java:2385)
       at java.io.ObjectInputStream$BlockDataInputStream.refill(ObjectInputStream.java:2452)
       at java.io.ObjectInputStream$BlockDataInputStream.read(ObjectInputStream.java:2524)
       at java.io.ObjectInputStream$BlockDataInputStream.readByte(ObjectInputStream.java:2673)
       at java.io.ObjectInputStream.readByte(ObjectInputStream.java:874)
       at org.jboss.mq.il.uil2.SocketManager$ReadTask.run(SocketManager.java:290)
       at java.lang.Thread.run(Thread.java:595)
      


      or

      2007-05-11 03:14:58,215 DEBUG [org.jboss.mq.il.uil2.ServerSocketManagerHandler] Exiting on IOE
      java.net.SocketException: Connection reset
       at java.net.SocketInputStream.read(SocketInputStream.java:168)
       at java.io.BufferedInputStream.fill(BufferedInputStream.java:218)
       at java.io.BufferedInputStream.read(BufferedInputStream.java:235)
       at org.jboss.util.stream.NotifyingBufferedInputStream.read(NotifyingBufferedInputStream.java:67)
       at java.io.ObjectInputStream$PeekInputStream.peek(ObjectInputStream.java:2205)
       at java.io.ObjectInputStream$BlockDataInputStream.readBlockHeader(ObjectInputStream.java:2385)
       at java.io.ObjectInputStream$BlockDataInputStream.refill(ObjectInputStream.java:2452)
       at java.io.ObjectInputStream$BlockDataInputStream.read(ObjectInputStream.java:2524)
       at java.io.ObjectInputStream$BlockDataInputStream.readByte(ObjectInputStream.java:2673)
       at java.io.ObjectInputStream.readByte(ObjectInputStream.java:874)
       at org.jboss.mq.il.uil2.SocketManager$ReadTask.run(SocketManager.java:290)
       at java.lang.Thread.run(Thread.java:595)
      


      OR

      2007-05-11 00:05:46,631 DEBUG [org.jboss.naming.NamingService] Error writing response to /192.168.0.1
      java.net.SocketException: Broken pipe
       at java.net.SocketOutputStream.socketWrite0(Native Method)
       at java.net.SocketOutputStream.socketWrite(SocketOutputStream.java:92)
       at java.net.SocketOutputStream.write(SocketOutputStream.java:136)
       at java.io.ObjectOutputStream$BlockDataOutputStream.drain(ObjectOutputStream.java:1682)
       at java.io.ObjectOutputStream$BlockDataOutputStream.setBlockDataMode(ObjectOutputStream.java:1591)
       at java.io.ObjectOutputStream.writeNonProxyDesc(ObjectOutputStream.java:1173)
       at java.io.ObjectOutputStream.writeClassDesc(ObjectOutputStream.java:1127)
       at java.io.ObjectOutputStream.writeOrdinaryObject(ObjectOutputStream.java:1284)
       at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1079)
       at java.io.ObjectOutputStream.writeFatalException(ObjectOutputStream.java:1398)
       at java.io.ObjectOutputStream.writeObject(ObjectOutputStream.java:305)
       at org.jnp.server.Main$BootstrapRequestHandler.run(Main.java:452)
       at org.jboss.util.threadpool.RunnableTaskWrapper.run(RunnableTaskWrapper.java:134)
       at EDU.oswego.cs.dl.util.concurrent.PooledExecutor$Worker.run(PooledExecutor.java:743)
       at java.lang.Thread.run(Thread.java:595)
      


      I am using the default UIL2 configuration settings on the server, but have configured the MessageCache to use a high amount of memory (800/900MB), and I'm unwisely using the default HSQLDB persistence store.

      My question for this post is what possible scenarios would cause the client to receive an EOFException like that? Are there any suggestions on how to diagnose this? The error doesn't mention a read timeout or ping timeout, so I haven't tried increasing these options on the server UIL2 configuration.

      Thank You

        • 1. Re: JBossMQ Remote Client receives java.io.EOFException

          FAQ

          • 2. Re: JBossMQ Remote Client receives java.io.EOFException
            alchemista

             

            "adrian@jboss.org" wrote:
            FAQ


            Adrian,

            Thanks for the reply, but this question is really not addressed in the FAQ. I did read that prior to posting, but it covers other cases of this message. My JMS Client does close connections, my network is configured correctly, I do have ChunkSize set so I'm not getting ping timeouts.

            I was looking more for some more ideas as to what causes these types of problems. Just saying the client or server closed the connection isn't sufficient because neither the client nor server is giving a reason why it would close a connection.

            • 3. Re: JBossMQ Remote Client receives java.io.EOFException
              malert

              Hi,
              I am also facing the same problem, if you got the solution for it please post the solution,

              The Exception i am getting is given below,
              <ExceptionListener Connection@11108602[token=ConnectionToken:ID:1/9a88c2dac04ced
              e24a07e26ff777e8e9 rcvstate=STARTED]><00:25:20:233> Exception in [java.io
              .EOFException ] At line ??? !
              org.jboss.mq.SpyJMSException: Exiting on IOE; - nested throwable: (java.io.EOFEx
              ception)
              at org.jboss.mq.SpyJMSException.getAsJMSException(SpyJMSException.java:7
              2)
              at org.jboss.mq.Connection.asynchFailure(Connection.java:421)
              at org.jboss.mq.il.uil2.UILClientILService.asynchFailure(UILClientILServ
              ice.java:174)
              at org.jboss.mq.il.uil2.SocketManager$ReadTask.handleStop(SocketManager.
              java:439)
              at org.jboss.mq.il.uil2.SocketManager$ReadTask.run(SocketManager.java:37
              1)
              at java.lang.Thread.run(Thread.java:595)
              Caused by: java.io.EOFException
              at java.io.ObjectInputStream$BlockDataInputStream.readByte(ObjectInputSt
              ream.java:2666)
              at java.io.ObjectInputStream.readByte(ObjectInputStream.java:875)
              at org.jboss.mq.il.uil2.SocketManager$ReadTask.run(SocketManager.java:31
              6)
              ... 1 more

              Thanks,
              Malert Jones

              • 4. Re: JBossMQ Remote Client receives java.io.EOFException
                alchemista

                There are two things that may be related, but I haven't been able to completely rule them out yet:

                1. O/S. In my case, Windows Server 2003 SP1. When I reviewed the SP2 bug fixes, there are a lot of TCP-related fixes done, and some of the bugs describe the issue as losing TCP connections. There was one bug that said Windows Server 2003 SP1 talking with a Linux system using sockets could have the connection drop as well. I'm planning on testing whether SP2 prevents these issues, but won't be able to test for a while.

                2. HSQLDB - Are you using HSQLDB? I recently upgraded the included HSQLDB version with the latest JAR file from HSQLDB and so far it appears to be much more reliable. With JBoss 4.0.2, JBoss only included a release candidate version of HSQLDB because they didn't have time to get in the final version. Thus, upgrading HSQLDB (if you use it for the message store) is a worthwhile test.


                I hope everyone will share what has/has not worked for them with this issue. There are several threads with this issue, and not all of them are covered by the FAQs that Adrian mentioned.

                • 5. Re: JBossMQ Remote Client receives java.io.EOFException
                  nanjunda

                  Hi ,

                  I am getting this error when i sent Message which size is big (Around 13MB). here below pasted my exception, Please any one can help me on this ?


                  DEBUG : 20070829.123141.972 : ServerSocketManagerHandler : UIL2.SocketManager.Re
                  adTask#11 client=138.8.17.249:32950 : Exiting on IOE
                  java.io.EOFException
                  at java.io.ObjectInputStream$BlockDataInputStream.readByte(ObjectInputSt
                  ream.java:2666)
                  at java.io.ObjectInputStream.readByte(ObjectInputStream.java:875)
                  at org.jboss.mq.il.uil2.SocketManager$ReadTask.run(SocketManager.java:31
                  7)
                  at java.lang.Thread.run(Thread.java:595)
                  DEBUG : 20070829.123142.464 : BasicQueue : UIL2.SocketManager.ReadTask#11 client
                  =138.8.17.249:32950 : Nacked 1 messages for removed subscription Subscription[su
                  bId=-2147483644connection=ConnectionToken:ID:6/d2d5abe2e5cf2bf55a65e2df3c248ee7
                  destination=QUEUE.SyncMarginRequest messageSelector= Local Create]
                  DEBUG : 20070829.123142.468 : SocketManager : UIL2.SocketManager.WriteTask#12 cl
                  ient=138.8.17.249:32950 : End WriteTask.run
                  DEBUG : 20070829.123142.468 : SocketManager : UIL2.SocketManager.ReadTask#11 cli
                  ent=138.8.17.249:32950 : End ReadTask.run