4 Replies Latest reply on Mar 13, 2013 5:54 PM by Nicolae Mihalache

    Session failure due to message decoding error

    Nicolae Mihalache Newbie

      Hello,

       

      I have a hornetq core client that loses the connection. I get the exception  in the SessionFailureListener:

      The client is receiveing from a  queue mwlmon that is bound to an address mwlmon. To the same address it is bound another queue mwlmon-relay and there is also configured a bridge that copies the messages to a second hornetq server.

      If I disable the bridge or both the bridge and the mwlmon-relay queue, then I don't get anymore this exception.

       

      It is difficult for me to create a standalone test case, but I could try running with modified versions of hornetq for further debugging.

       

      Any ideea would be appreciated.

       

      Thanks,

      nicolae

       

       

       

       

      WARNING: beforeReconnect

      HornetQException[errorCode=0 message=Netty exception]

              at org.hornetq.core.remoting.impl.netty.HornetQChannelHandler.exceptionCaught(HornetQChannelHandler.java:108)

              at org.jboss.netty.channel.SimpleChannelHandler.handleUpstream(SimpleChannelHandler.java:142)

              at org.jboss.netty.channel.StaticChannelPipeline.sendUpstream(StaticChannelPipeline.java:362)

              at org.jboss.netty.channel.StaticChannelPipeline$StaticChannelHandlerContext.sendUpstream(StaticChannelPipeline.java:514)

              at org.jboss.netty.channel.SimpleChannelUpstreamHandler.exceptionCaught(SimpleChannelUpstreamHandler.java:148)

              at org.jboss.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:122)

              at org.jboss.netty.channel.StaticChannelPipeline.sendUpstream(StaticChannelPipeline.java:362)

              at org.jboss.netty.channel.StaticChannelPipeline.sendUpstream(StaticChannelPipeline.java:357)

              at org.jboss.netty.channel.Channels.fireExceptionCaught(Channels.java:432)

              at org.jboss.netty.channel.AbstractChannelSink.exceptionCaught(AbstractChannelSink.java:52)

              at org.jboss.netty.channel.StaticChannelPipeline.notifyHandlerException(StaticChannelPipeline.java:427)

              at org.jboss.netty.channel.StaticChannelPipeline.sendUpstream(StaticChannelPipeline.java:364)

              at org.jboss.netty.channel.StaticChannelPipeline.sendUpstream(StaticChannelPipeline.java:357)

              at org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:274)

              at org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:261)

              at org.jboss.netty.channel.socket.oio.OioWorker.run(OioWorker.java:90)

              at org.jboss.netty.util.ThreadRenamingRunnable.run(ThreadRenamingRunnable.java:108)

              at org.jboss.netty.util.internal.DeadLockProofWorker$1.run(DeadLockProofWorker.java:44)

              at org.jboss.netty.util.VirtualExecutorService$ChildExecutorRunnable.run(VirtualExecutorService.java:181)

              at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)

              at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)

              at java.lang.Thread.run(Thread.java:662)

      Caused by: java.lang.IllegalArgumentException: estimatedLength: -1393754103

              at org.jboss.netty.buffer.DynamicChannelBuffer.<init>(DynamicChannelBuffer.java:54)

              at org.jboss.netty.buffer.DynamicChannelBuffer.<init>(DynamicChannelBuffer.java:49)

              at org.jboss.netty.buffer.ChannelBuffers.dynamicBuffer(ChannelBuffers.java:212)

              at org.jboss.netty.buffer.ChannelBuffers.dynamicBuffer(ChannelBuffers.java:202)

              at org.hornetq.core.remoting.impl.netty.HornetQFrameDecoder2.decode(HornetQFrameDecoder2.java:166)

              at org.hornetq.core.remoting.impl.netty.HornetQFrameDecoder2.messageReceived(HornetQFrameDecoder2.java:134)

              at org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:274)

              at org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:261)

              at org.jboss.netty.channel.socket.oio.OioWorker.run(OioWorker.java:90)

              ... 4 more

        • 2. Re: Session failure due to message decoding error
          Nicolae Mihalache Newbie

          I tried 2.2.16, 2.2.21 and 2.2.24-Snapshot from the Branch_2_2_AS7 git.

          Not using stomp, just core clients in java.

           

          Forgot one thing: the messages I'm publishing with the piece of code below. It is always sending roughly the same serializable object.

          When the object was smaller, then I didn't get an error. I'm not sure what the size is, but I can check.

           

           

           

           

          public void publish(String type, Serializable object) throws MonPublisherException {

                 

                  try {

                      ClientMessage msg = createMessage();

                      msg.putStringProperty(TYPE, type);

           

           

                      ByteArrayOutputStream baos = new ByteArrayOutputStream(256);

                      ObjectOutputStream oos = new ObjectOutputStream(baos);

                      oos.writeObject(object);

                      oos.close();

                     

                      msg.getBodyBuffer().writeBytes(baos.toByteArray());

                      getProducer().send(msg);

                  } catch (Exception e) {

                      throw new MonPublisherException(e);

                  }

              }

          • 3. Re: Session failure due to message decoding error
            Nicolae Mihalache Newbie

            I captured with tcpdump the offending message:

            20:10:17.477901 IP localhost.5445 > localhost.52633: P 190:948(758) ack 221 win 256 <nop,nop,timestamp 246812369 246810686>

                    0x0000:  4500 032a 5157 4000 4006 e874 7f00 0001  E..*QW@.@..t....

                    0x0010:  7f00 0001 1545 cd99 edf2 5a35 ed9d e300  .....E....Z5....

                    0x0020:  8018 0100 011f 0000 0101 080a 0eb6 0ed1  ................

                    0x0030:  0eb6 083e aced 0005 7372 003a 636f 6d2e  ...>....sr.:com.

                    0x0040:  7370 6163 6561 7070 6c69 6361 7469 6f6e  spaceapplication

                    0x0050:  732e 6163 6573 2e6d 776c 2e64 6174 612e  s.aces.mwl.data.

                    0x0060:  706f 6a6f 2e53 7461 7475 7349 6e66 6f72  pojo.StatusInfor

                    0x0070:  6d61 7469 6f6e 0000 0000 0000 0001 0200  mation..........

                    0x0080:  094a 0004 7469 6d65 4c00 0b63 646d 6373  .J..timeL..cdmcs

                    0x0090:  5374 6174 7573 7400 314c 636f 6d2f 7370  Statust.1Lcom/sp

                    0x00a0:  6163 6561 7070 6c69 6361 7469 6f6e 732f  aceapplications/

                    0x00b0:  6163 6573 2f6d 776c 2f64 6174 612f 706f  aces/mwl/data/po

                    0x00c0:  6a6f 2f53 7461 7475 733b 4c00 0a63 6f6d  jo/Status;L..com

                    0x00d0:  6d53 7461 7475 7371 007e 0001 4c00 0965  mStatusq.~..L..e

                    0x00e0:  6c74 5374 6174 7573 7100 7e00 014c 000b  ltStatusq.~..L..

                    0x00f0:  6d77 6c46 7353 7461 7475 7371 007e 0001  mwlFsStatusq.~..

                    0x0100:  4c00 0b6d 776c 4774 5374 6174 7573 7400  L..mwlGtStatust.

                    0x0110:  0f4c 6a61 7661 2f75 7469 6c2f 4d61 703b  .Ljava/util/Map;

                    0x0120:  4c00 086f 6353 7461 7475 7371 007e 0001  L..ocStatusq.~..

                    0x0130:  4c00 0e70 6861 7261 6f77 7353 7461 7475  L..pharaowsStatu

                    0x0140:  7371 007e 0001 4c00 0770 6c64 6576 656c  sq.~..L..pldevel

                    0x0150:  7100 7e00 0178 7000 0001 3d60 398d b97e  q.~..xp...=`9..~

                    0x0160:  7200 2f63 6f6d 2e73 7061 6365 6170 706c  r./com.spaceappl

                    0x0170:  6963 6174 696f 6e73 2e61 6365 732e 6d77  ications.aces.mw

                    0x0180:  6c2e 6461 7461 2e70 6f6a 6f2e 5374 6174  l.data.pojo.Stat

                    0x0190:  7573 0000 0000 0000 0000 1200 0078 7200  us...........xr.

                    0x01a0:  0e6a 6176 612e 6c61 6e67 2e45 6e75 6d00  .java.lang.Enum.

                    0x01b0:  0000 0000 0000 0012 0000 7870 7400 0545  ..........xpt..E

                    0x01c0:  5252 4f52 7e71 007e 0004 7400 024f 4b7e  RROR~q.~..t..OK~

                    0x01d0:  7100 7e00 0474 0007 554e 4b4e 4f57 4e71  q.~..t..UNKNOWNq

                    0x01e0:  007e 0008 7372 0011 6a61 7661 2e75 7469  .~..sr..java.uti

                    0x01f0:  6c2e 4861 7368 4d61 7005 07da c1c3 1660  l.HashMap......`

                    0x0200:  d103 0002 4600 0a6c 6f61 6446 6163 746f  ....F..loadFacto

                    0x0210:  7249 0009 7468 7265 7368 6f6c 6478 703f  rI..thresholdxp?

                    0x0220:  4000 0000 0000 0c77 0800 0000 1000 0000  @......w........

                    0x0230:  0b74 0003 3030 3471 007e 0006 7400 0330  .t..004q.~..t..0

                    0x0240:  3035 7100 7e00 0674 0003 3030 3671 007e  05q.~..t..006q.~

                    0x0250:  0006 7400 0330 3037 7100 7e00 0674 0003  ..t..007q.~..t..

                    0x0260:  3530 3171 007e 0006 7400 0331 3032 7100  501q.~..t..102q.

                    0x0270:  7e00 0674 0003 3130 3171 007e 0006 7400  ~..t..101q.~..t.

                    0x0280:  0330 3031 7100 7e00 0674 0003 3030 3271  .001q.~..t..002q

                    0x0290:  007e 0006 7400 0335 3030 7100 7e00 0674  .~..t..500q.~..t

                    0x02a0:  0003 3030 3371 007e 0006 7871 007e 0006  ..003q.~..xq.~..

                    0x02b0:  7100 7e00 0671 007e 0008 0000 02ea 0000  q.~..q.~........

                    0x02c0:  0007 0000 810f 0100 0000 0c6d 0077 006c  ...........m.w.l

                    0x02d0:  006d 006f 006e 0000 00ff 0000 0000 0000  .m.o.n..........

                    0x02e0:  0000 0000 013d 6039 8db9 0401 0000 0001  .....=`9........

                    0x02f0:  0000 0008 7400 7900 7000 6500 0a00 0000  ....t.y.p.e.....

                    0x0300:  0c73 0074 0061 0074 0075 0073 0000 0000  .s.t.a.t.u.s....

                    0x0310:  0000 0000 0000 0000 0100 0000 0000 0000  ................

                    0x0320:  0000 0000 0000 0000 0000                 ..........

            • 4. Re: Session failure due to message decoding error
              Nicolae Mihalache Newbie

              Seems nobody has any idea...

              I tried different configurations and the problem shows up quite easily. And it is not only one queue, but it appears whenever there is a queue bound to an adress where another queue is bound and used for a bridge.

              Frankly I'm really surprised that nobody had this issue, I really don't see what I'm doing wrong. One other info: the queue that has problems is a temporary queue.

               

               

              However, I think I found a solution: I created a divert from my original address to another address and I bind the queue that is used for the bridge to this secondary address. This configuration seems to solve the problem.