Session failure due to message decoding error

[xpromache]

Hello,

I have a hornetq core client that loses the connection. I get the exception  in the SessionFailureListener:

The client is receiveing from a  queue mwlmon that is bound to an address mwlmon. To the same address it is bound another queue mwlmon-relay and there is also configured a bridge that copies the messages to a second hornetq server.

If I disable the bridge or both the bridge and the mwlmon-relay queue, then I don't get anymore this exception.

It is difficult for me to create a standalone test case, but I could try running with modified versions of hornetq for further debugging.

Any ideea would be appreciated.

Thanks,

nicolae

WARNING: beforeReconnect

HornetQException[errorCode=0 message=Netty exception]

        at org.hornetq.core.remoting.impl.netty.HornetQChannelHandler.exceptionCaught(HornetQChannelHandler.java:108)

        at org.jboss.netty.channel.SimpleChannelHandler.handleUpstream(SimpleChannelHandler.java:142)

        at org.jboss.netty.channel.StaticChannelPipeline.sendUpstream(StaticChannelPipeline.java:362)

        at org.jboss.netty.channel.StaticChannelPipeline$StaticChannelHandlerContext.sendUpstream(StaticChannelPipeline.java:514)

        at org.jboss.netty.channel.SimpleChannelUpstreamHandler.exceptionCaught(SimpleChannelUpstreamHandler.java:148)

        at org.jboss.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:122)

        at org.jboss.netty.channel.StaticChannelPipeline.sendUpstream(StaticChannelPipeline.java:362)

        at org.jboss.netty.channel.StaticChannelPipeline.sendUpstream(StaticChannelPipeline.java:357)

        at org.jboss.netty.channel.Channels.fireExceptionCaught(Channels.java:432)

        at org.jboss.netty.channel.AbstractChannelSink.exceptionCaught(AbstractChannelSink.java:52)

        at org.jboss.netty.channel.StaticChannelPipeline.notifyHandlerException(StaticChannelPipeline.java:427)

        at org.jboss.netty.channel.StaticChannelPipeline.sendUpstream(StaticChannelPipeline.java:364)

        at org.jboss.netty.channel.StaticChannelPipeline.sendUpstream(StaticChannelPipeline.java:357)

        at org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:274)

        at org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:261)

        at org.jboss.netty.channel.socket.oio.OioWorker.run(OioWorker.java:90)

        at org.jboss.netty.util.ThreadRenamingRunnable.run(ThreadRenamingRunnable.java:108)

        at org.jboss.netty.util.internal.DeadLockProofWorker$1.run(DeadLockProofWorker.java:44)

        at org.jboss.netty.util.VirtualExecutorService$ChildExecutorRunnable.run(VirtualExecutorService.java:181)

        at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)

        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)

        at java.lang.Thread.run(Thread.java:662)

Caused by: java.lang.IllegalArgumentException: estimatedLength: -1393754103

        at org.jboss.netty.buffer.DynamicChannelBuffer.<init>(DynamicChannelBuffer.java:54)

        at org.jboss.netty.buffer.DynamicChannelBuffer.<init>(DynamicChannelBuffer.java:49)

        at org.jboss.netty.buffer.ChannelBuffers.dynamicBuffer(ChannelBuffers.java:212)

        at org.jboss.netty.buffer.ChannelBuffers.dynamicBuffer(ChannelBuffers.java:202)

        at org.hornetq.core.remoting.impl.netty.HornetQFrameDecoder2.decode(HornetQFrameDecoder2.java:166)

        at org.hornetq.core.remoting.impl.netty.HornetQFrameDecoder2.messageReceived(HornetQFrameDecoder2.java:134)

        at org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:274)

        at org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:261)

        at org.jboss.netty.channel.socket.oio.OioWorker.run(OioWorker.java:90)

        ... 4 more

[clebert.suconic]

Version?

Are you using stomp?

[xpromache]

I tried 2.2.16, 2.2.21 and 2.2.24-Snapshot from the Branch_2_2_AS7 git.

Not using stomp, just core clients in java.

Forgot one thing: the messages I'm publishing with the piece of code below. It is always sending roughly the same serializable object.

When the object was smaller, then I didn't get an error. I'm not sure what the size is, but I can check.

public void publish(String type, Serializable object) throws MonPublisherException {

        try {

            ClientMessage msg = createMessage();

            msg.putStringProperty(TYPE, type);

            ByteArrayOutputStream baos = new ByteArrayOutputStream(256);

            ObjectOutputStream oos = new ObjectOutputStream(baos);

            oos.writeObject(object);

            oos.close();

            msg.getBodyBuffer().writeBytes(baos.toByteArray());

            getProducer().send(msg);

        } catch (Exception e) {

            throw new MonPublisherException(e);

        }

    }

[xpromache]

I captured with tcpdump the offending message:

20:10:17.477901 IP localhost.5445 > localhost.52633: P 190:948(758) ack 221 win 256 <nop,nop,timestamp 246812369 246810686>

        0x0000:  4500 032a 5157 4000 4006 e874 7f00 0001  E..*QW@.@..t....

        0x0010:  7f00 0001 1545 cd99 edf2 5a35 ed9d e300  .....E....Z5....

        0x0020:  8018 0100 011f 0000 0101 080a 0eb6 0ed1  ................

        0x0030:  0eb6 083e aced 0005 7372 003a 636f 6d2e  ...>....sr.:com.

        0x0040:  7370 6163 6561 7070 6c69 6361 7469 6f6e  spaceapplication

        0x0050:  732e 6163 6573 2e6d 776c 2e64 6174 612e  s.aces.mwl.data.

        0x0060:  706f 6a6f 2e53 7461 7475 7349 6e66 6f72  pojo.StatusInfor

        0x0070:  6d61 7469 6f6e 0000 0000 0000 0001 0200  mation..........

        0x0080:  094a 0004 7469 6d65 4c00 0b63 646d 6373  .J..timeL..cdmcs

        0x0090:  5374 6174 7573 7400 314c 636f 6d2f 7370  Statust.1Lcom/sp

        0x00a0:  6163 6561 7070 6c69 6361 7469 6f6e 732f  aceapplications/

        0x00b0:  6163 6573 2f6d 776c 2f64 6174 612f 706f  aces/mwl/data/po

        0x00c0:  6a6f 2f53 7461 7475 733b 4c00 0a63 6f6d  jo/Status;L..com

        0x00d0:  6d53 7461 7475 7371 007e 0001 4c00 0965  mStatusq.~..L..e

        0x00e0:  6c74 5374 6174 7573 7100 7e00 014c 000b  ltStatusq.~..L..

        0x00f0:  6d77 6c46 7353 7461 7475 7371 007e 0001  mwlFsStatusq.~..

        0x0100:  4c00 0b6d 776c 4774 5374 6174 7573 7400  L..mwlGtStatust.

        0x0110:  0f4c 6a61 7661 2f75 7469 6c2f 4d61 703b  .Ljava/util/Map;

        0x0120:  4c00 086f 6353 7461 7475 7371 007e 0001  L..ocStatusq.~..

        0x0130:  4c00 0e70 6861 7261 6f77 7353 7461 7475  L..pharaowsStatu

        0x0140:  7371 007e 0001 4c00 0770 6c64 6576 656c  sq.~..L..pldevel

        0x0150:  7100 7e00 0178 7000 0001 3d60 398d b97e  q.~..xp...=`9..~

        0x0160:  7200 2f63 6f6d 2e73 7061 6365 6170 706c  r./com.spaceappl

        0x0170:  6963 6174 696f 6e73 2e61 6365 732e 6d77  ications.aces.mw

        0x0180:  6c2e 6461 7461 2e70 6f6a 6f2e 5374 6174  l.data.pojo.Stat

        0x0190:  7573 0000 0000 0000 0000 1200 0078 7200  us...........xr.

        0x01a0:  0e6a 6176 612e 6c61 6e67 2e45 6e75 6d00  .java.lang.Enum.

        0x01b0:  0000 0000 0000 0012 0000 7870 7400 0545  ..........xpt..E

        0x01c0:  5252 4f52 7e71 007e 0004 7400 024f 4b7e  RROR~q.~..t..OK~

        0x01d0:  7100 7e00 0474 0007 554e 4b4e 4f57 4e71  q.~..t..UNKNOWNq

        0x01e0:  007e 0008 7372 0011 6a61 7661 2e75 7469  .~..sr..java.uti

        0x01f0:  6c2e 4861 7368 4d61 7005 07da c1c3 1660  l.HashMap......`

        0x0200:  d103 0002 4600 0a6c 6f61 6446 6163 746f  ....F..loadFacto

        0x0210:  7249 0009 7468 7265 7368 6f6c 6478 703f  rI..thresholdxp?

        0x0220:  4000 0000 0000 0c77 0800 0000 1000 0000  @......w........

        0x0230:  0b74 0003 3030 3471 007e 0006 7400 0330  .t..004q.~..t..0

        0x0240:  3035 7100 7e00 0674 0003 3030 3671 007e  05q.~..t..006q.~

        0x0250:  0006 7400 0330 3037 7100 7e00 0674 0003  ..t..007q.~..t..

        0x0260:  3530 3171 007e 0006 7400 0331 3032 7100  501q.~..t..102q.

        0x0270:  7e00 0674 0003 3130 3171 007e 0006 7400  ~..t..101q.~..t.

        0x0280:  0330 3031 7100 7e00 0674 0003 3030 3271  .001q.~..t..002q

        0x0290:  007e 0006 7400 0335 3030 7100 7e00 0674  .~..t..500q.~..t

        0x02a0:  0003 3030 3371 007e 0006 7871 007e 0006  ..003q.~..xq.~..

        0x02b0:  7100 7e00 0671 007e 0008 0000 02ea 0000  q.~..q.~........

        0x02c0:  0007 0000 810f 0100 0000 0c6d 0077 006c  ...........m.w.l

        0x02d0:  006d 006f 006e 0000 00ff 0000 0000 0000  .m.o.n..........

        0x02e0:  0000 0000 013d 6039 8db9 0401 0000 0001  .....=`9........

        0x02f0:  0000 0008 7400 7900 7000 6500 0a00 0000  ....t.y.p.e.....

        0x0300:  0c73 0074 0061 0074 0075 0073 0000 0000  .s.t.a.t.u.s....

        0x0310:  0000 0000 0000 0000 0100 0000 0000 0000  ................

        0x0320:  0000 0000 0000 0000 0000                 ..........

[xpromache]

Seems nobody has any idea...

I tried different configurations and the problem shows up quite easily. And it is not only one queue, but it appears whenever there is a queue bound to an adress where another queue is bound and used for a bridge.

Frankly I'm really surprised that nobody had this issue, I really don't see what I'm doing wrong. One other info: the queue that has problems is a temporary queue.

However, I think I found a solution: I created a divert from my original address to another address and I bind the queue that is used for the bridge to this secondary address. This configuration seems to solve the problem.