12 Replies Latest reply on Dec 18, 2012 7:18 AM by xpromache

Wrong body size after depaging in a bridge config

xpromache Dec 8, 2012 4:04 AM

Hello,

I'm using hornetq 2.2.16 Core with netty 3.4.2 to encode Google Protocol Buffer inside the body of messages.

I have a server wich is bridging one queue to a remote server, and the addess is configured for paging. I see that if no paging is involved (i.e. when the remote server is up and the messages can be sent right away), then the messages arrive with the correct size. However, if I stop the remote server and the paging kicks in, then, after starting the remote server, the messages appear sistematically 70 bytes longer than they are, causing an exception from the ProtoBuf decoder.

See below for a dump of a buffer array where readerIndex is 17, writeIndex is 258 resulting in a message of 241 bytes. The real size of the message is 171 bytes, the readerIndex is correct but the writerIndex is not.

I wonder if the fact that I'm using a netty version newer than what was shipped with HornetQ can be related to this problem.

Any hint would be appreciated!

nicolae

0000019C4B000000000000000A0000010208DB8DBB95B7271294011BE5CF10008D3DEBF54FBA461300B351849B02034402030001C608C0000004BE04BE000004

C00098003902700B900516004702803502020201CC08C1000004B604BE009B04BC000004FE00840FFF040503010AFC4402040001CC08C0000004BF04BE000004

C0009803150255000100320AFF54103502030201CF08C1000004B704BE009B04BC00000039027803F700000047018018A68BBB95B72720909E97DF0100000101

00000000001CD1800000000000000000000000000000013B797AD0A9040100000001000000165F0079005F0075006E0069007100750065006900640006FBDDE0

98000000019400000000000009F8010000001879006F00700073002E0074006D005F00640075006D00700000000000000000000000000000013B797AD0A90401

000000020000001075007300650072006E0061006D0065000A00000018790061006D00630073002D00620072006900640067006500000000165F0079005F0075

006E0069007100750065006900640006FBDDE0980000000000000000000000010000000000000000000000000000000000000000000000000000000000000000

00000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000

00000000000000000000

1. Re: Wrong body size after depaging in a bridge config

clebert.suconic Dec 8, 2012 5:04 PM (in response to xpromache)

Can you try a later version of hornetq? I remember some issue with paging? maybe a simple check out of Branch_2_2_AS7?
Actions
2. Re: Wrong body size after depaging in a bridge config

xpromache Dec 12, 2012 4:34 AM (in response to clebert.suconic)

Sorry for the delay.
I did try with Branch_2_2_AS7 and we notice the same behaviour.

Do you think it is worth trying the 2.3 version?
Actions
3. Re: Wrong body size after depaging in a bridge config

clebert.suconic Dec 12, 2012 8:48 AM (in response to xpromache)

Do you think you could replicate the issue on a test?
Actions
4. Re: Wrong body size after depaging in a bridge config

xpromache Dec 12, 2012 10:10 PM (in response to clebert.suconic)

Here is a test with a producer and a consumer each embedding a hornetq server, and a bridge between them:
https://content.wuala.com/contents/nicolae.mihalache/hornetq-paging/pagingtest.tgz?key=MPN0ObBzqa2k&id=1,1125070,11-434,1152213,18

Strangely enough, the queues configuration in the hornetq-configuration-producer.xml does not seem to be applied so you will have to run the producer twice for the bridge to work.

If you start the consumer before the producer, then all messages come with the right size.
If you start the producer and let it finish, and then start the consumer, the depaged messages come in with the wrong size 662 instead of 600.

I also noticed that if the consumer is started after the paging has kicked in, the paging won't stop even if the consumer has cached up.

With the message size of 512, it works.
Actions
5. Re: Wrong body size after depaging in a bridge config

clebert.suconic Dec 12, 2012 10:32 PM (in response to xpromache)

thanks... What's the steps to run it? I mean... first consumer than producer? how do you do it?

I'm just asking so I follow the exact same steps as you
Actions
6. Re: Wrong body size after depaging in a bridge config

xpromache Dec 13, 2012 2:05 AM (in response to clebert.suconic)

First run the producer, let it send all the 9999 messages after wich it will just wait.

Then in another shell run the consumer. At around message 5000 it will start getting the messages with the wrong size (622 instead of 600)
Actions
7. Re: Wrong body size after depaging in a bridge config

clebert.suconic Dec 13, 2012 5:59 PM (in response to xpromache)

I will be fully on it tomorrow. if you could create a JIRA please?
Actions
8. Re: Wrong body size after depaging in a bridge config

xpromache Dec 13, 2012 7:15 PM (in response to clebert.suconic)

Just did:
https://issues.jboss.org/browse/HORNETQ-1101

Thanks a lot!
Actions
9. Re: Wrong body size after depaging in a bridge config

clebert.suconic Dec 14, 2012 5:14 PM (in response to xpromache)

As a workaround, set the message to persistent and the issue will go away. I will have a fix on monday for sure. (no need to set the queue as persistent)

and this has nothing to do with bridging.. a simple consumer over non-persistent messages will also have this issue.
Actions
10. Re: Wrong body size after depaging in a bridge config

clebert.suconic Dec 17, 2012 11:11 PM (in response to xpromache)

I have fixed this.. sending a PR now...

BTW: you're not being able to send messages between your examples because you're using non-persistent queues and you're starting the bridges before you created the queues... and some reconnection.... but I don't think that's relevant for your test.. and it's not a bug really.

I will close the JIRA as soon as I submit the PR and merge it.
Actions
11. Re: Wrong body size after depaging in a bridge config

clebert.suconic Dec 17, 2012 11:39 PM (in response to clebert.suconic)

Pull Requests are sent..
1 of 1 people found this helpful
Actions
12. Re: Wrong body size after depaging in a bridge config

xpromache Dec 18, 2012 7:18 AM (in response to clebert.suconic)

It works, thanks a lot!
1 of 1 people found this helpful
Actions

Go to original post