-
1. Re: Thread-safe routing calculation. BindingsImpl.getNextBinding implementation
clebert.suconic Sep 17, 2012 10:58 AM (in response to kzakhar)the routingPosition is used to determine when a message is load balanced to multiple queues with the same name.
Say you have the same queue name bound into two servers. The routing will load balance between these two servers. This will eventually send more messages to one server than the other (depending if you are using multi-thread), and not lose messages.
I agree the routing could be improved though.
-
2. Re: Thread-safe routing calculation. BindingsImpl.getNextBinding implementation
clebert.suconic Sep 17, 2012 11:04 AM (in response to clebert.suconic)We had some talk about this on IRC right now:
I was explaining an example:
clebert:fborges: you have a queue... on the address "SomeAddress" on serverA
[10:01am]clebert:say... SomeQueue on ServerA
[10:01am]clebert:serverB you will have SomeAddress/SomeQueue as well
[10:01am]clebert:the position here will be use to determine to what server to send to
[10:01am]clebert:Half of the messages will go to serverA, half to serverB
[10:02am]clebert:having a multithread calling route will eventually send more messages to A and more messages to B
[10:02am]clebert:which servers the purpose of the routine which is even distribute the queues
[10:02am]clebert:you could change the whole mechanism by a random calculation
[10:03am]clebert:we could improve the synchronization here.. but it won't buy us anything here
-
3. Re: Thread-safe routing calculation. BindingsImpl.getNextBinding implementation
kzakhar Sep 19, 2012 12:51 PM (in response to clebert.suconic)Thanks for the clarifications, Clebert.
What regarding the load-balancing, I have no strict opinion on this. If you would like to have it as it is, let it be. The only concern here may be to have the more self-descriptive and clear code to implement such load-balancing.
Actually, the multiple servers is not my case, so I was continuing looking into the codebase for the routing issues. The next not thread-safe victim in the routing is the implementation of filters matching.
Please take a look at the org.hornetq.core.filter.impl.FilterImpl and org.hornetq.core.filter.impl.Operator classes.
For those two, the not thread-safe implementation eventually lead to the wrong routing.
Attached the test case for using the FilterImpl from multiple threads - that is the case for the messages are being sent from the different sessions to the same destination.
Message was edited by: Konstantin Zakharov
-
TestFilter.java.zip 1.2 KB
-
-
4. Re: Thread-safe routing calculation. BindingsImpl.getNextBinding implementation
kzakhar Sep 19, 2012 4:33 PM (in response to kzakhar)Attached the test scenario for sending the messages from the different sessions to the same topic:
- 1 topic with 2 subscriptions
- each subscription has an exclusive message-selector
- messages are being sent in bursts, with the properties to be consumed half-and-half between subscriptions
- expired messages indicates the wrong routing
The simplest patch would be to add the synchronization to the FilterImpl.match(ServerMessage) method.
-
TestMDB-Routing.zip 12.5 KB
-
-
5. Re: Thread-safe routing calculation. BindingsImpl.getNextBinding implementation
kzakhar Sep 21, 2012 11:11 AM (in response to kzakhar)Can someone confirm it was not a temporary obfuscation, and I'll create a JIRA issue on this?
-
6. Re: Thread-safe routing calculation. BindingsImpl.getNextBinding implementation
clebert.suconic Sep 21, 2012 1:06 PM (in response to kzakhar)Can you create the JIRA? I could replicate it with a Unit Testing.
Will update you soon with the test I created.
-
8. Re: Thread-safe routing calculation. BindingsImpl.getNextBinding implementation
clebert.suconic Sep 21, 2012 1:52 PM (in response to clebert.suconic)and your fix is correct
-
9. Re: Thread-safe routing calculation. BindingsImpl.getNextBinding implementation
kzakhar Sep 21, 2012 2:27 PM (in response to clebert.suconic)Thank you, Clebert!
Best regards,
Konstantin