-
1. Re: Message grouping best practices
ataylor Feb 28, 2014 11:13 AM (in response to jiraiya-sama)How HornetQ manage relastionship between a group and a consumer?
One a new group id is used the broker will pick the next available consumer and use that for all the messages with this id.
I ask this because documentation speaks about an example with a group per stock, So associations between consumers and stocks are quite durable and cardinality should be stable.
In my case, for each group, I will have a new ID and so, if HornetQ keep trace indefinitely of all associations, I think I will have a problem after a certain execution duration.
what problem do you think you will have?
So finally, do you think I can use this ID for grouping or should I find something else more stable in term of cardinality?
can you explain what you mean by more stable in terms of cardinality?
-
2. Re: Message grouping best practice for defining IDs
jiraiya-sama Feb 28, 2014 12:14 PM (in response to ataylor)Hi, thanks for your fast feedback.
Per day, I have more or less 10,000,000 of messages with approximately 10 messages per group so 1,000,000 groups will be used per day. Messages will be consumed by approximately 10 permanent consumers.
HornetQ will be used with high availability features. So, for a complete year without any stop (but it is not my limit), I will have a huge number of id used (365,000,000). If HornetQ keeps trace indefinitely of all these associations between Ids and consumers, It is sure that there will be a memory or filesystem issue depending on storage mechanism, or performance issue on grouping handlers. In other words, it will be just like an hashtable which grows continuously. So the only question is when limit will be reached.
Because my groupids are only used once for a set of 10 messages, so for a short time, I am asking if HornetQ really keeps these associations permanently or if there is a mean to discard them (after a defined delay since last message or something like this).
If it is permanent, I will have to find another way to define my Id (per physical device for example: with this, I will have 10 000 group ids reused along the days) to have a more stable and lower number of groupid and so associations in memory like it is in documentation example with one ID per stock. But it is more complicated for me to use id per device, with potential performance issues.
I hope it is clear (I am french so I hope my english speaking is good enough).
Thanks in advance.
Yohann
-
3. Re: Message grouping best practice for defining IDs
ataylor Feb 28, 2014 1:05 PM (in response to jiraiya-sama)you can set a group-timout on the handler so groups are GC'd after not being used for this amount of time.
-
4. Re: Message grouping best practice for defining IDs
jiraiya-sama Feb 28, 2014 1:42 PM (in response to ataylor)Thanks a lot. It is exactly what I need. I did not see it in user manual but it is in configuration reference. Sorry for my lack of search...