-
1. Re: HornetQ Subscriber Limit? Use DB Instead?
timfox Jun 20, 2010 6:23 AM (in response to unsavory)I think the most efficient way to do this would be to use a topic, not a queue, and create and cache a non durable subscriber per subscriber id, when creating the subscription specify the subscriber id in a selector.
Should be no problem to cache hundreds if not thousands of subscribers. If you put the subscribers in a map keyed with subscriber id you can easily look them up when the web request comes in.
-
2. Re: HornetQ Subscriber Limit? Use DB Instead?
unsavory Jun 21, 2010 12:50 PM (in response to timfox)Thank you very much for the reply, Tim. I like your idea a lot.
The only question I have, is how to handle the last value piece using a topic. Can a last value topic be created? Or will I need to manage the messages in the topic manually (ie: programatically make sure only 1 message per subscriber is on the topic)?
The thing is, we only want our subscribers to have a maximum of 1 message to retrieve from the topic. And that 1 message should always be the last published message for that subscriber. I just don't see any examples of a last value topic, like there is for a queue.
-
3. Re: HornetQ Subscriber Limit? Use DB Instead?
timfox Jun 28, 2010 8:10 AM (in response to unsavory)1 of 1 people found this helpfulA JMS topic is just a set of subscriptions.
Each subscription is just a queue, like any other queue so the same semantics apply.
-
4. Re: HornetQ Subscriber Limit? Use DB Instead?
unsavory Jun 30, 2010 2:40 PM (in response to timfox)Thanks Tim!
We are up and running. The problem space was a little more complicated than I originally thought because we also need to know when to produce messages based on whether or not a client has been active in the past xx minutes, and because requests come in frequently, we did not want to fetch more data than is absolutely necessary.
Here is what we ended up doing in case it might help someone else.
1. A web request comes in and we check for any messages on the topic using a topic selector. We cache these topic consumers in a concurrent hashmap as suggested. If no topic consumer exists for this client, we create a new one and place it into the map. After checking for messages and consuming them if present, we ping a "client registry" (another concurrent hashmap) with a last request timestamp so we can know if the client is active.
2. There is a separate command queue that is used for retrieving data for the clients. The command queue producers know how often to produce commands of various types and only produce at a given rate (ie: every 5 minutes per client for this command, every 10 minutes per client for that command).
3. There are producers that check the entire client registry every xx seconds and if the timestamps are not expired, they use the command producers described in step 2 to produce data retrieval commands. Expired registry entries are removed from the map.
4. There is xx number of command consumers that feed off the command queue fetching data, and then finally place messages back onto the client's topic containing said data.
All this is a very complicated and round about way of saving database resources so we do not have to fetch data for our web clients except at given intervals for different types of data.
Now, I do have another unrelated question regarding system resources with all these cached topic consumers which I will post in another thread.
Thanks again for your help!