Is there a way for us to create a subscription based on an IN clause like filter?

[connie.yang]

Is there way for us to create a subscription based on an in-clause like message filter without sacrificing performance and such in HornetQ?

Here's my use case, our consumers want to create a subscription to receive messages that are associated/tagged with a set of predefined unique numeric ids which don't belong to a specific range or pattern.  Also, the filter set can be big (in the order of thousands).

My first thought was to use an IN clause like filter, but not sure if this is supported in HornetQ, or rather SQL92.  After thinking through a bit, I thought about having our consumer to create multiple subscriptions and sort the filter set.  Then, the subscriptions can be filtered on a specific range of numbers.

Is there better way for doing this?

Thanks in advance.

[jbertram]

The documentation states:

HornetQ provides a powerful filter language based on a subset of the SQL 92 expression syntax.

It is the same as the syntax used for JMS selectors, but the predefined identifiers are different. For documentation on JMS selector syntax please the JMS javadoc for javax.jms.Message.

The javax.jms.Message JavaDoc states:

A selector can contain:

...

• identifier [NOT] IN (string-literal1, string-literal2,...) (comparison operator where identifier has a String or NULL value)
  • "Country IN (' UK', 'US', 'France')" is true for 'UK' and false for 'Peru'; it is equivalent to the expression "(Country = ' UK') OR (Country = ' US') OR (Country = ' France')"
  • "Country NOT IN (' UK', 'US', 'France')" is false for 'UK' and true for 'Peru'; it is equivalent to the expression "NOT ((Country = ' UK') OR (Country = ' US') OR (Country = ' France'))"
  • If identifier of an IN or NOT IN operation is NULL, the value of the operation is unknown.

Therefore an "IN" clause should work.

[connie.yang]

Thanks for your reply!

Are you aware of any performance metrics/concern if the IN clause contains 1 million elements?

[jbertram]

No, I am not aware of any performance metrics involving an IN clause which contains 1 million elements.  Give it a try and let us know how it goes.

[noky]

Actually, there is an issue with large selectors like this. I reported a bug in 2010, selector strings over 4096 characters fail:

https://issues.jboss.org/browse/HORNETQ-545

Any chance of getting this fixed?

[jbertram]

It's fixed now.  It will be available in the next release.

[connie.yang]

Great!  With the fix, what's the maximum number of selector we can have without causing an issue?

Also, which release will this fix be available?

[clebert.suconic]

Creating selectors on durable subscriptions shouldn't be an issue... however if you have a really huge queue on create a Consumer on a queue, then you will be doing a scan loop for the selector that can be somewhat unperformant. (if you use too many scans like these)