Improving message selector performance

[adrian.brock]

It should be made optional (possibly per queue)
that the message header (jms and user properties) is kept in memory.

This will avoid having to load messages from the persistent cache
to run the selector inside the synchornized block.
This is at the expense of using more memory.

It can be implemented by keeping a hard reference to the header in the
MessageReference.

Regards,
Adrian

[adrian.brock]

Need to implement optimized access for a message selector per destination:
e.g.

 <mbean code="org.jboss.mq.server.jmx.Queue"
 name="jboss.mq.destination:service=Queue,name=A">
 <optimized-selector>JMSMessageID=?</optimized-selector>
 <depends optional-attribute-name="DestinationManager">jboss.mq:service=DestinationManager</depends>
 </mbean>

See also this thread from jbossmq-user
http://www.jboss.org/index.html?module=bb&op=viewtopic&t=48585

[genman]

A "cheap" way to improve performance would be to run the selector against messages added to the queue, add references to these messages into a hashtable (key would be the selector, value would be the sorted message IDs). When reading using a selector, if it matches a pre-existing optimized selector exactly, then the references would be pulled from the selector hashtable.

One nice benefit is the selector is only evaluated once.

A more sophisticated solution would include indexing each of the properties separately and using a tree.

[adrian.brock]

Yes, but only use maintain the hashtable/hashmap if there is an optimized-selector
and the receiver's selector matches the optimized selector.

Care to implement it?

[stephanenicoll]

I think that in some cases, more than one selector might be necessary (that is be able to specify multiple selectors and have some merge on the headers).

I can help if necessary but we need to coordinate.

Regards,

Stephane

[stephanenicoll]

Genman,

With regard to your proposal, what's happenning on server's restart? Is the hashtable rebuilt at startup?

Regards,

Stephane

[adrian.brock]

Supporting multiple selectors is too complicated for an initial version.

Rebuilding the HashMap (don't use a hashtable or vector or any collection that does synchronization unless you are lazy - often you are already synchronized on some other object) should be optimized in conjunction with this work.
http://www.jboss.org/index.html?module=bb&op=viewtopic&t=46186
where you would have to the optimized selector's value in a db column with an
index over it.

[stephanenicoll]

Adrian,

Had a look to JBossMQ code today. Looks like the place to be for the receiver side would be:

- public SpyMessage receive(...) in BasicQueue

(what about topics?)

For the hashamp, it should be linked to the Destination I assume. What about a Linked List as value of the hasmap?

I don't know what happen when a message is reposted (consider as a new one?) but we need to interecept changes in the headers values.

Regards,

Stephane

[stephanenicoll]

Or BasicQueueParameters maybe could contain the optimization.

What do you think?

Regards,

Stephane

[adrian.brock]

1) BasicQueueParameters is where the configuration should go,
setup in org.jboss.mq.server.jmx.Queue

2) It is irrelevent for Topics. If a topic subscription has a selector, messages that
do not match the selector are dropped. BasicQueue (or its subclasses) are used for
both Queues and Topic Subscriptions

3) Messages are "received" in
receive() - for MessageReceiver.receive[NoWait]()
addReceiver() - for MessageListeners
internalAddMessage() - for MessageReceiver.receive() and MessageListeners
when no message was available at the time but they are prepared to wait

4) The "HashMap" should be maintained in internalAddMessage and you
can probably extend the meaning of clearEvent(Message) to
tidyup(Message) for tidying it up the hash map along with any scheduled event.

5) What do you mean by reposted? The only complication you need to consider
is the RestoreMessageTask where a nacked message is setup for redelivery.
This changes some of properties which might be in the selector.