1 Reply Latest reply on Feb 23, 2018 3:30 PM by jbertram

update of messages in a queue/topic

pgiacome Feb 23, 2018 3:01 PM

Dear all,

I am working on a fork of HornetQ, that will allow to avoid any message duplication inside a queue.

The problem to solve is the following one:

I send messages to a queue but before sending into the queue I would like to query the queue to find if a message with the same content already exists inside the queue and it has not been consumed by some of the consumers..

Speaking in other terms the producer before sending the message in the queue it need first to check if in the queue there are messages with the same content already present. In such case it won't insert the message on the queue.

Any ideas on how to implement this?

Many thanks

1. Re: update of messages in a queue/topic

jbertram Feb 23, 2018 3:30 PM (in response to pgiacome)

Off the top of my head I think you could implement this with an in-memory data structure that holds a checksum of the data you want to compare as well as the corresponding ID for each message. When a message is sent to the broker you could perform a checksum on the incoming message and then ensure that checksum doesn't exist in the data structure. Also, when messages are acknowledge you can remove the corresponding entry from the data structure (which is why you would need to track the message ID and not just the checksum).

Keep in mind that this could have quite a significant negative impact on performance. As the queue grows in size the checksum comparison will take increasingly long and the data structure will occupy more & more memory.

It's also worth noting that the HornetQ code-base has not been under active development in several years since it was donated to the Apache ActiveMQ community. The code-base is continuing now as the ActiveMQ Artemis broker. If you're going to fork something I recommend you fork Artemis. Most of the internals are the same.
Actions