Reliable delivery and JobManagement with Persistent Queues

[smjain]

I have few more questions ...Hope you can answer

So we want to introduce a notion of job management where each job is composed of processing code behind multiple listeners

As an example

Say we have 2 processors(listeners) each behind a different queue..Now the job is completed when each listener executed successfully.

If say one listener execution fails(it will have a retry mechanism) and finally abort if retry fails..When we abort we push message to a DLQ(Dea letter Queue)

Now from a UI which we will develop we should be able to monitor jobs(I am thinking message itself is the job as it contains the execution context itself)

So if say a job has two execution points(A and B ) and B has failed..From Ui we should be manaully able to restart execution from B itself..

In that case message will be pulled from the DLQ and pushed to the Queue which has listener for processing action B..

I think by using this architecture we achieve reliability as well as dynamic behaviour by introducing more processors and wiring them at a layer above.

Is this a good/valid idea or do u suggest anything else..

Thanks and Regards

Shashank

We want to introduce a notion of job management where each job is composed of processing code behind multiple listeners

As an example

Say we have 2 processors(listeners) each behind a different queue..Now the job is completed when each listener executed successfully.

If say one listener execution fails(it will have a retry mechanism) and finally abort if retry fails..When we abort we push message to a DLQ(Dead letter Queue)

Now from a UI which we will develop we should be able to monitor jobs(I am thinking message itself is the job as it contains the execution context itself)

So if say a job has two execution points(A and B ) and B has failed..From Ui we should be manaully able to restart execution from B itself..

In that case message will be pulled from the DLQ and pushed to the Queue which has listener for processing action B..

I think by using this architecture we achieve reliability as well as dynamic behaviour by introducing more processors and wiring them at a layer above.

Is it possible to query the Dead letter queue to get a hold of failed messages based on certain attributes..If that is possible we can push them back for redelivery..

Thanks and Regards

Shashank

[jaikiran]

Shashank Jain wrote:

I have few more questions ...Hope you can answer

So we want to introduce a notion of job management where each job is composed of processing code behind multiple listeners

As an example

Say we have 2 processors(listeners) each behind a different queue..Now the job is completed when each listener executed successfully.

If say one listener execution fails(it will have a retry mechanism) and finally abort if retry fails..When we abort we push message to a DLQ(Dea letter Queue)

Now from a UI which we will develop we should be able to monitor jobs(I am thinking message itself is the job as it contains the execution context itself)

So if say a job has two execution points(A and B ) and B has failed..From Ui we should be manaully able to restart execution from B itself..

In that case message will be pulled from the DLQ and pushed to the Queue which has listener for processing action B..

I think by using this architecture we achieve reliability as well as dynamic behaviour by introducing more processors and wiring them at a layer above.

Is this a good/valid idea or do u suggest anything else..

Thanks and Regards

Shashank

Is it possible to query the Dead letter queue to get a hold of failed messages based on certain attributes..

Would a listener with a message selector on the DLQ help?

[smjain]

So you mean whenever we want to drain the message we can use the selector mechanism on consumer to pull the message we are looking for.

I think thats a good suggestion and should work..

Once that is pulled we can send it back to the intended queue for further processing..

Thanks

Shashank

[smjain]

I probably forgot to put this.. We might need to query the queue on certain attributes. Will that be possible without actually draining the message off.