Clustered environment with non persistent messages ?

[kemp40]

Hi,
We are running under JBoss 4.0.5 using JBossMQ as a messaging engine. 0ur platform is based on very extensive messaging - there are about 400 producers multiplexing jms messages to about 400 consumers which listen on bind queues (queue per consumer). There are no selectors used due to performance tip (http://wiki.jboss.org/wiki/Wiki.jsp?page=JBMPerf point 5.) - routing is done on the application level. The messages also must be persisted – data loss is unacceptable.
Because our traffic load increases constantly we are slowly just about to lean against the performance wall. So we are seriously thinking about switching to JBossMessaging(1.4) to benefit from performance enhancements but also, which is more important we want to avoid persisting messages – because as every child knows - persistence is a performance killer for JMS. In order to avoid data loss we are thinking of configuring clustered environment with distributed queues – thanks to that while one node crushes enqueued messages are still available. Surly if all nodes crush the data loss is inevitable – but we can cope with that … but the target is bright : performance.

So after this long introduction the questions are:
+ Is it possible to configure clustered environment with not-persistent messages ?
+ What happens with the messages when one of nodes crushes – can we avoid data loss ?

Thanks in advance for any help !

[mvecera]

Hello,

I'm afraid it is not very safe to have non-persistent messages here. The clustered queue shares the messages in database. If the messages aren't persistent and one node in the cluster has a message ready for processing, the message can be lost in the case of failure.

But I would suggest you first switch to JBoss Messaging 1.4.0.SP3 and ideally a newer JBoss AS (4.2)...

[kemp40]

That's bad news.

Anyway, as you said - I'm gonna first switch to JBM1.4 + JBOSS4.2, do some real - life stress tests, compare to actual deployment configuration and we'll see.

Btw, wouldn't be a really cool idea to replicate non-persistent messages among active nodes to gain performance, it's really unlikely to have all nodes crush ... ;)

Thanks for answer !

[timfox]

"kemp40" wrote:

Btw, wouldn't be a really cool idea to replicate non-persistent messages among active nodes to gain performance, it's really unlikely to have all nodes crush ... ;)

Hello kemp,

Martin's comment was not correct, JBoss Mesaging *does* cluster non persistent messages too :)

[kemp40]

Tim,
That's exactly what I wanted to hear.
I'm just about to configure such environment - I'll report users about the test results in the comparison with current environment:

JBossMQ+JBoss4.0.5 (persistent msg) VS. JBM+JBoss4.2(clustered + non-persistent msg)

Much appreciated,

Kemp

[mvecera]

Hello,

I didn't say that it doesn't support clustering of non-persistent messages. It is possible to cluster them but the message can be lost! kemp40 said that this is unacceptable to him...

[timfox]

The point with non persistent messages, is they are not persisted, ==> they do not survive failure.

Users use non persistent messages when they want higher performance but they can cope with message loss.

If we had to persist non persistent messages too, that would kind of defeat the purpose for their existence! :)

E.g. a stock ticker application would probably use non persistent messages, since market data goes out of date very quickly (sub second) and later messages can supercede earlier messages so you don't care if you lose messages.

If we didn't have non persistent messages and only had persistent messages then we would be losing performance for applications that don't need the reliability

It would be silly if we persisted non persistent messages too - since they wouldn't be non persistent any more.

The JMS spec is quite clear on this.