I am trying to understand how redistribution of messages works and what it really means. Basically i want to run a cluster without backup nodes, where all nodes can receive messages, i.e. all are live nodes (while listening on a queue with the same name on each of the nodes). I thought, when one of the nodes dies while processing a message, the message would be resend to another node .
To test this behavious, i made a cluster of two JBoss EAP 6.1 nodes (domain mode) and sent some messages to them. All fine, both of the nodes could process messages in their MDBs. Then i just killed one of the JBoss nodes while it was processing a message (i put a wait statement in the processing part of the code to be sure i was killing the node before the message could be processed/the transaction could commit).
The other node noticed the dead of the killed node and finally cleared the resources:
[org.hornetq.core.server] (Thread-4 (HornetQ-client-global-threads-1253674008)) HQ222095: Connection failed with failedOver=false: HornetQException[errorType=NOT_CONNECTED message=HQ119006: Channel disconnected]
[org.hornetq.core.server] (hornetq-failure-check-thread) HQ222061: Client connection failed, clearing up resources for session 62512bf7-4068-11e3-93ce-11e98f587942
[org.hornetq.core.server] (hornetq-failure-check-thread) HQ222107: Cleared up resources for session 62512bf7-4068-11e3-93ce-11e98f587942
But then nothing happened, the message was not delivered to the active node. Only when i did a restart of the whole cluster, the message was redelivered (i used a durable queue).
So my question is: Is there a way to have this kind of high availability without a backup node, just with running instances that actually are doing something? Or is this really a failover scenario, which is only possible with backup nodes? But what is meant by redistribution in that case?
Additionally, are those backup nodes (in case of JBoss cluster server instances) just backup nodes for hornetq, or backup nodes for all other resources, too (e.g. stateless session beans etc)? I mean, is a backup node actually doing anything except waiting until the live node is not working anymore?
Thank you for your answers.
redistribution means redistributing messages from one node to another when it has no consumers. for node failure you need backup servers