Split brain recovery/merge (standalone / symmetric cluster)

[franksmash]

Hey folks, I have a I've got an interesting question.

We have a cluster 2.4.0.Final HQ standalone cluster consisting of 2 primaries and 2 backups. We've encountered a split brain problem and while it's clear how to avoid the problem in the future, we can't figure out how to recover from it without risk of message loss. Has anyone developed a procedure for this?

Thanks!

[jbertram]

My first thought would be to start both brokers in isolation and have a remote consumer that pulls the messages from both and performs some kind of de-duplication logic on them and then sends the messages to another broker.