Sparsely-connected dynamic network of brokers

[gtamboise]

Hello,

Has anybody tried to use ActiveMQ / Fuse Message Broker in an environment with

-> A network of 100+ brokers, where only ~10 of them "see each other" at any point in time. You can think of it as a set of ad-hoc mobile networks where nodes occasionally jump between networks.

-> Each broker serves a couple of consumers local to the node.

-> Brokers use a mix of few static and mostly dynamic discovery (multicast infrastructure is already in place)

-> Topic subscriptions are typically durable and every broker has local consumers potentially interested in every topic.

Or am I stretching the concept a bit?

[garytully]

it may be a bit of a stretch, but it is not impossible.

The main problem with large networks is the overhead of advisories, because advisory messages are the way the brokers become aware of demand (new consumers) and destinations. If the consumers are dynamic, the additional overhead of informing a large network can be significant.

If the network never grows to the full 100 and the consumers and destinations are relatively static, it will work ok.

Durable subscriptions in a network are not completely faithful to message order. There is currently no way to migrate a subscription from one broker to another. So it is possible to get out of order messages (and large gaps) if the durable sub bounces around the network.

You may want to consider using virtual topics and networking the consumer destinations.