As for (1), performance is not good but seems good enough once inventory is imported and there aren't a lot of inventory or availability changes. There basically needs to be optimizations to fetching data so that Hibernate does not do N+1 selects.
I've played around with batch sizes and whatnot, but I'm not sure how much it helps.
(2) I suppose can be solved once RHQ supports NAT connections. Basically everything would be fine if RHQ agents connected to a specific server and the servers sent messages through a message bus.
(3) I've gotten multiple DCs to work with Cassandra. It requires this patch: https://bugzilla.redhat.com/show_bug.cgi?id=1033943...Otherwise, inserting metrics can be too slow, causing agents to time out. If the ping time from one DC to another is 100ms, then inserting 1000 metrics at a time then takes 200 seconds, give or take. Locally, 200 metric inserts happen in under a second.