Do we know how graph performance is with titan/C* vs. titan/ES? Same for metric storage/retrieval.
I agree that we should keep the number of backend datastores as small as possible, as otherwise the "management of the management" app will kill us.
I am not too familiar with Titan, but I am not sure that the usage of Cassandra and Elastic Search is mutually exclusive. Titan can use Cassandra as a primary data store, and can use Elastic Search as an index. This leads me to believe that we could have Titan/C*, Titan/ES/some_other_storage, or Titan/C*/ES.
With respect to sharing a Cassandra cluster, I think it is feasible and something to consider. I would expect the performance characteristics between rhq-metrics and rhq-inventory to be quite different as well. Compaction, compression, and row caching are configurable per table. Consistency is configurable per request. Data directories can be stored in different locations as well making it easier to utilize hardware better suited for different workloads. I think that the shared JVM heap is something to think about; however, most things have been moved off heap as you read about in http://www.datastax.com/dev/blog/off-heap-memtables-in-cassandra-2-1.