The metric data in the database is "compressed" and purged for exactly the reasons you describe. As the metric data gets "older", the datapoints are "collapsed" into 1hour, 6hour and 24hour "buckets" (each bucket maintaining only the min, max and average of the original data). This way we limit the raw size of the data to store. Additionally  we purge metric data older than 2 years (i think).

Also, because you mention "near real-time monitoring" - I'd like to urge you to do very rigorous testing of your deployment before you go live with it. Even though we try hard to be as efficient when storing metric data into the database as possible, it has been a known performance bottleneck of RHQ. So you have to weigh the amount of metric data coming into the database with the size of your inventory and actual functional requirements you have on the monitoring.

Hope this helps,

Lukas

That is a very large installation.  As Lukas mentions, you should definitely stage this and add in agents incrementally.  It will require a significant database backend and tight control over the metrics being collected.  Note that RHQ will not allow metric collection at intervals less than 30s, so that may affect your need for near-real time monitoring.

For that number of clients you will likely want several servers, look in the documentation for High Availability configurations.  Note that multiple servers will still funnel all data to a single backend database, so it must be configured for adequate numbers of connections and with tablespaces adequate for the load.

You can customize purge times to some degree.  Look in Administration system settings.

I am not sure there are any RHQ installations with that number of agents, perhaps anyone else with a large installation can comment.