Metrics aggregation - how to support aggregation of old data

[genman]

I have metrics data that is using a timestamp that may be a few hours old. The agent resource is gathering data from a reporting database that has older data, not of the current hour.

The problem is that the aggregation routine seems to ignore data that's not of the current past hour. Obviously, this may be outside of the scope of the design for the aggregation system, but might be something worth supporting, if it is easy to support this in the code.

My initial thought is that it isn't easy to support. The reason is that the data insert time would need to be tracked and this would mean a change in schema, etc.

[john.sanda]

You observation is correct. In the current implementation, we do not aggregate past data. For example, suppose the current time is 15:20. The server receives a measurement report with data having timestamps from the 13:00 hour. We will persist those metrics, but they won't get aggregated. That's the bad news. The good news is that we will handle this in RHQ 4.12. The changes are described at https://docs.jboss.org/author/display/RHQ/Aggregation+Schema+Changes.

[genman]

Does it have some sort of performance impact if this happens? I'm wondering if I have late collections (all the time) for a schedule that it may make aggregation take longer. I'm guessing not, but just confirming.

[john.sanda]

The late collections will make the aggregation take longer simply because this data would not have been aggregated at all previously. We only aggregate the late data for those measurement schedules reporting late data. The performance impact therefore depends on the number of measurement schedules having late data.