Adding multiple CallTimeData for one scheduleId

[fbrueseke]

This thread is migrated over from the RHQ-forum, which does not seem to work properly. Please read the thread over there for full details: http://forums.rhq-project.org/viewtopic.php?f=5&t=102&start=0&st=0&sk=t&sd=a

Summary:
The problem is that when you are developing an RHQ plugin to measure call time data (e.g. EJB response times) you are not able to include more than one CallTimeData object into the MeasurementReport per schedule id. The reason is that you cannot enter any additional objects as the Set used for storing the data thinks the object is already in. This behaviour is caused by a weak hashCode() method supplied by the CallTimeData class.

My findings are that the best fix for me would be to delete the overriden equals() and hashCode() methods. If that is not an option in RHQ it is best to refine those methods, e.g. with the following fix:

49,50d48
< private long firstBeginTime = Long.MAX_VALUE;
<
133,135d130
< if (beginTime.getTime() < this.firstBeginTime)
< firstBeginTime = beginTime.getTime();
<
156c151
< return (this.scheduleId == other.scheduleId && this.firstBeginTime == other.firstBeginTime);
---
> return (this.scheduleId == other.scheduleId);
164d158
< result = (prime * result) + (int) (firstBeginTime ^ (firstBeginTime >>> 32));

Essentially this change introduces a firstBeginTime variable that is set on data being inserted reflecting the earliest beginTime entered. This variable is then considered when checking for equality and when computing the hash code.

What are your thoughts on this proposal?

[mazz]

You would have to also look on the server-side to see how its processed, to see if it can work with the new object you created.

I thought the way the calltime data worked was you added individual call time metrics and that object aggregated it for you (as you added things to it, it would calculate the min/max/avg).

Ian would understand more about the impact of your changes - I'll let him chime in.

[ips]

Hi,

The limit of one CallTimeData object per schedule per measurement report is by design. To add multiple call times, for each call time, call the addCallData() method on your CallTimeData object. An example of this can be seen in ResponseTimeLogParser.parseLog() which reads a response time log file line by line and calls addCallData() for each line of the file.

In other cases, where you have call time data that has already been aggregated, the addAggregatedCallData() method should be used instead of addCallData().

-Ian

[fbrueseke]

Hi,

perhaps it helps if I explain a bit more what I'm trying to achieve. We want to measure response times with a pretty high resolution, i.e. 100ms to 1sec. To do so we gather the data in the observed application. It already aggregates the data and creates a new statistics whenever the resolution time span has exceeded.

To get this data to the agent we let the agent fetch the whole set of statistics as a block. The agent then repackages everything and forwards the measured data to the RHQ server.

There are three reasons why we do not let the agent directly get the statistics:
- First "getValues" is called at most every 30 seconds which does not allow us to gather data with the needed resolution.
- Second we need to instrument the measured application anyways. So this code can just as well aggregate data.
- Third we save roundtrips between agent and measured application this way.

That being said I think that the most effective solution to that problem is to include more than one CallTimeData object in the MeasurementReport.

My other options are:
- Aggregate all in one CallTimeData object => 30s resolution :(
- Change the metrics to one or several numeric metrics.

What do you think is the best solution for this scenario?

[mazz]

The 30 seconds is just the collection interval when the calltime data it collected (and later sent to the server). The actual call time data can be anything. If the resolution of the calltime data is sub-second, that should be OK (that's how everything else works that uses in - for example, Apache response times for almost all URL requests are normally under 30 seconds).

Now, if you mean you want to COLLECT response times that fast (such that the agent collects the calltime data and sends it up to the server sub-second), that's too fast and would clobber both the agent and server process, spinning CPU and gobbling I/O.

So, I guess I still don't understand what you are doing. What do you mean "measure response times with a pretty high resolution"? Can you provide examples of the kind of data you are collecting and aggregating?

[fbrueseke]

"measure response times with a pretty high resolution" means that the aggregated data itself does not cover any more data than 100 milliseconds for example. So resolution means time for that data will aggregated.

So we get the following picture:
[measured program] <-- collection time 30s <-- [RHQ agent] <-- some interval <-- [RHQ server]
[internal: 100ms-1s]
^^ this is the resolution

As you have said collecting response times that fast would not work, hence my concept is to poll muliple CallTimeData objects in one go every 30 seconds. But as I've stated this does not work cause the measurement report does not support it.
Polling response time with the rate of the resolution time is just a possible (ugly) workaround.

Hope that helps
Frank

[fbrueseke]

As time goes by I found an appropriate workaround. Subclassing the CallTimeData class actually works. I include the code here so you have an example how this could work:

public class DistinctCallTimeData extends CallTimeData {

 private static final long serialVersionUID = 1L;

 private Long firstBeginTime = null;

 public DistinctCallTimeData(MeasurementScheduleRequest schedule) {
 super(schedule);
 }

 @Override
 public boolean equals(Object o) {
 if (this == o) {
 return true;
 }

 if ((o == null) || (!(o instanceof CallTimeData))) {
 return false;
 }

 final CallTimeData other = (CallTimeData) o;
 if (other instanceof DistinctCallTimeData)
 return (this.getScheduleId() == other.getScheduleId() && this.firstBeginTime == ((DistinctCallTimeData) other).firstBeginTime);
 else
 return (this.getScheduleId() == other.getScheduleId());
 }

 @Override
 public int hashCode() {
 final int prime = 31;
 int result = 1;
 result = prime * result + this.getScheduleId();
 result = (prime * result) + (int) (firstBeginTime ^ (firstBeginTime >>> 32));

 return result;
 }

 public Long getFirstBeginTime() {
 return firstBeginTime;
 }

 public void setFirstBeginTime(Long firstBeginTime) {
 this.firstBeginTime = firstBeginTime;
 }
}

Please think about if the restriction of having only one CallTimeData object per MeasurementReport and scheduleId is right.

As I have stated in the case that you want to gather a set of aggregated measurement data object collected at a rate faster you want to collect data from the measured application. So if you want to decouple your "resolution" or "sampling rate" from your collection rate it is essential to be able to add more than one CallTimeData object per MeasurementReport.

As in the example above one CallTimeData per MeasurementReport, scheduleId and timespan would be much more appropriate.

I hope you consider this in a next version of RHQ.