Destination statistics

[alexfu.novell]

I'm working on JBMESSAGING-122 (Implement all the statistics MBean operations on topics and queues) and have the following proposal:

Functionalities:
1. Statistics can be enabled/disabled dynamically.
2. Statistics report for now is a message throughput table (based on sample rate, which decides how many data will be saved in the table). The readable report will look like (sample rate set to 1 (per hour)):

 11AM 12PM 1PM
MsgRate 300msg/s 10msg/s 0msg/s

Later we can add other measurement like failure rate in.

Two attributes and one operation are added to MBean:
set/get enableStatistics
set/get sampleRate
getStatisticsReportAsText

Implementation:
I was thinking creating wrapper statistics class (or a proxy) for core topic/queue so instead of creating topic/queue directly by

new Queue(id, ms, pm, true, fullSize, pageSize, downCacheSize);

we use a factory to instantiate either the core or the wrapper class based on configuration. However, this approach doesn't allow enabling/disabling dynamically. Also it will cause a lot of code changes.

Another thought is to use dynamic AOP but I'm not sure about it.

So, maybe I just hardcode it for now and later switch to other approach if needed. It may affect the performance somehow even the statistics is off, e.g. in org.jboss.messaging.core.local.Queue

 public Delivery handle(DeliveryObserver sender, Routable r, Transaction tx)
 {
 synchronized (m_statistics)
 {
 if (m_statistics.enabled())
 {
 m_statistics.handle(r); // do the measurement
 }
 }
 return super.handle(sender, r, tx);
 }

Suggestions/thoughts are really appreciated.

Thanks,
-Alex

[genman]

http://jira.jboss.com/jira/browse/JBAS-2325

I know thi s is for MDBs, hence at a higher level than JMS, though the metrics might be shared ... I was told that JSR77 is the right way to expose this sort of data.

http://www.jboss.org/index.html?module=bb&op=viewtopic&p=3898820

[ovidiu.feodorov]

Good point. JSR77.

[timfox]

Other than JSR77, some other points:

"AlexFu.Novell" wrote:
I'm working on JBMESSAGING-122 (Implement all the statistics MBean operations on topics and queues) and have the following proposal:

Functionalities:
1. Statistics can be enabled/disabled dynamically.
2. Statistics report for now is a message throughput table (based on sample rate, which decides how many data will be saved in the table). The readable report will look like (sample rate set to 1 (per hour)):

 11AM 12PM 1PM
MsgRate 300msg/s 10msg/s 0msg/s

Good stats would be messages received/sec and messages consumed/sec

Later we can add other measurement like failure rate in.

Not sure what you mean by failure rate.

Two attributes and one operation are added to MBean:
set/get enableStatistics
set/get sampleRate
getStatisticsReportAsText

Implementation:
I was thinking creating wrapper statistics class (or a proxy) for core topic/queue so instead of creating topic/queue directly by

new Queue(id, ms, pm, true, fullSize, pageSize, downCacheSize);

we use a factory to instantiate either the core or the wrapper class based on configuration. However, this approach doesn't allow enabling/disabling dynamically. Also it will cause a lot of code changes.

Another thought is to use dynamic AOP but I'm not sure about it.

So, maybe I just hardcode it for now and later switch to other approach if needed. It may affect the performance somehow even the statistics is off, e.g. in org.jboss.messaging.core.local.Queue

 public Delivery handle(DeliveryObserver sender, Routable r, Transaction tx)
 {
 synchronized (m_statistics)
 {
 if (m_statistics.enabled())
 {
 m_statistics.handle(r); // do the measurement
 }
 }
 return super.handle(sender, r, tx);
 }

Suggestions/thoughts are really appreciated.

Seems like a bad design smell. The core channel should not have a dependency on the statistics "manager" (however it is implemented).

The statistics "manager" should call operations on the queue like "getMessageCount" or whatever else it needs at certain intervals (the "sampling period") and from that it can work out the statistics.

[alexfu.novell]

Good stats would be messages received/sec and messages consumed/sec

I agree.

Not sure what you mean by failure rate.

I mean for any reason the message is not deliveried (due to consumer unregistered, tx rolls back, etc.)

Seems like a bad design smell. The core channel should not have a dependency on the statistics "manager" (however it is implemented).
The statistics "manager" should call operations on the queue like "getMessageCount" or whatever else it needs at certain intervals (the "sampling period") and from that it can work out the statistics.

I agree. Ideally, statistics manager should call queue/topic to retrieve data periodly. However, some data are not collected by the current queue/topic. e.g. getMessageCount() can only tell how many messages in the queue/topic. You won't know how many messages are in during the sampling period until you collect the data in handle().
m_statistics is only a lightweight statistics data collector (or called manager).

Thanks,
-Alex

[peterj]

Please also see http://www.jboss.org/index.html?module=bb&op=viewtopic&t=71908 for one user's opinion on message statistics.

Also, I get the impression that the design is leaning towards formatted output. Please don't because that makes it more difficult for the admin console to provide a consistent look. Instead, returning the unformatted data (as a collection or a data object) would be better. (I have no objection to providing both formatted and unformatted output.)

[alexfu.novell]

"PeterJ" wrote:
Please also see http://www.jboss.org/index.html?module=bb&op=viewtopic&t=71908 for one user's opinion on message statistics.

Also, I get the impression that the design is leaning towards formatted output. Please don't because that makes it more difficult for the admin console to provide a consistent look. Instead, returning the unformatted data (as a collection or a data object) would be better. (I have no objection to providing both formatted and unformatted output.)

Peter, both formatted and raw output will be provided. For the raw data, is there any special data structure requirement from admin console?

[peterj]

Alex, there are no special data structure requirements for the raw output. Just as long as we don't have to parse html to get at the data we'll be happy.

Thanks for asking.

[timfox]

"AlexFu.Novell" wrote:

I agree. Ideally, statistics manager should call queue/topic to retrieve data periodly. However, some data are not collected by the current queue/topic. e.g. getMessageCount() can only tell how many messages in the queue/topic. You won't know how many messages are in during the sampling period until you collect the data in handle().

I don't think handle is the best place to do this. The counts should be maintained in the channel state.

If you keep a count of the messages added and another separate count of the message removed in the channel state. Then you can prevent having to lock a single counter in adding and receiving threads which would result in lock contention.

If the external sampler then calls into the channel state with period t, then you can produce statistics for messages/added sec messages/removed per sec etc

Key point here is not to calculate the statistics in the channel, rather to make sure enough information is provided so something *else* can calcuate the statistics.