8 Replies Latest reply on Apr 11, 2006 4:58 AM by Tim Fox

    Destination statistics

    Alex Fu Newbie

      I'm working on JBMESSAGING-122 (Implement all the statistics MBean operations on topics and queues) and have the following proposal:

      Functionalities:
      1. Statistics can be enabled/disabled dynamically.
      2. Statistics report for now is a message throughput table (based on sample rate, which decides how many data will be saved in the table). The readable report will look like (sample rate set to 1 (per hour)):

       11AM 12PM 1PM
      MsgRate 300msg/s 10msg/s 0msg/s
      

      Later we can add other measurement like failure rate in.

      Two attributes and one operation are added to MBean:
      set/get enableStatistics
      set/get sampleRate
      getStatisticsReportAsText

      Implementation:
      I was thinking creating wrapper statistics class (or a proxy) for core topic/queue so instead of creating topic/queue directly by
      new Queue(id, ms, pm, true, fullSize, pageSize, downCacheSize);
      we use a factory to instantiate either the core or the wrapper class based on configuration. However, this approach doesn't allow enabling/disabling dynamically. Also it will cause a lot of code changes.

      Another thought is to use dynamic AOP but I'm not sure about it.

      So, maybe I just hardcode it for now and later switch to other approach if needed. It may affect the performance somehow even the statistics is off, e.g. in org.jboss.messaging.core.local.Queue
       public Delivery handle(DeliveryObserver sender, Routable r, Transaction tx)
       {
       synchronized (m_statistics)
       {
       if (m_statistics.enabled())
       {
       m_statistics.handle(r); // do the measurement
       }
       }
       return super.handle(sender, r, tx);
       }
      


      Suggestions/thoughts are really appreciated.

      Thanks,
      -Alex

        • 1. Re: Destination statistics
          Elias Ross Master


          http://jira.jboss.com/jira/browse/JBAS-2325

          I know thi s is for MDBs, hence at a higher level than JMS, though the metrics might be shared ... I was told that JSR77 is the right way to expose this sort of data.

          http://www.jboss.org/index.html?module=bb&op=viewtopic&p=3898820

          • 2. Re: Destination statistics
            Ovidiu Feodorov Master

            Good point. JSR77.

            • 3. Re: Destination statistics
              Tim Fox Master

              Other than JSR77, some other points:

              "AlexFu.Novell" wrote:
              I'm working on JBMESSAGING-122 (Implement all the statistics MBean operations on topics and queues) and have the following proposal:

              Functionalities:
              1. Statistics can be enabled/disabled dynamically.
              2. Statistics report for now is a message throughput table (based on sample rate, which decides how many data will be saved in the table). The readable report will look like (sample rate set to 1 (per hour)):
               11AM 12PM 1PM
              MsgRate 300msg/s 10msg/s 0msg/s
              



              Good stats would be messages received/sec and messages consumed/sec



              Later we can add other measurement like failure rate in.



              Not sure what you mean by failure rate.



              Two attributes and one operation are added to MBean:
              set/get enableStatistics
              set/get sampleRate
              getStatisticsReportAsText

              Implementation:
              I was thinking creating wrapper statistics class (or a proxy) for core topic/queue so instead of creating topic/queue directly by
              new Queue(id, ms, pm, true, fullSize, pageSize, downCacheSize);
              we use a factory to instantiate either the core or the wrapper class based on configuration. However, this approach doesn't allow enabling/disabling dynamically. Also it will cause a lot of code changes.

              Another thought is to use dynamic AOP but I'm not sure about it.

              So, maybe I just hardcode it for now and later switch to other approach if needed. It may affect the performance somehow even the statistics is off, e.g. in org.jboss.messaging.core.local.Queue
               public Delivery handle(DeliveryObserver sender, Routable r, Transaction tx)
               {
               synchronized (m_statistics)
               {
               if (m_statistics.enabled())
               {
               m_statistics.handle(r); // do the measurement
               }
               }
               return super.handle(sender, r, tx);
               }
              


              Suggestions/thoughts are really appreciated.



              Seems like a bad design smell. The core channel should not have a dependency on the statistics "manager" (however it is implemented).

              The statistics "manager" should call operations on the queue like "getMessageCount" or whatever else it needs at certain intervals (the "sampling period") and from that it can work out the statistics.

              • 4. Re: Destination statistics
                Alex Fu Newbie

                 


                Good stats would be messages received/sec and messages consumed/sec

                I agree.

                Not sure what you mean by failure rate.

                I mean for any reason the message is not deliveried (due to consumer unregistered, tx rolls back, etc.)

                Seems like a bad design smell. The core channel should not have a dependency on the statistics "manager" (however it is implemented).
                The statistics "manager" should call operations on the queue like "getMessageCount" or whatever else it needs at certain intervals (the "sampling period") and from that it can work out the statistics.

                I agree. Ideally, statistics manager should call queue/topic to retrieve data periodly. However, some data are not collected by the current queue/topic. e.g. getMessageCount() can only tell how many messages in the queue/topic. You won't know how many messages are in during the sampling period until you collect the data in handle().
                m_statistics is only a lightweight statistics data collector (or called manager).

                Thanks,
                -Alex

                • 5. Re: Destination statistics
                  Peter Johnson Master

                  Please also see http://www.jboss.org/index.html?module=bb&op=viewtopic&t=71908 for one user's opinion on message statistics.

                  Also, I get the impression that the design is leaning towards formatted output. Please don't because that makes it more difficult for the admin console to provide a consistent look. Instead, returning the unformatted data (as a collection or a data object) would be better. (I have no objection to providing both formatted and unformatted output.)

                  • 6. Re: Destination statistics
                    Alex Fu Newbie

                     

                    "PeterJ" wrote:
                    Please also see http://www.jboss.org/index.html?module=bb&op=viewtopic&t=71908 for one user's opinion on message statistics.

                    Also, I get the impression that the design is leaning towards formatted output. Please don't because that makes it more difficult for the admin console to provide a consistent look. Instead, returning the unformatted data (as a collection or a data object) would be better. (I have no objection to providing both formatted and unformatted output.)


                    Peter, both formatted and raw output will be provided. For the raw data, is there any special data structure requirement from admin console?

                    • 7. Re: Destination statistics
                      Peter Johnson Master

                      Alex, there are no special data structure requirements for the raw output. Just as long as we don't have to parse html to get at the data we'll be happy.

                      Thanks for asking.

                      • 8. Re: Destination statistics
                        Tim Fox Master

                         

                        "AlexFu.Novell" wrote:

                        I agree. Ideally, statistics manager should call queue/topic to retrieve data periodly. However, some data are not collected by the current queue/topic. e.g. getMessageCount() can only tell how many messages in the queue/topic. You won't know how many messages are in during the sampling period until you collect the data in handle().


                        I don't think handle is the best place to do this. The counts should be maintained in the channel state.

                        If you keep a count of the messages added and another separate count of the message removed in the channel state. Then you can prevent having to lock a single counter in adding and receiving threads which would result in lock contention.

                        If the external sampler then calls into the channel state with period t, then you can produce statistics for messages/added sec messages/removed per sec etc

                        Key point here is not to calculate the statistics in the channel, rather to make sure enough information is provided so something *else* can calcuate the statistics.