10 Replies Latest reply on Oct 28, 2011 6:15 AM by Graham Hounslow

    Drift Management in RHQ 4.1

    Graham Hounslow Newbie

      Hopefully a simple question does it work or is it still really a beta feature?

       

      We've been looking at RHQ and quite like the drift management though it doesn't seem to work for us as though there are no errors when we set-up a change set no files seem to be show up.

       

      I've been trying to set it up to monitor the conf directory for our servers and have tried the various options but the files in /server/default/conf just don't show up,  I just don't get my version 0 baseline.

       

      The servers are running on Windows 2003 server currently using java wrapper to run as a windows service. So far everything else seems to work ok.

       

      If I do a detect now I can see it being triggered in the agent log

       

      2011-10-14 15:27:03,237 INFO  [WorkerThread#0[10.196.129.208:1160]] (rhq.core.pc.drift.DriftManager)- Received request to schedule drift detection immediately for [resourceId: 10734, driftConfigurationId: 10072, driftConfigurationName: Application Conf]

       

      and shortly afterwards a message - is this an unreleated message though?

       

      2011-10-14 15:27:26,581 INFO  [MeasurementManager.sender-1] (rhq.core.pc.measurement.MeasurementSenderRunner)- Measurement collection for [431] metrics took 1266ms - sending report to Server...

       

       

      Any advice or help would be much appreciated.

       

      regards


      Graham

        • 1. Re: Drift Management in RHQ 4.1
          John Sanda Apprentice

          Drift managent is stable in the RHQ 4.1 release; however, there is a lot of functionality planned for the feature that was not yet implemented for the 4.1 release. There are some some bugs that I believe were found right after after that release that you may be hitting. Have you verified that the base directory for your drift definition (formerly called drift configuration) does in fact exist on the agent machine? There is an outstanding issue where drift detection fails if the base directory does not exist. This will be fixed in the next release. You might also want to turn on debug logging for drift on the agent. You can do so by adding the following to <AGENT_HOME/conf/log4j.xml,

           

          <category name="org.rhq.core.pc.drift">

            <priority value="DEBUG"/>

          </category>

           

          Hope that helps.

           

          - John

          • 2. Re: Drift Management in RHQ 4.1
            Graham Hounslow Newbie

            Hi John, thanks for your answer.

             

            I'm not in front of the machine at the moment might seem a bit vague and/or confused. I've tried the default option for a monitored Jboss AS server, pluginconfiguration option which is set to jbosshome. I've also tried the filesystem option as well. Neither of which seemed to work. I've also tried it with back and forward slashes (c:\temp and c:/temp) just in case this caused a problem.

             

            I also changed to log level, though I didn't see anything wrong.

             

            I'll grab some screenshots on Monday and double check the log file settings as that should make it clearer what settings I'm trying.

             

            cheers

             

            Graham

            • 3. Re: Drift Management in RHQ 4.1
              Graham Hounslow Newbie

              After restarting the agent and triggering a "Detect Now" I'm now seeing an error message in the log files.

               

              Checking in the directory mentioned there are two empty files, changeset.txt and a dirft-changeset.txt.

               

              Log message

              2011-10-17 10:16:15,383 INFO  [WorkerThread#0[10.196.129.208:3677]] (rhq.core.pc.drift.DriftManager)- Received request to schedule drift detection immediately for [resourceId: 10743, driftConfigurationId: 10091, driftConfigurationName: Core Files]

              2011-10-17 10:16:15,383 DEBUG [WorkerThread#0[10.196.129.208:3677]] (rhq.core.pc.drift.DriftManager)- Resetting DriftDetectionSchedule[resourceId: 10743, driftConfigurationId: 0, driftConfigurationName: Core Files] for immediate detection.

              2011-10-17 10:16:15,383 DEBUG [WorkerThread#0[10.196.129.208:3677]] (rhq.core.pc.drift.DriftManager)- DriftDetectionSchedule[resourceId: 10743, driftConfigurationId: 0, driftConfigurationName: Core Files] has been added to ScheduleQueue[DriftDetectionSchedule[resourceId: 10743, driftConfigurationId: 0, driftConfigurationName: Core Files]] for immediate detection.

              2011-10-17 10:16:19,352 INFO  [MeasurementManager.sender-1] (rhq.core.pc.measurement.MeasurementSenderRunner)- Measurement collection for [14] metrics took 626ms - sending report to Server...

              2011-10-17 10:16:48,883 DEBUG [pool-3-thread-1] (rhq.core.pc.drift.DriftDetector)- Processing DriftDetectionSchedule[resourceId: 10743, driftConfigurationId: 0, driftConfigurationName: Core Files]

              2011-10-17 10:16:48,883 DEBUG [pool-3-thread-1] (rhq.core.pc.drift.DriftDetector)- Generating drift change set for DriftDetectionSchedule[resourceId: 10743, driftConfigurationId: 0, driftConfigurationName: Core Files]

              2011-10-17 10:16:48,883 ERROR [pool-3-thread-1] (org.rhq.common.drift.ChangeSetReaderImpl)- Unable to read headers from C:\rhq\rhq-agent\data\changesets\10743\Core Files\changeset.txt: null

              2011-10-17 10:16:48,883 ERROR [pool-3-thread-1] (rhq.core.pc.drift.DriftDetector)- An error occurred while scanning for drift

              org.rhq.common.drift.ChangeSetReaderException: Unable to read headers from C:\rhq\rhq-agent\data\changesets\10743\Core Files\changeset.txt

                        at org.rhq.common.drift.ChangeSetReaderImpl.<init>(ChangeSetReaderImpl.java:70)

                        at org.rhq.core.pc.drift.ChangeSetManagerImpl.getChangeSetReader(ChangeSetManagerImpl.java:96)

                        at org.rhq.core.pc.drift.DriftDetector.generateDriftChangeSet(DriftDetector.java:126)

                        at org.rhq.core.pc.drift.DriftDetector.run(DriftDetector.java:100)

                        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441)

                        at java.util.concurrent.FutureTask$Sync.innerRunAndReset(FutureTask.java:317)

                        at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:150)

                        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(ScheduledThreadPoolExecutor.java:98)

                        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.runPeriodic(ScheduledThreadPoolExecutor.java:181)

                        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:205)

                        at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)

                        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)

                        at java.lang.Thread.run(Thread.java:619)

              Caused by: java.lang.NumberFormatException: null

                        at java.lang.Integer.parseInt(Integer.java:417)

                        at java.lang.Integer.parseInt(Integer.java:499)

                        at org.rhq.common.drift.ChangeSetReaderImpl.readHeaders(ChangeSetReaderImpl.java:96)

                        at org.rhq.common.drift.ChangeSetReaderImpl.<init>(ChangeSetReaderImpl.java:64)

                        ... 12 more

              2011-10-17 10:17:15,899 DEBUG [WorkerThread#0[10.196.129.208:3677]] (jboss.remoting.transport.socket.ServerThread)- WorkerThread#0[10.196.129.208:3677] closing socketWrapper: ServerSocketWrapper[Socket[addr=/10.196.129.208,port=3677,localport=16163].1bbf45e]

              2011-10-17 10:17:15,899 DEBUG [WorkerThread#0[10.196.129.208:3677]] (jboss.remoting.transport.socket.ServerSocketWrapper)- wrote CLOSING

              2011-10-17 10:17:15,899 DEBUG [WorkerThread#0[10.196.129.208:3677]] (jboss.remoting.transport.socket.SocketWrapper)- ServerSocketWrapper[Socket[addr=/10.196.129.208,port=3677,localport=16163].1bbf45e] closing

              2011-10-17 10:17:18,805 INFO  [MeasurementManager.sender-1] (rhq.core.pc.measurement.MeasurementSenderRunner)- Measurement collection for [6] metrics took 0ms - sending report to Server...

              2011-10-17 10:17:48,883 DEBUG [pool-3-thread-1] (rhq.core.pc.drift.DriftDetector)- Processing DriftDetectionSchedule[resourceId: 10743, driftConfigurationId: 0, driftConfigurationName: Core Files]

              2011-10-17 10:17:48,883 DEBUG [pool-3-thread-1] (rhq.core.pc.drift.DriftDetector)- Skipping DriftDetectionSchedule[resourceId: 10743, driftConfigurationId: 0, driftConfigurationName: Core Files] because it is too early to do the next detection.

               

              I've also uploaded some images show the steps I've taken to set-up the change set in the console.

               

              Any ideas what I've done wrong?

               

              step1.png

              step2.png

               

              step4.png

              • 4. Re: Drift Management in RHQ 4.1
                John Sanda Apprentice

                Hi Graham,

                 

                Whatever error you originally hit is not being properly handled by the agent. The agent (with respect to your drift definition) is an inconsistent state. That is what you are running into with that ChangeSetReaderException. In short that should never happen. This is one of the issues we identified post 4.1 and it will be resolved in the 4.2 release. You will have to go to the drift UI, delete the definition/configuration, and then recreate it in order to get out of that inconsistent state. This does not however address the original issue that led to the inconsistent state. After you've recreated the drift definition, hopefully we can get a better idea of what is happening now that you have debug logging enabled.

                 

                - John

                • 5. Re: Drift Management in RHQ 4.1
                  Graham Hounslow Newbie

                  I got the above error message(s) when I first set the change set up. This was with nothing already set-up. 

                   

                   

                  To try and resolve this;

                       I've deleted the change set, and deleted the changeset.txt and dirft-changeset.txt. Then recreated the change set in the console as above.

                   

                  I've not had a chance to have a good look but it looks like its only created an empty changeset.txt file. I've only had a quick look and I couldn't see any errors the log file. I'll have a chance to have a proper look tomorrow and post any updates.

                   

                  Graham

                  • 6. Re: Drift Management in RHQ 4.1
                    Graham Hounslow Newbie

                    Gone through and deleted all drift configurations, stopped and started the agent, created a new configuration. I removed the space from the configuration name just in case that was causing a problem on Windows.

                     

                    Log file

                     

                    2011-10-19 11:20:02,303 INFO  [WorkerThread#0[10.196.129.208:2180]] (rhq.core.pc.drift.DriftManager)- Received request to update schedule for [resourceId: 10743, driftConfigurationId: 0, driftConfigurationName: CoreFiles]

                    2011-10-19 11:20:02,303 INFO  [WorkerThread#0[10.196.129.208:2180]] (rhq.core.pc.drift.DriftManager)- Adding DriftDetectionSchedule[resourceId: 10743, driftConfigurationId: 0, driftConfigurationName: CoreFiles] to ScheduleQueue[DriftDetectionSchedule[resourceId: 10743, driftConfigurationId: 0, driftConfigurationName: Core Files], DriftDetectionSchedule[resourceId: 10743, driftConfigurationId: 0, driftConfigurationName: Core Files]]

                    2011-10-19 11:20:02,303 DEBUG [WorkerThread#0[10.196.129.208:2180]] (rhq.core.pc.drift.DriftManager)- DriftDetectionSchedule[resourceId: 10743, driftConfigurationId: 0, driftConfigurationName: CoreFiles] has been added to ScheduleQueue[DriftDetectionSchedule[resourceId: 10743, driftConfigurationId: 0, driftConfigurationName: CoreFiles], DriftDetectionSchedule[resourceId: 10743, driftConfigurationId: 0, driftConfigurationName: Core Files], DriftDetectionSchedule[resourceId: 10743, driftConfigurationId: 0, driftConfigurationName: Core Files]]

                    2011-10-19 11:20:14,507 DEBUG [pool-3-thread-1] (rhq.core.pc.drift.DriftDetector)- Processing DriftDetectionSchedule[resourceId: 10743, driftConfigurationId: 0, driftConfigurationName: CoreFiles]

                    2011-10-19 11:20:14,507 DEBUG [pool-3-thread-1] (rhq.core.pc.drift.DriftDetector)- Generating coverage change set for DriftDetectionSchedule[resourceId: 10743, driftConfigurationId: 0, driftConfigurationName: CoreFiles]

                    2011-10-19 11:20:44,444 INFO  [MeasurementManager.sender-1] (rhq.core.pc.measurement.MeasurementSenderRunner)- Measurement collection for [6] metrics took 0ms - sending report to Server...

                    2011-10-19 11:21:02,835 DEBUG [WorkerThread#0[10.196.129.208:2180]] (jboss.remoting.transport.socket.ServerThread)- WorkerThread#0[10.196.129.208:2180] closing socketWrapper: ServerSocketWrapper[Socket[addr=/10.196.129.208,port=2180,localport=16163].1a0d6c9]

                    2011-10-19 11:21:02,835 DEBUG [WorkerThread#0[10.196.129.208:2180]] (jboss.remoting.transport.socket.ServerSocketWrapper)- wrote CLOSING

                    2011-10-19 11:21:02,835 DEBUG [WorkerThread#0[10.196.129.208:2180]] (jboss.remoting.transport.socket.SocketWrapper)- ServerSocketWrapper[Socket[addr=/10.196.129.208,port=2180,localport=16163].1a0d6c9] closing

                     

                     

                    It creates an empty changeset.txt file, no drift-changeset.txt file is created.

                     

                    Stopped and started the agent and an empty drift-changeset.txt file is created - though no errors in the log file.

                    • 7. Re: Drift Management in RHQ 4.1
                      John Sanda Apprentice

                      Hi Graham,

                       

                      It is difficult for me to pinpoint the issue since there has been some many changes (including bug fixes) since the 4.1 release. Based on the information that you have already provided, I am inclined to think that the problem might be with parsing a path name. Can you post a directory listing where the drift detection is running?

                      • 8. Re: Drift Management in RHQ 4.1
                        Graham Hounslow Newbie

                        The directories are

                         

                        Jboss install - e:\apps\healthcare\jboss423\

                        This is where jboss_home is set to and also the connection settings in RHQ point here as well.

                         

                        The agent is installed in c:\rhq\rhq-agent

                         

                        It's by default trying to scan bin, client and lib these are listed below - I can't see anything to obviously wrong.

                         

                        I've also tried setting up a change set using the file system option with the base directory being as above, but this doesn't work either - or works the same as the plugin configuration option depending on how you look at it.

                         

                        Directory of E:\apps\healthcare\jboss423\bin

                         

                        Bin Directory

                        19/10/2011  16:32    <DIR>          .

                        19/10/2011  16:32    <DIR>          ..

                        18/07/2008  14:40             3,535 classpath.sh

                        12/01/2011  10:39             1,273 InstallApp-NT-node-01.bat

                        13/08/2009  10:05             1,265 InstallApp-NT.bat

                        19/10/2011  09:56                 6 jboss-wrapper.pid

                        19/10/2011  09:56                 6 jboss.pid

                        18/07/2008  14:41             8,808 jboss_init_hpux.sh

                        18/07/2008  14:41             3,639 jboss_init_redhat.sh

                        18/07/2008  14:41             4,671 jboss_init_suse.sh

                        18/07/2008  14:40               535 probe.bat

                        18/07/2008  14:40               918 probe.sh

                        18/07/2008  14:40             3,527 run.bat

                        18/07/2008  14:40             1,519 run.conf

                        18/07/2008  14:44            43,560 run.jar

                        18/07/2008  14:40             7,107 run.sh

                        18/07/2008  14:41             1,665 shutdown.bat

                        18/07/2008  14:44            21,710 shutdown.jar

                        18/07/2008  14:41             1,903 shutdown.sh

                        09/06/2011  15:43         1,447,793 stacktrace.log

                        07/10/2011  15:58               110 startdef.bat

                        18/07/2008  14:42             1,919 twiddle.bat

                        18/07/2008  14:44            47,369 twiddle.jar

                        18/07/2008  14:42             2,348 twiddle.sh

                        12/01/2011  10:56             1,277 UninstallApp-NT-node-01.bat

                        13/08/2009  10:05             1,269 UninstallApp-NT.bat

                        13/08/2009  10:05           167,936 wrapper.exe

                        13/08/2009  10:05             2,065 wrapper.log

                        13/08/2009  10:05             2,773 wsconsume.bat

                        13/08/2009  10:05             3,803 wsconsume.sh

                        13/08/2009  10:05             2,859 wsprovide.bat

                        13/08/2009  10:05             3,885 wsprovide.sh

                        13/08/2009  10:05             3,009 wsrunclient.bat

                        13/08/2009  10:05             3,972 wsrunclient.sh

                        13/08/2009  10:05             2,378 wstools.bat

                        13/08/2009  10:05             3,380 wstools.sh

                         

                         

                        Lib directory

                        Directory of E:\apps\healthcare\jboss423\lib

                         

                         

                        13/08/2009  10:05    <DIR>          .

                        13/08/2009  10:05    <DIR>          ..

                        18/07/2008  14:44            47,469 commons-codec.jar

                        18/07/2008  14:44           214,798 commons-httpclient.jar

                        18/07/2008  14:44            55,423 commons-logging.jar

                        18/07/2008  14:44           255,512 concurrent.jar

                        13/08/2009  10:05    <DIR>          endorsed

                        18/07/2008  14:44            15,857 getopt.jar

                        18/07/2008  14:44           406,562 jboss-common.jar

                        18/07/2008  14:44           880,190 jboss-jmx.jar

                        18/07/2008  14:44           267,175 jboss-system.jar

                        18/07/2008  14:44           376,342 jboss-xml-binding.jar

                        18/07/2008  14:44           128,400 log4j-boot.jar

                                      10 File(s)      2,647,728 bytes

                         

                         

                        Directory of E:\apps\healthcare\jboss423\lib\endorsed

                         

                         

                        13/08/2009  10:05    <DIR>          .

                        13/08/2009  10:05    <DIR>          ..

                        13/08/2009  10:05            89,967 jaxb-api.jar

                        18/07/2008  14:44            29,383 jboss-jaxrpc.jar

                        18/07/2008  14:44            33,293 jboss-jaxws-ext.jar

                        18/07/2008  14:44            34,715 jboss-jaxws.jar

                        18/07/2008  14:44            26,981 jboss-saaj.jar

                        13/08/2009  10:05            29,165 jbossws-native-jaxrpc.jar

                        13/08/2009  10:05            20,141 jbossws-native-jaxws-ext.jar

                        13/08/2009  10:05            47,411 jbossws-native-jaxws.jar

                        13/08/2009  10:05            27,251 jbossws-native-saaj.jar

                        18/07/2008  14:44           188,992 serializer.jar

                        18/07/2008  14:44         1,713,709 xalan.jar

                        18/07/2008  14:44         1,200,040 xercesImpl.jar

                         

                         

                        Client Directory

                        Directory of E:\apps\healthcare\jboss423\client

                         

                         

                        13/08/2009  10:05    <DIR>          .

                        13/08/2009  10:05    <DIR>          ..

                        18/07/2008  14:44            58,223 activation.jar

                        18/07/2008  14:44           355,300 antlr.jar

                        18/07/2008  14:41               596 auth.conf

                        18/07/2008  14:44            73,334 avalon-framework.jar

                        18/07/2008  14:44            47,469 commons-codec.jar

                        18/07/2008  14:44           214,798 commons-httpclient.jar

                        18/07/2008  14:44            55,423 commons-logging.jar

                        18/07/2008  14:44           255,512 concurrent.jar

                        18/07/2008  14:44            52,219 ejb3-persistence.jar

                        13/08/2009  10:05           291,779 FastInfoset.jar

                        18/07/2008  14:44            15,857 getopt.jar

                        18/07/2008  14:44           444,510 hibernate-annotations.jar

                        18/07/2008  14:44           246,649 hibernate-client.jar

                        18/07/2008  14:44         4,717,899 jacorb.jar

                        18/07/2008  14:44           599,742 javassist.jar

                        13/08/2009  10:05            89,967 jaxb-api.jar

                        13/08/2009  10:05           849,481 jaxb-impl.jar

                        13/08/2009  10:05         3,126,060 jaxb-xjc.jar

                        13/08/2009  10:05         1,187,868 jaxws-rt.jar

                        13/08/2009  10:05           509,137 jaxws-tools.jar

                        18/07/2008  14:44            41,485 jboss-annotations-ejb3.jar

                        18/07/2008  14:44           106,590 jboss-aop-jdk50-client.jar

                        18/07/2008  14:44            46,156 jboss-aspect-jdk50-client.jar

                        18/07/2008  14:44           193,844 jboss-client.jar

                        18/07/2008  14:44           406,562 jboss-common-client.jar

                        18/07/2008  14:44            59,446 jboss-deployment.jar

                        18/07/2008  14:44           300,322 jboss-ejb3-client.jar

                        18/07/2008  14:44            20,389 jboss-ejb3x.jar

                        18/07/2008  14:44           152,441 jboss-iiop-client.jar

                        18/07/2008  14:44           423,725 jboss-j2ee.jar

                        18/07/2008  14:44            28,411 jboss-jsr77-client.jar

                        18/07/2008  14:44           907,987 jboss-remoting.jar

                        18/07/2008  14:44           125,850 jboss-serialization.jar

                        18/07/2008  14:44            22,227 jboss-srp-client.jar

                        18/07/2008  14:44            26,313 jboss-system-client.jar

                        18/07/2008  14:44            53,275 jboss-transaction-client.jar

                        18/07/2008  14:44           376,342 jboss-xml-binding.jar

                        18/07/2008  14:44         5,000,359 jbossall-client.jar

                        18/07/2008  14:44            22,226 jbosscx-client.jar

                        18/07/2008  14:44            58,339 jbossha-client.jar

                        18/07/2008  14:44           166,785 jbossjmx-ant.jar

                        18/07/2008  14:44           333,776 jbossmq-client.jar

                        18/07/2008  14:44           151,240 jbosssx-client.jar

                        13/08/2009  10:05           117,113 jbossws-common.jar

                        13/08/2009  10:05            95,527 jbossws-framework.jar

                        13/08/2009  10:05            79,729 jbossws-jboss423.jar

                        13/08/2009  10:05             4,826 jbossws-native-client.jar

                        13/08/2009  10:05         1,662,592 jbossws-native-core.jar

                        13/08/2009  10:05            29,165 jbossws-native-jaxrpc.jar

                        13/08/2009  10:05            20,141 jbossws-native-jaxws-ext.jar

                        13/08/2009  10:05            47,411 jbossws-native-jaxws.jar

                        13/08/2009  10:05            27,251 jbossws-native-saaj.jar

                        13/08/2009  10:05           107,896 jbossws-spi.jar

                        13/08/2009  10:05            54,187 jettison.jar

                        18/07/2008  14:44           454,151 jmx-client.jar

                        18/07/2008  14:44             4,809 jmx-invoker-adaptor-client.jar

                        18/07/2008  14:44            33,147 jnp-client.jar

                        18/07/2008  14:44           355,751 log4j.jar

                        18/07/2008  14:44            86,380 logkit.jar

                        18/07/2008  14:44           387,689 mail.jar

                        13/08/2009  10:05            39,688 policy.jar

                        18/07/2008  14:44            82,773 scout.jar

                        18/07/2008  14:44            89,677 servlet-api.jar

                        13/08/2009  10:05            23,346 stax-api.jar

                        13/08/2009  10:05            11,074 stax-ex.jar

                        13/08/2009  10:05            57,984 streambuffer.jar

                        18/07/2008  14:44           385,226 trove.jar

                        13/08/2009  10:05           148,522 wsdl4j.jar

                        13/08/2009  10:05           525,813 wstx.jar

                        18/07/2008  14:44           289,208 xmlsec.jar

                         

                         

                        It might be something we're have to leave until 4.2 comes out maybe.


                        • 9. Re: Drift Management in RHQ 4.1
                          John Sanda Apprentice

                          Hi Graham,

                           

                          My apologies for not replying sooner. We have been heads down trying to get things wrapped up for a 4.2 release. Unfortunately I was not able to determine the source of your problem, though I wasn't able to spend too much time looking at it. An RHQ 4.2 release is due out very soon, and for drift management, that means lots of bug fixes and a whole bunch of new features. I hope you are willing and able to give it another try with 4.2. The community feedback is invaluable. I will be sure to post a note on the forum in the coming days about changes and enhancements with respect to drift management.

                          • 10. Re: Drift Management in RHQ 4.1
                            Graham Hounslow Newbie

                            Hi John,

                             

                            no need to apologise, I'm really grateful for the help you've been able to give.

                             

                            Really looking forward to the 4.2 release, as soon as we get it on our servers I'll let you know how it goes.

                             

                            We're really pleased with RHQ and it's great to monitor our environments. Still trying to get my head round the bundle and provisioning though that's next on the list.

                             

                            cheers

                             

                            Graham