4 Replies Latest reply on Feb 21, 2014 5:53 AM by orochimaru19860000

    Supervision Jboss AS 7 with Zabbix - Leak memory

    ryuma
    ryuma Jan 29, 2013 4:34 AM

      Hi,

       

      I use Jboss as 7.1.1 Final and i would want monitored it. For that, i used Zabbix which allow a JMX supervision but i met problems with Jboss.

      Jboss used a Remoting-jmx protocol while Zabbix JMX Agent used rmi protocol to connect about JMX Server.

       

      So, i changed the JMX agent of Zabbix to allow the connection with a remoting-jmx protocol between Zabbix and Jboss and my JMX supervision succeed.

       

      However, now there is a leak memory with Jboss and after 2-day, Jboss crash.

       

      I can see in Jboss log errors appear durring this 2 days:

       

      22:33:53,880 ERROR [org.jboss.remoting.remote.connection] (Remoting "mcptest1" read-1) JBREM000200: Remote connection failed: java.io.IOException: Connection reset by peer

      22:37:54,229 ERROR [org.jboss.remoting.remote.connection] (Remoting "mcptest1:MANAGEMENT" read-1) JBREM000200: Remote connection failed: java.io.IOException: Connection reset by peer

       

      I used memory Analyser to locate the leak memory and i found:

      -2 889 instances of "org.jboss.remoting3.remote.RemoteConnection", loaded by "org.jboss.modules.ModuleClassLoader @ 0x92ab0420" occupy 49 737 024 (28,61%) bytes. These instances are referenced from one instance of "java.util.HashMap$Entry[]", loaded by "<system class loader>"

       

      -2 889 instances of "org.jboss.remoting3.remote.RemoteConnectionChanne l", loaded by "org.jboss.modules.ModuleClassLoader @ 0x92ab0420" occupy 19 182 960 (11,04%) bytes. These instances are referenced from one instance of "java.util.HashMap$Entry[]", loaded by "<system class loader>"

      When Jboss crash, we can see a following error:

      ERROR [org.xnio.listener] (Remoting "mcptest1:MANAGEMENT" read-1) A channel event listener threw an exception: java.lang.OutOfMemoryError: Java heap space

      For the moment, i work on the Zabbix JMX Agent to fix this problem but i don't find any problems.

      So i post this thread to know if isn't a Jboss JMX problem and find solution.

      Thank you and if you have an idea, contact me .

       

       

      • 5493 Views
      • Tags:
        • 1. Re: Supervision Jboss AS 7 with Zabbix - Leak memory
          ryuma
          ryuma Jan 29, 2013 10:22 AM (in response to ryuma)

          I have a new information. With the JMX Agent, i find this error:

          [Remoting "endpoint" task-2] DEBUG org.jboss.remoting.remote.client - Client authentication failed for mechanism JBOSS-LOCAL-USER: javax.security.sasl.SaslException: Failed to read server challenge [Caused by java.io.FileNotFoundException: C:\JBoss\jboss-as-7.1.1.Final\standalone\tmp\auth\challenge-2169748 (No such file or directory)]

           

           

           

          However, the PATH "C:\JBoss\jboss-as-7.1.1.Final\" is false(I'm in CentOS). I want to change this PATH but i don"t know where i can find it ...

            • Actions
          • 2. Re: Supervision Jboss AS 7 with Zabbix - Leak memory
            dlofthouse
            dlofthouse Jan 29, 2013 10:26 AM (in response to ryuma)

            The memory leak you are encountering is a known issue, upgrading to AS 7.1.3 should cover that scenario although there is a second leak that will be fixed in AS 7.2.0.

             

            Regarding the SASL message, that message is just a DEBUG message.  I am assuming you are running JBoss AS on Windows and the agent on CentOS - that message is fine it just shows that using the local authentication mechanisms was attempted but failed and it should then have automatically moved on to use a supported authentication mechanism.

              • Actions
            • 3. Re: Supervision Jboss AS 7 with Zabbix - Leak memory
              ryuma
              ryuma Jan 29, 2013 11:12 AM (in response to dlofthouse)

              Ok, thanks for your answer. So I going to wait Jboss AS 7.1.3 (or 7.2.0 if the bug persist) and begin to make Jboss template for the supervision.

               

              However, my Jboss AS are running on Linux. For the moment, i tested my supervision only on CentOS. So, i don"t know why i have this PATH ... It's strange ! ^^

               

              So, thanks for your information .

               

              edit:

              Could you give me the JIRA issue about this memory leak please ?

                • Actions
              • 4. Re: Supervision Jboss AS 7 with Zabbix - Leak memory
                orochimaru19860000
                orochimaru19860000 Feb 21, 2014 5:53 AM (in response to ryuma)

                Hello Nicolas,

                 

                I'm trying to get Zabbix monitor Jboss 7 and I'm encountering the same problem:

                 

                "Jboss used a Remoting-jmx protocol while Zabbix JMX Agent used rmi protocol to connect about JMX Server"

                 

                Can you tell me how did you manager to change the JMX agent of Zabbix to allow connection with remoting-jmx protocol between Zabbix and Jboss please. Or at least point me to the documentation you used.

                 

                I would be very grateful.

                 

                Thanks and regards.

                 

                Medo

                  • Actions