Supervision Jboss AS 7 with Zabbix - Leak memory

[ryuma]

Hi,

I use Jboss as 7.1.1 Final and i would want monitored it. For that, i used Zabbix which allow a JMX supervision but i met problems with Jboss.

Jboss used a Remoting-jmx protocol while Zabbix JMX Agent used rmi protocol to connect about JMX Server.

So, i changed the JMX agent of Zabbix to allow the connection with a remoting-jmx protocol between Zabbix and Jboss and my JMX supervision succeed.

However, now there is a leak memory with Jboss and after 2-day, Jboss crash.

I can see in Jboss log errors appear durring this 2 days:

22:33:53,880 ERROR [org.jboss.remoting.remote.connection] (Remoting "mcptest1" read-1) JBREM000200: Remote connection failed: java.io.IOException: Connection reset by peer

22:37:54,229 ERROR [org.jboss.remoting.remote.connection] (Remoting "mcptest1:MANAGEMENT" read-1) JBREM000200: Remote connection failed: java.io.IOException: Connection reset by peer

I used memory Analyser to locate the leak memory and i found:

-2 889 instances of "org.jboss.remoting3.remote.RemoteConnection", loaded by "org.jboss.modules.ModuleClassLoader @ 0x92ab0420" occupy 49 737 024 (28,61%) bytes. These instances are referenced from one instance of "java.util.HashMap$Entry[]", loaded by "<system class loader>"

-2 889 instances of "org.jboss.remoting3.remote.RemoteConnectionChanne l", loaded by "org.jboss.modules.ModuleClassLoader @ 0x92ab0420" occupy 19 182 960 (11,04%) bytes. These instances are referenced from one instance of "java.util.HashMap$Entry[]", loaded by "<system class loader>"

When Jboss crash, we can see a following error:

ERROR [org.xnio.listener] (Remoting "mcptest1:MANAGEMENT" read-1) A channel event listener threw an exception: java.lang.OutOfMemoryError: Java heap space

For the moment, i work on the Zabbix JMX Agent to fix this problem but i don't find any problems.

So i post this thread to know if isn't a Jboss JMX problem and find solution.

Thank you and if you have an idea, contact me .

[ryuma]

I have a new information. With the JMX Agent, i find this error:

[Remoting "endpoint" task-2] DEBUG org.jboss.remoting.remote.client - Client authentication failed for mechanism JBOSS-LOCAL-USER: javax.security.sasl.SaslException: Failed to read server challenge [Caused by java.io.FileNotFoundException: C:\JBoss\jboss-as-7.1.1.Final\standalone\tmp\auth\challenge-2169748 (No such file or directory)]

However, the PATH "C:\JBoss\jboss-as-7.1.1.Final\" is false(I'm in CentOS). I want to change this PATH but i don"t know where i can find it ...

[dlofthouse]

The memory leak you are encountering is a known issue, upgrading to AS 7.1.3 should cover that scenario although there is a second leak that will be fixed in AS 7.2.0.

Regarding the SASL message, that message is just a DEBUG message.  I am assuming you are running JBoss AS on Windows and the agent on CentOS - that message is fine it just shows that using the local authentication mechanisms was attempted but failed and it should then have automatically moved on to use a supported authentication mechanism.

[ryuma]

Ok, thanks for your answer. So I going to wait Jboss AS 7.1.3 (or 7.2.0 if the bug persist) and begin to make Jboss template for the supervision.

However, my Jboss AS are running on Linux. For the moment, i tested my supervision only on CentOS. So, i don"t know why i have this PATH ... It's strange ! ^^

So, thanks for your information .

edit:

Could you give me the JIRA issue about this memory leak please ?

[orochimaru19860000]

Hello Nicolas,

I'm trying to get Zabbix monitor Jboss 7 and I'm encountering the same problem:

"Jboss used a Remoting-jmx protocol while Zabbix JMX Agent used rmi protocol to connect about JMX Server"

Can you tell me how did you manager to change the JMX agent of Zabbix to allow connection with remoting-jmx protocol between Zabbix and Jboss please. Or at least point me to the documentation you used.

I would be very grateful.

Thanks and regards.

Medo