Problem by monitoring Tomcat with JON

[thomas2008ch]

I use JON 2.3.

Till now I use Jon to monitoring the Jboss server and it work fine.

In the last days I began to install the Jon-agent on LINUX machine where tow Tomcat 5.5.27 running.

The plugin for Tomcat is installed on Jon-Server.

I've added the following codes at the beginning of file bin/startup.sh (Tomcat):

# JON/Jopr Management
JAVA_OPTS="${JAVA_OPTS} -Dcom.sun.management.jmxremote.port=8999"
JAVA_OPTS="${JAVA_OPTS} -Dcom.sun.management.jmxremote.ssl=false"

JAVA_OPTS="${JAVA_OPTS} -Dcom.sun.management.jmxremote.authenticate=false"
export JAVA_OPTS

And I add the following code in field 'Manager URL' under Inventory/Connection in Jon-GUI:

service:jmx:rmi:///jndi/rmi://172.23.17.36:8999/jmxrmi

As I open the JON-GUI, I can see the nodes of these tow Tomcat but they are marked as red. And there is no any other sub nodes. In agent.log one can see followings.

What could be the problem?

2010-07-08 14:21:00,675 WARN  [ResourceContainer.invoker.daemon-1] (jboss.on.plugins.tomcat.TomcatServerComponent)- Could not establish connection to the Tomcat instance [1] times for resource [/data/ivs/bestandsservice-tomcat]
org.mc4j.ems.connection.EmsConnectException: Could not connect [service:jmx:rmi:///jndi/rmi://172.23.17.36:8999/jmxrmi] org.mc4j.ems.connection.EmsConnectException: IOException: Check service availability
    at org.mc4j.ems.impl.jmx.connection.support.providers.AbstractConnectionProvider.connect(AbstractConnectionProvider.java:103)
    at org.jboss.on.plugins.tomcat.TomcatServerComponent.loadConnection(TomcatServerComponent.java:218)
    at org.jboss.on.plugins.tomcat.TomcatServerComponent.start(TomcatServerComponent.java:298)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
    at java.lang.reflect.Method.invoke(Method.java:597)
    at org.rhq.core.pc.inventory.ResourceContainer$ComponentInvocationThread.call(ResourceContainer.java:525)
    at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
    at java.util.concurrent.FutureTask.run(FutureTask.java:138)
    at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
    at java.lang.Thread.run(Thread.java:619)
Caused by: org.mc4j.ems.connection.EmsConnectException: IOException: Check service availability
    at org.mc4j.ems.impl.jmx.connection.support.providers.JMXRemotingConnectionProvider.doConnect(JMXRemotingConnectionProvider.java:163)
    at org.mc4j.ems.impl.jmx.connection.support.providers.AbstractConnectionProvider.connect(AbstractConnectionProvider.java:99)
    ... 12 more
Caused by: java.io.IOException: Failed to retrieve RMIServer stub: javax.naming.ServiceUnavailableException [Root exception is java.rmi.ConnectException: Connection refused to host: 172.23.17.36; nested exception is:
    java.net.ConnectException: Connection refused]
    at javax.management.remote.rmi.RMIConnector.connect(RMIConnector.java:338)
    at javax.management.remote.JMXConnectorFactory.connect(JMXConnectorFactory.java:248)
    at org.mc4j.ems.impl.jmx.connection.support.providers.JMXRemotingConnectionProvider.doConnect(JMXRemotingConnectionProvider.java:144)
    ... 13 more
Caused by: javax.naming.ServiceUnavailableException [Root exception is java.rmi.ConnectException: Connection refused to host: 172.23.17.36; nested exception is:
    java.net.ConnectException: Connection refused]
    at com.sun.jndi.rmi.registry.RegistryContext.lookup(RegistryContext.java:101)
    at com.sun.jndi.toolkit.url.GenericURLContext.lookup(GenericURLContext.java:185)
    at javax.naming.InitialContext.lookup(InitialContext.java:392)
    at javax.management.remote.rmi.RMIConnector.findRMIServerJNDI(RMIConnector.java:1886)
    at javax.management.remote.rmi.RMIConnector.findRMIServer(RMIConnector.java:1856)
    at javax.management.remote.rmi.RMIConnector.connect(RMIConnector.java:257)
    ... 15 more
Caused by: java.rmi.ConnectException: Connection refused to host: 172.23.17.36; nested exception is:
    java.net.ConnectException: Connection refused
    at sun.rmi.transport.tcp.TCPEndpoint.newSocket(TCPEndpoint.java:601)
    at sun.rmi.transport.tcp.TCPChannel.createConnection(TCPChannel.java:198)
    at sun.rmi.transport.tcp.TCPChannel.newConnection(TCPChannel.java:184)
    at sun.rmi.server.UnicastRef.newCall(UnicastRef.java:322)
    at sun.rmi.registry.RegistryImpl_Stub.lookup(Unknown Source)
    at com.sun.jndi.rmi.registry.RegistryContext.lookup(RegistryContext.java:97)
    ... 20 more
Caused by: java.net.ConnectException: Connection refused
    at java.net.PlainSocketImpl.socketConnect(Native Method)
    at java.net.PlainSocketImpl.doConnect(PlainSocketImpl.java:333)
    at java.net.PlainSocketImpl.connectToAddress(PlainSocketImpl.java:195)
    at java.net.PlainSocketImpl.connect(PlainSocketImpl.java:182)
    at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:366)
    at java.net.Socket.connect(Socket.java:525)
    at java.net.Socket.connect(Socket.java:475)
    at java.net.Socket.<init>(Socket.java:372)
    at java.net.Socket.<init>(Socket.java:186)
    at sun.rmi.transport.proxy.RMIDirectSocketFactory.createSocket(RMIDirectSocketFactory.java:22)
    at sun.rmi.transport.proxy.RMIMasterSocketFactory.createSocket(RMIMasterSocketFactory.java:128)
    at sun.rmi.transport.tcp.TCPEndpoint.newSocket(TCPEndpoint.java:595)
    ... 25 more

[jayshaughnessy]

The settings look correct in the startup.sh.  The connection url looks good as well but perhaps you have an issue with the IP or port?  Assuming the TC instance and the RHQ agent are on the same host you could try using

service:jmx:rmi:///jndi/rmi://localhost:8999/jmxrmi as opposed to

service:jmx:rmi:///jndi/rmi://172.23.17.36:8999/jmxrmi.  Perhaps avoiding a firewall issue.  Try using jconsole to see whether it  an establish a remote connection with the url you are entering into JON. If jconsole works JON should as well

[thomas2008ch]

The Tomecat (TC) instance and the agent are on the same machine. But as I change to use the localhost, it is the same as before.

There is no firewall, since I installed a Jboss instance on the same machine and it works fine.

I use the jconsole and use the 'service:jmx:rmi:///jndi/rmi://localhost:8999/jmxrmi' in the text field 'Remote Prozess' but I failed. I get error: The connection to 'service:jmx:rmi:///jndi/rmi://localhost:8999/jmxrmi' did not succeed.

I try the URL of 'service:jmx:rmi:///jndi/rmi://172.23.17.36:8999/jmxrmi' in jconsole, but it's the same error.

[jayshaughnessy]

"In the last days I began to install the Jon-agent on LINUX machine where  two Tomcat 5.5.27 running."

Are you starting both Tomcat instances with the same jmx port of 8999?  Perhaps you have some sort of a conflict.  Until you can get JConsole to establish a remote connection using the host:port you must have an issue with either the ability to resolve/contact the host, or the port is blocked in some way.