JBossAS plugin says JBoss state is down but not

[makkoca]

We've a pair of solaris+jboss-4.2.3 server running with same environment and server configs. RHQ3.0 JBossAS plugin works ok on machine1 while it gets below error on machine2. Actualy plugin discovers jboss but found it's state DOWN because of the error.

Do you have any idea what can cause plugin to act different?

2010-09-01 13:46:17,976 DEBUG [InventoryManager.discovery-1]
(org.rhq.plugins.jbossas.JBossASServerComponent)- Could not connect to
the JBoss AS instance for resource [/app/jboss-4.2.3.GA/server/INTERNET]
org.mc4j.ems.connection.EmsConnectException:
java.io.StreamCorruptedException:
javax.management.MalformedObjectNameException: jboss.monitor:service= is
not a valid ObjectName. Invalid key/value data service
        at
org.mc4j.ems.impl.jmx.connection.support.providers.proxy.GenericMBeanServerProxy.invoke(GenericMBeanServerProxy.java:156)
        at $Proxy53.queryNames(Unknown Source)
        at org.mc4j.ems.impl.jmx.connection.DConnection.loadSynchronous(DConnection.java:137)
        at org.rhq.plugins.jbossas.JBossASServerComponent.loadConnection(JBossASServerComponent.java:1112)

     Both server's JBoss startup arguments are here.

       Erroneous one,

[jboss@intjb01 ~]\ pargs 1243
1243:   /usr/jdk/instances/jdk1.6.0/bin/amd64/java -Dprogram.name=run.sh -server -Xms51
argv[0]: /usr/jdk/instances/jdk1.6.0/bin/amd64/java
argv[1]: -Dprogram.name=run.sh
argv[2]: -server
argv[3]: -Xms512m
argv[4]: -Xmx1024m
argv[5]: -XX:MaxPermSize=512m
argv[6]: -Dfile.encoding=iso-8859-9
argv[7]: -Dsun.rmi.dgc.client.gcInterval=3600000
argv[8]: -Dsun.rmi.dgc.server.gcInterval=3600000
argv[9]: -Dcom.sun.management.jmxremote.port=19988
argv[10]: -Dcom.sun.management.jmxremote.ssl=false
argv[11]: -Dcom.sun.management.jmxremote.authenticate=false
argv[12]: -Djava.endorsed.dirs=/app/jboss-4.2.3.GA/lib/endorsed
argv[13]: -classpath
argv[14]: /app/jboss-4.2.3.GA/bin/run.jar
argv[15]: org.jboss.Main
argv[16]: -c
argv[17]: INTERNET
argv[18]: -b
argv[19]: 0.0.0.0

       Working one,

[jboss@intjb02 ~]\ pargs 21242
21242:  /usr/jdk/instances/jdk1.6.0/bin/amd64/java -Dprogram.name=run.sh -server -Xms51
argv[0]: /usr/jdk/instances/jdk1.6.0/bin/amd64/java
argv[1]: -Dprogram.name=run.sh
argv[2]: -server
argv[3]: -Xms512m
argv[4]: -Xmx1024m
argv[5]: -XX:MaxPermSize=512m
argv[6]: -Dfile.encoding=iso-8859-9
argv[7]: -Dsun.rmi.dgc.client.gcInterval=3600000
argv[8]: -Dsun.rmi.dgc.server.gcInterval=3600000
argv[9]: -Dcom.sun.management.jmxremote.port=1095
argv[10]: -Dcom.sun.management.jmxremote.authenticate=false
argv[11]: -Dcom.sun.management.jmxremote.ssl=false
argv[12]: -Djava.endorsed.dirs=/app/jboss-4.2.3.GA/lib/endorsed
argv[13]: -classpath
argv[14]: /app/jboss-4.2.3.GA/bin/run.jar
argv[15]: org.jboss.Main
argv[16]: -c
argv[17]: INTERNET
argv[18]: -b
argv[19]: 0.0.0.0

I added agents first time discovery logs. I've analyzed that logs with WinMerge but nothing seemed wrong to me.

[mazz]

take a look at your JBossAS's MBeanServer (either through the jmx-console if you've deployed that war or perhaps through JConsole) and see what MBeans are in the domain "jboss.monitor", specifically those that have the "service" key (e.g. jboss.monitor:service=...something here...). What are the full MBean ObjectNames that you see there?

Perhaps there is a problem due to non-ASCII characters in the value?

[makkoca]

Thanks Maz,

Your hints were very helpful to see what the problem is.

Problematic JBoss has "jboss.monitor" domain while working jboss hasn't (see jboss.monitoring.png) when looking trough jmx-console.

Because problematic JBoss has a previously created MBean monitor.  (See monitor.png for the detail)

The strange thing is "Monitoring" main title does not exist when looking trough web-console(see missing_monitoring.png).

It supposed to be the place where management of monitors (listing,removing,..) in jboss web-console.

So, I somehow have caused disappering of "Monitoring" as far as i understand.

I think there is a previously created monitor that has an invalid name as you told before.

Now I'm going to find a way to remove that monitor(s).

[makkoca]

There was only one monitor definition in deploy/management/monitors/-service.xml and the name attribute was empty.

I removed monitors folder and everything turned normal as it should be.

JBoss Web-Console allows creating monitors with empty name but complains when querying them.

Thanks for help again.