Periodic down alert Agent 1.2 beta 1 server 2.2

[swapyeole]

In Jopr server Jboss instance goes down after approximate 59 to 60 min state down for 1 min and agene come back. This behavior is seen in only one server. All other are working fine.
I tried to remove this jboss instance form monitoring but it came out with some error now I am not able remove it from inventory. i took every possible action which I can but no help.
Env :
windows server
Jboss configured as windows service with jboss-native
Java : jdk1.5.0_14
After enabling debugging mode in Agent I got these error:

2009-03-23 16:25:52,641 INFO [InventoryManager.discovery-1] (org.rhq.plugins.agent.AgentJavaServiceWrapperDiscoveryComponent)- Discovering RHQ Agent's JSW service...
2009-03-23 16:25:56,922 INFO [InventoryManager.discovery-1] (rhq.core.pc.inventory.RuntimeDiscoveryExecutor)- Scanned 0 servers and found 0 total descendant Resources.
2009-03-23 16:25:56,922 INFO [InventoryManager.discovery-1] (rhq.core.pc.inventory.InventoryManager)- Sending runtime inventory report to Server...
2009-03-23 16:25:57,063 INFO [InventoryManager.discovery-1] (rhq.core.pc.inventory.InventoryManager)- Syncing local inventory with Server inventory...
2009-03-23 16:26:31,590 INFO [MeasurementManager.sender-1] (rhq.core.pc.measurement.MeasurementCollectorRunner)- Measurement collection for [8] metrics took 62ms - sending report to Server...
2009-03-23 16:26:38,792 INFO [InventoryManager.availability-1] (rhq.core.pc.inventory.InventoryManager)- Sending availability report to Server...
2009-03-23 16:27:31,552 INFO [MeasurementManager.sender-1] (rhq.core.pc.measurement.MeasurementCollectorRunner)- Measurement collection for [8] metrics took 0ms - sending report to Server...
2009-03-23 16:27:39,051 INFO [InventoryManager.availability-1] (rhq.core.pc.inventory.InventoryManager)- Sending availability report to Server...
2009-03-23 16:28:39,294 INFO [InventoryManager.availability-1] (rhq.core.pc.inventory.InventoryManager)- Sending availability report to Server...
2009-03-23 16:29:01,541 INFO [MeasurementManager.sender-1] (rhq.core.pc.measurement.MeasurementCollectorRunner)- Measurement collection for [5] metrics took 0ms - sending report to Server...
2009-03-23 16:29:39,568 INFO [InventoryManager.availability-1] (rhq.core.pc.inventory.InventoryManager)- Sending availability report to Server...
2009-03-23 16:30:01,534 INFO [MeasurementManager.sender-1] (rhq.core.pc.measurement.MeasurementCollectorRunner)- Measurement collection for [8] metrics took 0ms - sending report to Server...
2009-03-23 16:30:31,578 INFO [ConfigurationManager.threadpool-1] (rhq.core.pc.configuration.ConfigurationCheckExecutor)- Starting configuration update check
2009-03-23 16:30:31,578 INFO [ConfigurationManager.threadpool-1] (rhq.core.pc.configuration.ConfigurationCheckExecutor)- New configuration version detected on resource: Resource[id=500052, type=RHQ Agent, key=gsd1w173t.CORP.ROOT.GLOBAL RHQ Agent, name=gsd1w173t.CORP.ROOT.GLOBAL RHQ Agent, version=1.2.0.Beta1]
2009-03-23 16:30:31,578 INFO [ConfigurationManager.threadpool-1] (rhq.core.pc.configuration.ConfigurationCheckExecutor)- New configuration version detected on resource: Resource[id=500068, type=java.util.logging, key=java.util.logging:type=Logging, name=Logging]
2009-03-23 16:30:31,578 INFO [ConfigurationManager.threadpool-1] (rhq.core.pc.configuration.ConfigurationCheckExecutor)- New configuration version detected on resource: Resource[id=500067, type=VM Class Loading System, key=java.lang:type=ClassLoading, name=Class Loading]
2009-03-23 16:30:31,578 INFO [ConfigurationManager.threadpool-1] (rhq.core.pc.configuration.ConfigurationCheckExecutor)- New configuration version detected on resource: Resource[id=500064, type=Threading, key=java.lang:type=Threading, name=Threading]
2009-03-23 16:30:31,593 INFO [ConfigurationManager.threadpool-1] (rhq.core.pc.configuration.ConfigurationCheckExecutor)- New configuration version detected on resource: Resource[id=500065, type=VM Memory System, key=java.lang:type=Memory, name=Memory Subsystem]
2009-03-23 16:30:31,640 INFO [ConfigurationManager.threadpool-1] (rhq.core.pc.configuration.ConfigurationCheckExecutor)- New configuration version detected on resource: Resource[id=500062, type=Java Service Wrapper Launcher, key=jsw, name=RHQ Agent Java Service Wrapper, version=1.2.0.Beta1]
2009-03-23 16:30:31,640 INFO [ConfigurationManager.threadpool-1] (rhq.core.pc.configuration.ConfigurationCheckExecutor)- New configuration version detected on resource: Resource[id=500061, type=Environment Setup Script, key=environment-setup-script, name=rhq-agent-env.bat, version=1.2.0.Beta1]
2009-03-23 16:30:32,078 INFO [ConfigurationManager.threadpool-1] (rhq.core.pc.configuration.ConfigurationCheckExecutor)- New configuration version detected on resource: Resource[id=502082, type=Datasource, key=jboss.jca:name=DefaultDS,service=DataSourceBinding, name=DefaultDS Datasource]
2009-03-23 16:30:32,171 INFO [ConfigurationManager.threadpool-1] (rhq.core.pc.configuration.ConfigurationCheckExecutor)- New configuration version detected on resource: Resource[id=502081, type=Datasource, key=jboss.jca:name=UserDataBase,service=DataSourceBinding, name=UserDataBase Datasource]
2009-03-23 16:30:32,265 INFO [ConfigurationManager.threadpool-1] (rhq.core.pc.configuration.ConfigurationCheckExecutor)- New configuration version detected on resource: Resource[id=502582, type=JMQ JMS Queue, key=jboss.mq.destination:name=testQueue,service=Queue, name=testQueue JMS Queue]
2009-03-23 16:30:44,733 INFO [InventoryManager.availability-1] (rhq.core.pc.inventory.InventoryManager)- Sending availability report to Server...
2009-03-23 16:31:01,543 INFO [MeasurementManager.sender-1] (rhq.core.pc.measurement.MeasurementCollectorRunner)- Measurement collection for [18] metrics took 31ms - sending report to Server...
2009-03-23 16:31:02,262 WARN [ConfigurationManager.threadpool-1] (rhq.core.pc.configuration.ConfigurationCheckExecutor)- Unable to check for updated configuration
org.rhq.core.pc.inventory.TimeoutException: Call to org.rhq.plugins.jbossas.JMSComponent.loadResourceConfiguration() with args [] timed out.
at org.rhq.core.pc.inventory.ResourceContainer$ResourceComponentInvocationHandler.invokeInNewThreadWithLock(ResourceContainer.java:428)
at org.rhq.core.pc.inventory.ResourceContainer$ResourceComponentInvocationHandler.invoke(ResourceContainer.java:381)
at $Proxy49.loadResourceConfiguration(Unknown Source)
at org.rhq.core.pc.configuration.ConfigurationCheckExecutor.checkConfigurations(ConfigurationCheckExecutor.java:104)
at org.rhq.core.pc.configuration.ConfigurationCheckExecutor.checkConfigurations(ConfigurationCheckExecutor.java:136)
at org.rhq.core.pc.configuration.ConfigurationCheckExecutor.checkConfigurations(ConfigurationCheckExecutor.java:136)
at org.rhq.core.pc.configuration.ConfigurationCheckExecutor.checkConfigurations(ConfigurationCheckExecutor.java:136)
at org.rhq.core.pc.configuration.ConfigurationCheckExecutor.call(ConfigurationCheckExecutor.java:68)
at org.rhq.core.pc.configuration.ConfigurationCheckExecutor.run(ConfigurationCheckExecutor.java:61)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:417)
at java.util.concurrent.FutureTask$Sync.innerRunAndReset(FutureTask.java:280)
at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:135)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(ScheduledThreadPoolExecutor.java:65)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.runPeriodic(ScheduledThreadPoolExecutor.java:142)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:166)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:650)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:675)
at java.lang.Thread.run(Thread.java:595)

[mazz]

I don't know what the problem is specifically but I can tell you the following:

1) you say something happens every "59-60 minutes". By default, every 60 minutes the agent will ask all resources for their configuration and if they differ from their last known config, the agent will tell the server someone changed the resource config. See this flash demo for why this config-change-detection is a useful feature: http://management-platform.blogspot.com/2008/12/configuration-change-detection-in-jopr.html

2) the exception you get is probably part of this config check I mention above:

2009-03-23 16:31:02,262 WARN [ConfigurationManager.threadpool-1] (rhq.core.pc.configuration.ConfigurationCheckExecutor)- Unable to check for updated configuration org.rhq.core.pc.inventory.TimeoutException: Call to org.rhq.plugins.jbossas.JMSComponent.loadResourceConfiguration()

Notice it says, "unable to check for updated config" - this looks like that hourly config detection check. But your JMS resource is not able to load the configuration in a timely manner and thus you get a timeout exception. I don't know why this happened, but its possible that when this happened, your JBossAS server would have then been considered DOWN.

Perhaps find out if this resource-change-detection is putting too much load on your JBossAS server and causing it to respond very slowly and look DOWN to the agent.

[mazz]

Hmm... for some reason, that flash demo is missing from redhat.com. I'll ask around to find out what happened to it. But that blog entry talks about this config-change-detection stuff.

[swapyeole]

Thanks for the quick response.
I to don’t know what’s wrong with this server, to test anything any possibility I have to wait till 2 hr,
I don’t have proper error no where not in jopr server / agent and jbossAS. I don’t know how to solve this.

[swapyeole]

My bad luck this time everything go’s fine and it haven’t went down. While discovery processor just spice to 100%, JbossAS was just fine. Could this is something to do with “Messages Driven EJB� this is the only partially down in this instance(Audit & SME) is working fine but some of them are red with the same name