We are having the same issue.
You could see how long it takes your jboss GCs, maybe this is the cause your Jboss availability response takes more than 5 secs.
In the agent log I see that, in some cases, availability check takes 5 secs, in our case the jboss gc is fine. I think the threads is interrupted after this 5 secs.
2013-03-11 09:40:00,325 INFO [InventoryManager.availability-1] (rhq.core.pc.inventory.AvailabilityExecutor)- Scan Starting: Mon Mar 11 09:40:00 UYT 2013
2013-03-11 09:40:05,684 INFO [InventoryManager.availability-1] (rhq.core.pc.inventory.AvailabilityExecutor)- Scan Ended : Mon Mar 11 09:40:05 UYT 2013 : Scan [startTime=1363005600325, endTime=1363005605684, runtime=5359, isFull=false, isForced=false, numResources=186, numGetAvailabilityCalls=15, numScheduledRandomly=0, numPushedByInterval=14, numAvailabilityChanges=1, numDeferToParent=0]
2013-03-11 09:40:40,354 INFO [InventoryManager.availability-1] (rhq.core.pc.inventory.AvailabilityExecutor)- Scan Starting: Mon Mar 11 09:40:40 UYT 2013
2013-03-11 09:40:45,360 INFO [InventoryManager.availability-1] (rhq.core.pc.inventory.AvailabilityExecutor)- Scan Ended : Mon Mar 11 09:40:45 UYT 2013 : Scan [startTime=1363005640354, endTime=1363005645360, runtime=5006, isFull=false, isForced=false, numResources=186, numGetAvailabilityCalls=5, numScheduledRandomly=0, numPushedByInterval=8, numAvailabilityChanges=143, numDeferToParent=149]
2013-03-11 09:41:20,620 INFO [InventoryManager.availability-1] (rhq.core.pc.inventory.AvailabilityExecutor)- Scan Starting: Mon Mar 11 09:41:20 UYT 2013
2013-03-11 09:41:20,622 INFO [InventoryManager.availability-1] (rhq.core.pc.inventory.AvailabilityExecutor)- Scan Ended : Mon Mar 11 09:41:20 UYT 2013 : Scan [startTime=1363005680620, endTime=1363005680622, runtime=2, isFull=false, isForced=false, numResources=186, numGetAvailabilityCalls=2, numScheduledRandomly=0, numPushedByInterval=10, numAvailabilityChanges=0, numDeferToParent=149]
2013-03-11 09:41:50,628 INFO [InventoryManager.availability-1] (rhq.core.pc.inventory.AvailabilityExecutor)- Scan Starting: Mon Mar 11 09:41:50 UYT 2013
2013-03-11 09:41:55,888 INFO [InventoryManager.availability-1] (rhq.core.pc.inventory.AvailabilityExecutor)- Scan Ended : Mon Mar 11 09:41:55 UYT 2013 : Scan [startTime=1363005710627, endTime=1363005715888, runtime=5261, isFull=false, isForced=false, numResources=186, numGetAvailabilityCalls=154, numScheduledRandomly=149, numPushedByInterval=4, numAvailabilityChanges=144, numDeferToParent=0]
2013-03-11 09:42:26,155 INFO [InventoryManager.availability-1] (rhq.core.pc.inventory.AvailabilityExecutor)- Scan Starting: Mon Mar 11 09:42:26 UYT 2013
2013-03-11 09:42:31,273 INFO [InventoryManager.availability-1] (rhq.core.pc.inventory.AvailabilityExecutor)- Scan Ended : Mon Mar 11 09:42:31 UYT 2013 : Scan [startTime=1363005746155, endTime=1363005751273, runtime=5118, isFull=false, isForced=false, numResources=186, numGetAvailabilityCalls=7, numScheduledRandomly=0, numPushedByInterval=6, numAvailabilityChanges=1, numDeferToParent=0]
2013-03-11 09:43:01,291 INFO [InventoryManager.availability-1] (rhq.core.pc.inventory.AvailabilityExecutor)- Scan Starting: Mon Mar 11 09:43:01 UYT 2013
2013-03-11 09:43:06,297 INFO [InventoryManager.availability-1] (rhq.core.pc.inventory.AvailabilityExecutor)- Scan Ended : Mon Mar 11 09:43:06 UYT 2013 : Scan [startTime=1363005781291, endTime=1363005786297, runtime=5006, isFull=false, isForced=false, numResources=186, numGetAvailabilityCalls=6, numScheduledRandomly=0, numPushedByInterval=16, numAvailabilityChanges=143, numDeferToParent=149]
2013-03-11 09:43:41,565 INFO [InventoryManager.availability-1] (rhq.core.pc.inventory.AvailabilityExecutor)- Scan Starting: Mon Mar 11 09:43:41 UYT 2013
2013-03-11 09:43:41,568 INFO [InventoryManager.availability-1] (rhq.core.pc.inventory.AvailabilityExecutor)- Scan Ended : Mon Mar 11 09:43:41 UYT 2013 : Scan [startTime=1363005821565, endTime=1363005821568, runtime=3, isFull=false, isForced=false, numResources=186, numGetAvailabilityCalls=6, numScheduledRandomly=0, numPushedByInterval=15, numAvailabilityChanges=0, numDeferToParent=149]
2013-03-11 09:44:11,573 INFO [InventoryManager.availability-1] (rhq.core.pc.inventory.AvailabilityExecutor)- Scan Starting: Mon Mar 11 09:44:11 UYT 2013
2013-03-11 09:44:17,047 INFO [InventoryManager.availability-1] (rhq.core.pc.inventory.AvailabilityExecutor)- Scan Ended : Mon Mar 11 09:44:17 UYT 2013 : Scan [startTime=1363005851573, endTime=1363005857047, runtime=5474, isFull=false, isForced=false, numResources=186, numGetAvailabilityCalls=157, numScheduledRandomly=149, numPushedByInterval=7, numAvailabilityChanges=144, numDeferToParent=0]
2013-03-11 09:44:47,312 INFO [InventoryManager.availability-1] (rhq.core.pc.inventory.AvailabilityExecutor)- Scan Starting: Mon Mar 11 09:44:47 UYT 2013
2013-03-11 09:44:47,743 INFO [InventoryManager.availability-1] (rhq.core.pc.inventory.AvailabilityExecutor)- Scan Ended : Mon Mar 11 09:44:47 UYT 2013 :
I'm going to try setting rhq.agent.plugins.availability-scan.timeout in more than 5 secs in the agent.
Juan