5 Replies Latest reply on Mar 29, 2011 10:19 AM by spolti

    Agent callback failled

    krz-yamaneko

      Hi maybe someone can help me, my agent doesn't send information to my server, I have many errors on agent and server logs.

      The agent.log on the client:

      2009-02-10 10:26:54,946 INFO [main] (org.rhq.enterprise.communications.ServiceContainer)- {ServiceContainer.global-concurrency-limit-disabled}Global concurrency limit has been disabled - there is no limit to the number of incoming commands allowed
      2009-02-10 10:26:54,997 INFO [main] (enterprise.communications.command.server.CommandServiceDirectory)- {CommandServiceDirectory.dynamic-discovery-allowed}Command Service Directory has been allowed to perform dynamic discovery
      2009-02-10 10:26:55,008 INFO [main] (enterprise.communications.command.server.CommandServiceDirectory)- {CommandServiceDirectory.new-supported-command}Command [echo] is now supported by [rhq.remoting:type=command,subsystem=RHQ,id=1]
      2009-02-10 10:26:55,008 INFO [main] (enterprise.communications.command.server.CommandServiceDirectory)- {CommandServiceDirectory.new-supported-command}Command [identify] is now supported by [rhq.remoting:type=command,subsystem=RHQ,id=2]
      2009-02-10 10:26:55,010 INFO [main] (enterprise.communications.command.server.CommandServiceDirectory)- {CommandServiceDirectory.new-supported-command}Command [remotepojo] is now supported by [rhq.remoting:type=command,subsystem=RHQ,id=3]
      2009-02-10 10:26:55,061 INFO [main] (org.rhq.enterprise.communications.ServiceContainer)- {ServiceContainer.started}Service container started - ready to accept incoming commands
      2009-02-10 10:26:55,153 INFO [RHQ Server Polling Thread] (org.rhq.enterprise.agent.AgentMain)- {AgentMain.failed-over-to-server}The agent has triggered its failover mechanism and switched to server [servlet://althea-desktop:7080/jboss-remoting-servlet-invoker/ServerInvokerServlet]
      2009-02-10 10:26:55,154 WARN [RHQ Server Polling Thread] (org.rhq.enterprise.agent.FailoverFailureCallback)- {AgentMain.too-many-failover-attempts}Too many failover attempts have been made [1]. Exception that triggered the failover: [org.jboss.remoting.CannotConnectException: Can not connect http client invoker.]
      2009-02-10 10:26:55,155 ERROR [RHQ Server Polling Thread] (enterprise.communications.command.client.JBossRemotingRemoteCommunicator)- {JBossRemotingRemoteCommunicator.init-callback-failed}The initialize callback has failed. It will be tried again. Cause: org.jboss.remoting.CannotConnectException:Can not connect http client invoker. -> java.net.UnknownHostException:althea-desktop. Cause: org.jboss.remoting.CannotConnectException: Can not connect http client invoker.
      2009-02-10 10:26:56,170 WARN [ClientCommandSenderTask Timer Thread #0] (org.rhq.enterprise.agent.FailoverFailureCallback)- {AgentMain.too-many-failover-attempts}Too many failover attempts have been made [1]. Exception that triggered the failover: [org.jboss.remoting.CannotConnectException: Can not connect http client invoker.]
      2009-02-10 10:26:56,170 ERROR [RHQ Agent Registration Thread] (enterprise.communications.command.client.ClientCommandSenderTask)- {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[registerAgent], targetInterfaceName=org.rhq.core.clientapi.server.core.CoreServerService}]]. Cause: org.jboss.remoting.CannotConnectException:Can not connect http client invoker. -> java.net.UnknownHostException:althea-desktop. Cause: org.jboss.remoting.CannotConnectException: Can not connect http client invoker.
      2009-02-10 10:26:56,181 WARN [RHQ Agent Registration Thread] (org.rhq.enterprise.agent.AgentMain)- {AgentMain.agent-registration-failure}Agent failed to register with the server. retry=[true], retry interval=[2,000]. Cause: org.jboss.remoting.CannotConnectException:Can not connect http client invoker. -> java.net.UnknownHostException:althea-desktop. Cause: org.jboss.remoting.CannotConnectException: Can not connect http client invoker.
      2009-02-10 10:26:58,182 WARN [ClientCommandSenderTask Timer Thread #0] (org.rhq.enterprise.agent.FailoverFailureCallback)- {AgentMain.too-many-failover-attempts}Too many failover attempts have been made [1]. Exception that triggered the failover: [org.jboss.remoting.CannotConnectException: Can not connect http client invoker.]
      2009-02-10 10:26:58,182 ERROR [RHQ Agent Registration Thread] (enterprise.communications.command.client.ClientCommandSenderTask)- {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[registerAgent], targetInterfaceName=org.rhq.core.clientapi.server.core.CoreServerService}]]. Cause: org.jboss.remoting.CannotConnectException:Can not connect http client invoker. -> java.net.UnknownHostException:althea-desktop. Cause: org.jboss.remoting.CannotConnectException: Can not connect http client invoker.
      2009-02-10 10:26:58,186 WARN [RHQ Agent Registration Thread] (org.rhq.enterprise.agent.AgentMain)- {AgentMain.agent-registration-failure}Agent failed to register with the server. retry=[true], retry interval=[4,000]. Cause: org.jboss.remoting.CannotConnectException:Can not connect http client invoker. -> java.net.UnknownHostException:althea-desktop. Cause: org.jboss.remoting.CannotConnectException: Can not connect http client invoker.
      2009-02-10 10:27:02,188 WARN [ClientCommandSenderTask Timer Thread #0] (org.rhq.enterprise.agent.FailoverFailureCallback)- {AgentMain.too-many-failover-attempts}Too many failover attempts have been made [1]. Exception that triggered the failover: [org.jboss.remoting.CannotConnectException: Can not connect http client invoker.]
      2009-02-10 10:27:02,188 ERROR [RHQ Agent Registration Thread] (enterprise.communications.command.client.ClientCommandSenderTask)- {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[registerAgent], targetInterfaceName=org.rhq.core.clientapi.server.core.CoreServerService}]]. Cause: org.jboss.remoting.CannotConnectException:Can not connect http client invoker. -> java.net.UnknownHostException:althea-desktop. Cause: org.jboss.remoting.CannotConnectException: Can not connect http client invoker.
      2009-02-10 10:27:02,192 WARN [RHQ Agent Registration Thread] (org.rhq.enterprise.agent.AgentMain)- {AgentMain.agent-registration-failure}Agent failed to register with the server. retry=[true], retry interval=[8,000]. Cause: org.jboss.remoting.CannotConnectException:Can not connect http client invoker. -> java.net.UnknownHostException:althea-desktop. Cause: org.jboss.remoting.CannotConnectException: Can not connect http client invoker.
      2009-02-10 10:27:10,214 WARN [ClientCommandSenderTask Timer Thread #0] (org.rhq.enterprise.agent.AgentMain)- {AgentMain.failover-failed}Failed to failover to another server. Cause: org.jboss.remoting.CannotConnectException: Can not connect http client invoker.
      2009-02-10 10:27:10,215 WARN [ClientCommandSenderTask Timer Thread #0] (org.rhq.enterprise.agent.FailoverFailureCallback)- {AgentMain.too-many-failover-attempts}Too many failover attempts have been made [1]. Exception that triggered the failover: [org.jboss.remoting.CannotConnectException: Can not connect http client invoker.]
      2009-02-10 10:27:10,215 ERROR [RHQ Agent Registration Thread] (enterprise.communications.command.client.ClientCommandSenderTask)- {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[registerAgent], targetInterfaceName=org.rhq.core.clientapi.server.core.CoreServerService}]]. Cause: org.jboss.remoting.CannotConnectException:Can not connect http client invoker. -> java.net.UnknownHostException:althea-desktop. Cause: org.jboss.remoting.CannotConnectException: Can not connect http client invoker.
      2009-02-10 10:27:10,224 WARN [RHQ Agent Registration Thread] (org.rhq.enterprise.agent.AgentMain)- {AgentMain.agent-registration-failure}Agent failed to register with the server. retry=[true], retry interval=[16,000]. Cause: org.jboss.remoting.CannotConnectException:Can not connect http client invoker. -> java.net.UnknownHostException:althea-desktop. Cause: org.jboss.remoting.CannotConnectException: Can not connect http client invoker.
      2009-02-10 10:27:26,244 WARN [ClientCommandSenderTask Timer Thread #0] (org.rhq.enterprise.agent.AgentMain)- {AgentMain.failover-failed}Failed to failover to another server. Cause: org.jboss.remoting.CannotConnectException: Can not connect http client invoker.
      2009-02-10 10:27:26,244 WARN [ClientCommandSenderTask Timer Thread #0] (org.rhq.enterprise.agent.FailoverFailureCallback)- {AgentMain.too-many-failover-attempts}Too many failover attempts have been made [1]. Exception that triggered the failover: [org.jboss.remoting.CannotConnectException: Can not connect http client invoker.]
      2009-02-10 10:27:26,245 ERROR [RHQ Agent Registration Thread] (enterprise.communications.command.client.ClientCommandSenderTask)- {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[registerAgent], targetInterfaceName=org.rhq.core.clientapi.server.core.CoreServerService}]]. Cause: org.jboss.remoting.CannotConnectException:Can not connect http client invoker. -> java.net.UnknownHostException:althea-desktop. Cause: org.jboss.remoting.CannotConnectException: Can not connect http client invoker.
      2009-02-10 10:27:26,253 WARN [RHQ Agent Registration Thread] (org.rhq.enterprise.agent.AgentMain)- {AgentMain.agent-registration-failure}Agent failed to register with the server. retry=[true], retry interval=[32,000]. Cause: org.jboss.remoting.CannotConnectException:Can not connect http client invoker. -> java.net.UnknownHostException:althea-desktop. Cause: org.jboss.remoting.CannotConnectException: Can not connect http client invoker.
      2009-02-10 10:27:55,302 INFO [ClientCommandSenderTask Timer Thread #0] (org.rhq.enterprise.agent.AgentMain)- {AgentMain.failed-over-to-server}The agent has triggered its failover mechanism and switched to server [servlet://althea-desktop:7080/jboss-remoting-servlet-invoker/ServerInvokerServlet]
      2009-02-10 10:27:55,303 WARN [ClientCommandSenderTask Timer Thread #0] (org.rhq.enterprise.agent.FailoverFailureCallback)- {AgentMain.too-many-failover-attempts}Too many failover attempts have been made [1]. Exception that triggered the failover: [org.jboss.remoting.CannotConnectException: Can not connect http client invoker.]
      2009-02-10 10:27:55,303 ERROR [ClientCommandSenderTask Timer Thread #0] (enterprise.communications.command.client.JBossRemotingRemoteCommunicator)- {JBossRemotingRemoteCommunicator.init-callback-failed}The initialize callback has failed. It will be tried again. Cause: org.jboss.remoting.CannotConnectException:Can not connect http client invoker. -> java.net.UnknownHostException:althea-desktop. Cause: org.jboss.remoting.CannotConnectException: Can not connect http client invoker.
      2009-02-10 10:27:55,303 WARN [RHQ Server Polling Thread] (org.rhq.enterprise.agent.AgentMain)- {AgentMain.plugin-update-failure}Failed to update the plugins.. Cause: org.jboss.remoting.CannotConnectException: Can not connect http client invoker.
      2009-02-10 10:27:55,304 INFO [RHQ Server Polling Thread] (enterprise.communications.command.client.ServerPollingThread)- {ServerPollingThread.server-online}The server has come back online; client has been told to start sending commands again
      2009-02-10 10:27:57,107 INFO [main] (org.rhq.core.pc.PluginContainer)- Initializing Plugin Container v1.1.0.GA...
      2009-02-10 10:27:58,254 WARN [ClientCommandSenderTask Timer Thread #0] (org.rhq.enterprise.agent.FailoverFailureCallback)- {AgentMain.too-many-failover-attempts}Too many failover attempts have been made [1]. Exception that triggered the failover: [org.jboss.remoting.CannotConnectException: Can not connect http client invoker.]
      2009-02-10 10:27:58,255 ERROR [RHQ Agent Registration Thread] (enterprise.communications.command.client.ClientCommandSenderTask)- {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[registerAgent], targetInterfaceName=org.rhq.core.clientapi.server.core.CoreServerService}]]. Cause: org.jboss.remoting.CannotConnectException:Can not connect http client invoker. -> java.net.UnknownHostException:althea-desktop. Cause: org.jboss.remoting.CannotConnectException: Can not connect http client invoker.
      2009-02-10 10:27:58,262 WARN [RHQ Agent Registration Thread] (org.rhq.enterprise.agent.AgentMain)- {AgentMain.agent-registration-failure}Agent failed to register with the server. retry=[true], retry interval=[64,000]. Cause: org.jboss.remoting.CannotConnectException:Can not connect http client invoker. -> java.net.UnknownHostException:althea-desktop. Cause: org.jboss.remoting.CannotConnectException: Can not connect http client invoker.
      2009-02-10 10:28:00,334 INFO [main] (rhq.core.pc.inventory.InventoryManager)- Initializing Inventory Manager...
      2009-02-10 10:28:00,336 INFO [main] (rhq.core.pc.inventory.InventoryManager)- Loading inventory from persistent data file
      2009-02-10 10:28:00,380 INFO [main] (rhq.core.pc.inventory.InventoryManager)- Inventory size [17] initialized from disk in [44ms]
      2009-02-10 10:28:00,415 INFO [main] (rhq.core.pc.inventory.InventoryManager)- Inventory Manager initialized.
      2009-02-10 10:28:00,419 INFO [main] (rhq.core.pc.inventory.ResourceFactoryManager)- Initializing
      2009-02-10 10:28:00,420 INFO [main] (rhq.core.pc.content.ContentManager)- Initializing Content Manager...
      2009-02-10 10:28:00,420 INFO [main] (rhq.core.pc.content.ContentManager)- Initializing scheduled content discovery...
      2009-02-10 10:28:00,421 INFO [main] (rhq.core.pc.content.ContentManager)- Content Manager initialized...
      2009-02-10 10:28:00,430 INFO [RHQ Primary Server Switchover Thread] (org.rhq.enterprise.agent.AgentMain)- {PrimaryServerSwitchoverThread.started}The primary server switchover thread has started.
      2009-02-10 10:28:00,431 WARN [ClientCommandSenderTask Timer Thread #0] (org.rhq.enterprise.agent.FailoverFailureCallback)- {AgentMain.too-many-failover-attempts}Too many failover attempts have been made [1]. Exception that triggered the failover: [org.jboss.remoting.CannotConnectException: Can not connect http client invoker.]
      2009-02-10 10:28:00,431 ERROR [ClientCommandSenderTask Timer Thread #0] (enterprise.communications.command.client.JBossRemotingRemoteCommunicator)- {JBossRemotingRemoteCommunicator.init-callback-failed}The initialize callback has failed. It will be tried again. Cause: org.jboss.remoting.CannotConnectException:Can not connect http client invoker. -> java.net.UnknownHostException:althea-desktop. Cause: org.jboss.remoting.CannotConnectException: Can not connect http client invoker.
      2009-02-10 10:28:00,431 WARN [RHQ Primary Server Switchover Thread] (org.rhq.enterprise.agent.AgentMain)- {AgentMain.failover-list-download-failure}Failed to download an updated server failover list. Cause: org.jboss.remoting.CannotConnectException: Can not connect http client invoker.
      2009-02-10 10:28:10,413 INFO [InventoryManager.discovery-1] (rhq.core.pc.inventory.AutoDiscoveryExecutor)- Executing server discovery scan...
      2009-02-10 10:28:10,476 INFO [InventoryManager.discovery-1] (rhq.core.pc.inventory.AutoDiscoveryExecutor)- Process scan auto-detected new server resource: scan=[ProcessScan: query=[process|basename|match=sshd,process|basename|nomatch|parent=sshd], name=[SSHD]], discovered-process=[process: pid=[3101], name=[/usr/sbin/sshd], ppid=[1]]
      2009-02-10 10:28:10,577 INFO [InventoryManager.discovery-1] (org.rhq.plugins.tomcat.TomcatDiscoveryComponent)- Discovering Tomcat servers
      2009-02-10 10:28:10,579 INFO [InventoryManager.discovery-1] (org.rhq.plugins.agent.AgentDiscoveryComponent)- Discovering RHQ Agent...
      2009-02-10 10:28:10,629 INFO [InventoryManager.discovery-1] (rhq.core.pc.inventory.AutoDiscoveryExecutor)- Process scan auto-detected new server resource: scan=[ProcessScan: query=[process|basename|match=^java.*,arg|org.jboss.Main|match=.*], name=[JBoss4]], discovered-process=[process: pid=[14165], name=[/opt/jdk1.6.0_10/bin/java], ppid=[14000]]
      2009-02-10 10:28:10,629 INFO [InventoryManager.discovery-1] (rhq.core.pc.inventory.AutoDiscoveryExecutor)- Process scan auto-detected new server resource: scan=[ProcessScan: query=[process|basename|match=^java.*,arg|org.jboss.Main|match=.*], name=[JBoss4]], discovered-process=[process: pid=[14152], name=[/opt/jdk1.6.0_10/bin/java], ppid=[14010]]
      2009-02-10 10:28:10,629 INFO [InventoryManager.discovery-1] (rhq.core.pc.inventory.AutoDiscoveryExecutor)- Process scan auto-detected new server resource: scan=[ProcessScan: query=[process|basename|match=^java.*,arg|org.jboss.Main|match=.*], name=[JBoss4]], discovered-process=[process: pid=[14226], name=[/opt/jdk1.6.0_10/bin/java], ppid=[14060]]
      2009-02-10 10:28:10,629 INFO [InventoryManager.discovery-1] (rhq.core.pc.inventory.AutoDiscoveryExecutor)- Process scan auto-detected new server resource: scan=[ProcessScan: query=[process|basename|match=^java.*,arg|org.jboss.Main|match=.*], name=[JBoss4]], discovered-process=[process: pid=[14223], name=[/opt/jdk1.6.0_10/bin/java], ppid=[14028]]
      2009-02-10 10:28:10,629 INFO [InventoryManager.discovery-1] (rhq.core.pc.inventory.AutoDiscoveryExecutor)- Process scan auto-detected new server resource: scan=[ProcessScan: query=[process|basename|match=^java.*,arg|org.jboss.Main|match=.*], name=[JBoss4]], discovered-process=[process: pid=[14281], name=[/opt/jdk1.6.0_10/bin/java], ppid=[14094]]
      2009-02-10 10:28:10,629 INFO [InventoryManager.discovery-1] (rhq.core.pc.inventory.AutoDiscoveryExecutor)- Process scan auto-detected new server resource: scan=[ProcessScan: query=[process|basename|match=^java.*,arg|org.jboss.Main|match=.*], name=[JBoss4]], discovered-process=[process: pid=[14323], name=[/opt/jdk1.6.0_10/bin/java], ppid=[14146]]
      2009-02-10 10:28:10,629 INFO [InventoryManager.discovery-1] (rhq.core.pc.inventory.AutoDiscoveryExecutor)- Process scan auto-detected new server resource: scan=[ProcessScan: query=[process|basename|match=^java.*,arg|org.jboss.Main|match=.*], name=[JBoss4]], discovered-process=[process: pid=[14307], name=[/opt/jdk1.6.0_10/bin/java], ppid=[14122]]
      2009-02-10 10:28:10,629 INFO [InventoryManager.discovery-1] (rhq.core.pc.inventory.AutoDiscoveryExecutor)- Process scan auto-detected new server resource: scan=[ProcessScan: query=[process|basename|match=^java.*,arg|org.jboss.Main|match=.*], name=[JBoss4]], discovered-process=[process: pid=[14337], name=[/opt/jdk1.6.0_10/bin/java], ppid=[14187]]
      2009-02-10 10:28:10,629 INFO [InventoryManager.discovery-1] (rhq.core.pc.inventory.AutoDiscoveryExecutor)- Process scan auto-detected new server resource: scan=[ProcessScan: query=[process|basename|match=^java.*,arg|org.jboss.Main|match=.*], name=[JBoss4]], discovered-process=[process: pid=[14347], name=[/opt/jdk1.6.0_10/bin/java], ppid=[14173]]
      2009-02-10 10:28:10,629 INFO [InventoryManager.discovery-1] (rhq.core.pc.inventory.AutoDiscoveryExecutor)- Process scan auto-detected new server resource: scan=[ProcessScan: query=[process|basename|match=^java.*,arg|org.jboss.Main|match=.*], name=[JBoss4]], discovered-process=[process: pid=[14349], name=[/opt/jdk1.6.0_10/bin/java], ppid=[14133]]
      2009-02-10 10:28:10,629 INFO [InventoryManager.discovery-1] (org.rhq.plugins.jbossas.JBossASDiscoveryComponent)- Discovering JBossAS 3.2.x and 4.x servers...
      2009-02-10 10:28:10,841 INFO [InventoryManager.discovery-1] (rhq.core.pc.inventory.AutoDiscoveryExecutor)- Process scan auto-detected new server resource: scan=[ProcessScan: query=[process|basename|match=^mysqld$,process|basename|nomatch|parent=^mysqld$], name=[unix]], discovered-process=[process: pid=[3200], name=[/usr/sbin/mysqld], ppid=[3158]]
      2009-02-10 10:28:10,842 INFO [InventoryManager.discovery-1] (org.rhq.plugins.mysql.MySqlDiscoveryComponent)- Discovered a mysql process: ProcessScanResult: scan=[ProcessScan: query=[process|basename|match=^mysqld$,process|basename|nomatch|parent=^mysqld$], name=[unix]], info=[process: pid=[3200], name=[/usr/sbin/mysqld], ppid=[3158]]
      2009-02-10 10:28:10,885 INFO [InventoryManager.discovery-1] (rhq.core.pc.inventory.AutoDiscoveryExecutor)- Process scan auto-detected new server resource: scan=[ProcessScan: query=[process|basename|match=apache2,process|basename|nomatch|parent=apache2], name=[Apache2OnUnix]], discovered-process=[process: pid=[3572], name=[/usr/sbin/apache2], ppid=[1]]
      2009-02-10 10:28:11,041 INFO [InventoryManager.discovery-1] (rhq.plugins.www.snmp.SNMPClient)- Initialized SNMP session for agent at /127.0.0.1:1610
      2009-02-10 10:28:11,150 INFO [InventoryManager.discovery-1] (rhq.core.pc.inventory.InventoryManager)- Sending server inventory report to Server...
      2009-02-10 10:28:11,185 INFO [ClientCommandSenderTask Timer Thread #0] (org.rhq.enterprise.agent.AgentMain)- {AgentMain.failed-over-to-server}The agent has triggered its failover mechanism and switched to server [servlet://althea-desktop:7080/jboss-remoting-servlet-invoker/ServerInvokerServlet]
      2009-02-10 10:28:11,186 WARN [ClientCommandSenderTask Timer Thread #0] (org.rhq.enterprise.agent.FailoverFailureCallback)- {AgentMain.too-many-failover-attempts}Too many failover attempts have been made [1]. Exception that triggered the failover: [org.jboss.remoting.CannotConnectException: Can not connect http client invoker.]
      2009-02-10 10:28:11,186 ERROR [ClientCommandSenderTask Timer Thread #0] (enterprise.communications.command.client.JBossRemotingRemoteCommunicator)- {JBossRemotingRemoteCommunicator.init-callback-failed}The initialize callback has failed. It will be tried again. Cause: org.jboss.remoting.CannotConnectException:Can not connect http client invoker. -> java.net.UnknownHostException:althea-desktop. Cause: org.jboss.remoting.CannotConnectException: Can not connect http client invoker.
      2009-02-10 10:28:11,186 WARN [InventoryManager.discovery-1] (rhq.core.pc.inventory.AutoDiscoveryExecutor)- Exception caught while running server discovery
      org.jboss.remoting.CannotConnectException: Can not connect http client invoker.
       at org.jboss.remoting.transport.http.HTTPClientInvoker.useHttpURLConnection(HTTPClientInvoker.java:332)
       at org.jboss.remoting.transport.http.HTTPClientInvoker.transport(HTTPClientInvoker.java:135)
       at org.jboss.remoting.MicroRemoteClientInvoker.invoke(MicroRemoteClientInvoker.java:122)
       at org.jboss.remoting.Client.invoke(Client.java:1550)
       at org.jboss.remoting.Client.invoke(Client.java:530)
       at org.rhq.enterprise.communications.command.client.JBossRemotingRemoteCommunicator.sendWithoutInitializeCallback(JBossRemotingRemoteCommunicator.java:449)
       at org.rhq.enterprise.agent.AgentMain.sendConnectRequestToServer(AgentMain.java:1516)
       at org.rhq.enterprise.agent.ConnectAgentInitializeCallback.sendingInitialCommand(ConnectAgentInitializeCallback.java:43)
       at org.rhq.enterprise.communications.command.client.JBossRemotingRemoteCommunicator.invokeInitializeCallbackIfNeeded(JBossRemotingRemoteCommunicator.java:557)
       at org.rhq.enterprise.communications.command.client.JBossRemotingRemoteCommunicator.send(JBossRemotingRemoteCommunicator.java:488)
       at org.rhq.enterprise.communications.command.client.AbstractCommandClient.invoke(AbstractCommandClient.java:143)
       at org.rhq.enterprise.communications.command.client.ClientCommandSender.send(ClientCommandSender.java:994)
       at org.rhq.enterprise.communications.command.client.ClientCommandSenderTask.send(ClientCommandSenderTask.java:243)
       at org.rhq.enterprise.communications.command.client.ClientCommandSenderTask.call(ClientCommandSenderTask.java:112)
       at org.rhq.enterprise.communications.command.client.ClientCommandSenderTask.call(ClientCommandSenderTask.java:56)
       at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
       at java.util.concurrent.FutureTask.run(FutureTask.java:138)
       at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
       at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
       at java.lang.Thread.run(Thread.java:619)
      Caused by: java.net.UnknownHostException: althea-desktop
       at java.net.PlainSocketImpl.connect(PlainSocketImpl.java:177)
       at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:366)
       at java.net.Socket.connect(Socket.java:519)
       at java.net.Socket.connect(Socket.java:469)
       at sun.net.NetworkClient.doConnect(NetworkClient.java:163)
       at sun.net.www.http.HttpClient.openServer(HttpClient.java:394)
       at sun.net.www.http.HttpClient.openServer(HttpClient.java:529)
       at sun.net.www.http.HttpClient.<init>(HttpClient.java:233)
       at sun.net.www.http.HttpClient.New(HttpClient.java:306)
       at sun.net.www.http.HttpClient.New(HttpClient.java:323)
       at sun.net.www.protocol.http.HttpURLConnection.getNewHttpClient(HttpURLConnection.java:837)
       at sun.net.www.protocol.http.HttpURLConnection.plainConnect(HttpURLConnection.java:778)
       at sun.net.www.protocol.http.HttpURLConnection.connect(HttpURLConnection.java:703)
       at sun.net.www.protocol.http.HttpURLConnection.getOutputStream(HttpURLConnection.java:881)
       at org.jboss.remoting.transport.http.HTTPClientInvoker.useHttpURLConnection(HTTPClientInvoker.java:274)
       ... 19 more
      2009-02-10 10:28:11,188 INFO [InventoryManager.discovery-1] (rhq.core.pc.inventory.AutoDiscoveryExecutor)- Found 1 servers.
      2009-02-10 10:28:20,414 INFO [InventoryManager.discovery-1] (rhq.core.pc.inventory.RuntimeDiscoveryExecutor)- Running runtime discovery scan rooted at platform...
      2009-02-10 10:28:20,415 INFO [InventoryManager.discovery-1] (rhq.core.pc.inventory.RuntimeDiscoveryExecutor)- Scanned 0 servers and found 0 total descendant Resources.
      2009-02-10 10:28:20,415 INFO [InventoryManager.discovery-1] (rhq.core.pc.inventory.InventoryManager)- Sending runtime inventory report to Server...
      2009-02-10 10:28:20,416 WARN [ClientCommandSenderTask Timer Thread #0] (org.rhq.enterprise.agent.FailoverFailureCallback)- {AgentMain.too-many-failover-attempts}Too many failover attempts have been made [1]. Exception that triggered the failover: [org.jboss.remoting.CannotConnectException: Can not connect http client invoker.]
      2009-02-10 10:28:20,416 ERROR [ClientCommandSenderTask Timer Thread #0] (enterprise.communications.command.client.JBossRemotingRemoteCommunicator)- {JBossRemotingRemoteCommunicator.init-callback-failed}The initialize callback has failed. It will be tried again. Cause: org.jboss.remoting.CannotConnectException:Can not connect http client invoker. -> java.net.UnknownHostException:althea-desktop. Cause: org.jboss.remoting.CannotConnectException: Can not connect http client invoker.
      2009-02-10 10:28:20,416 ERROR [InventoryManager.discovery-1] (rhq.core.pc.inventory.RuntimeDiscoveryExecutor)- Error running runtime report
      org.jboss.remoting.CannotConnectException: Can not connect http client invoker.
       at org.jboss.remoting.transport.http.HTTPClientInvoker.useHttpURLConnection(HTTPClientInvoker.java:332)
       at org.jboss.remoting.transport.http.HTTPClientInvoker.transport(HTTPClientInvoker.java:135)
       at org.jboss.remoting.MicroRemoteClientInvoker.invoke(MicroRemoteClientInvoker.java:122)
       at org.jboss.remoting.Client.invoke(Client.java:1550)
       at org.jboss.remoting.Client.invoke(Client.java:530)
       at org.rhq.enterprise.communications.command.client.JBossRemotingRemoteCommunicator.sendWithoutInitializeCallback(JBossRemotingRemoteCommunicator.java:449)
       at org.rhq.enterprise.agent.AgentMain.sendConnectRequestToServer(AgentMain.java:1516)
       at org.rhq.enterprise.agent.ConnectAgentInitializeCallback.sendingInitialCommand(ConnectAgentInitializeCallback.java:43)
       at org.rhq.enterprise.communications.command.client.JBossRemotingRemoteCommunicator.invokeInitializeCallbackIfNeeded(JBossRemotingRemoteCommunicator.java:557)
       at org.rhq.enterprise.communications.command.client.JBossRemotingRemoteCommunicator.send(JBossRemotingRemoteCommunicator.java:488)
       at org.rhq.enterprise.communications.command.client.AbstractCommandClient.invoke(AbstractCommandClient.java:143)
       at org.rhq.enterprise.communications.command.client.ClientCommandSender.send(ClientCommandSender.java:994)
       at org.rhq.enterprise.communications.command.client.ClientCommandSenderTask.send(ClientCommandSenderTask.java:243)
       at org.rhq.enterprise.communications.command.client.ClientCommandSenderTask.call(ClientCommandSenderTask.java:112)
       at org.rhq.enterprise.communications.command.client.ClientCommandSenderTask.call(ClientCommandSenderTask.java:56)
       at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
       at java.util.concurrent.FutureTask.run(FutureTask.java:138)
       at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
       at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
       at java.lang.Thread.run(Thread.java:619)
      Caused by: java.net.UnknownHostException: althea-desktop
       at java.net.PlainSocketImpl.connect(PlainSocketImpl.java:177)
       at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:366)
       at java.net.Socket.connect(Socket.java:519)
       at java.net.Socket.connect(Socket.java:469)
       at sun.net.NetworkClient.doConnect(NetworkClient.java:163)
       at sun.net.www.http.HttpClient.openServer(HttpClient.java:394)
       at sun.net.www.http.HttpClient.openServer(HttpClient.java:529)
       at sun.net.www.http.HttpClient.<init>(HttpClient.java:233)
       at sun.net.www.http.HttpClient.New(HttpClient.java:306)
       at sun.net.www.http.HttpClient.New(HttpClient.java:323)
       at sun.net.www.protocol.http.HttpURLConnection.getNewHttpClient(HttpURLConnection.java:837)
       at sun.net.www.protocol.http.HttpURLConnection.plainConnect(HttpURLConnection.java:778)
       at sun.net.www.protocol.http.HttpURLConnection.connect(HttpURLConnection.java:703)
       at sun.net.www.protocol.http.HttpURLConnection.getOutputStream(HttpURLConnection.java:881)
       at org.jboss.remoting.transport.http.HTTPClientInvoker.useHttpURLConnection(HTTPClientInvoker.java:274)
       ... 19 more
      2009-02-10 10:28:30,987 INFO [Thread-4] (org.rhq.enterprise.agent.AgentMain)- {AgentMain.shutting-down}Agent is being shut down...
      2009-02-10 10:28:30,987 INFO [RHQ Primary Server Switchover Thread] (org.rhq.enterprise.agent.AgentMain)- {PrimaryServerSwitchoverThread.stopped}The primary server switchover thread has stopped.
      2009-02-10 10:28:30,987 INFO [Thread-4] (rhq.core.pc.content.ContentManager)- Shutting down Content Manager...
      2009-02-10 10:28:30,988 INFO [MeasurementManager.collector-1] (rhq.core.pc.measurement.MeasurementManager)- Shutting down measurement collection...
      2009-02-10 10:28:31,029 INFO [ClientCommandSenderTask Timer Thread #0] (org.rhq.enterprise.agent.AgentMain)- {AgentMain.failed-over-to-server}The agent has triggered its failover mechanism and switched to server [servlet://althea-desktop:7080/jboss-remoting-servlet-invoker/ServerInvokerServlet]
      2009-02-10 10:28:31,030 WARN [ClientCommandSenderTask Timer Thread #0] (org.rhq.enterprise.agent.FailoverFailureCallback)- {AgentMain.too-many-failover-attempts}Too many failover attempts have been made [1]. Exception that triggered the failover: [org.jboss.remoting.CannotConnectException: Can not connect http client invoker.]
      2009-02-10 10:28:31,030 ERROR [ClientCommandSenderTask Timer Thread #0] (enterprise.communications.command.client.JBossRemotingRemoteCommunicator)- {JBossRemotingRemoteCommunicator.init-callback-failed}The initialize callback has failed. It will be tried again. Cause: org.jboss.remoting.CannotConnectException:Can not connect http client invoker. -> java.net.UnknownHostException:althea-desktop. Cause: org.jboss.remoting.CannotConnectException: Can not connect http client invoker.
      2009-02-10 10:28:31,033 WARN [Thread-4] (org.rhq.enterprise.agent.AgentMain)- {AgentMain.server-shutdown-notification-failure}Agent failed to notify the server of the pending shutdown. Cause: org.jboss.remoting.CannotConnectException:Can not connect http client invoker. -> java.net.UnknownHostException:althea-desktop
      2009-02-10 10:28:31,033 INFO [Thread-4] (org.rhq.enterprise.communications.ServiceContainer)- {ServiceContainer.shutting-down}Service container shutting down...
      2009-02-10 10:28:31,034 INFO [Thread-4] (org.rhq.enterprise.communications.ServiceContainer)- {ServiceContainer.shutdown}Service container shut down - no longer accepting incoming commands
      2009-02-10 10:28:31,034 INFO [Thread-4] (org.rhq.enterprise.agent.AgentMain)- {AgentMain.shut-down}Agent has been shut down
      


      The embedded-agent.log on my server :

      10:26:31,343 ERROR [RHQ Agent Registration Thread] (enterprise.communications.command.client.ClientCommandSenderTask)- {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[registerAgent], targetInterfaceName=org.rhq.core.clientapi.server.core.CoreServerService}]]. Cause: org.jboss.remoting.CannotConnectException:Can not connect http client invoker. -> java.net.ConnectException:Connection refused. Cause: org.jboss.remoting.CannotConnectException: Can not connect http client invoker.
      10:26:31,344 WARN [RHQ Agent Registration Thread] (org.rhq.enterprise.agent.AgentMain)- {AgentMain.agent-registration-failure}Agent failed to register with the server. retry=[true], retry interval=[60,000]. Cause: org.jboss.remoting.CannotConnectException:Can not connect http client invoker. -> java.net.ConnectException:Connection refused. Cause: org.jboss.remoting.CannotConnectException: Can not connect http client invoker.
      10:27:31,353 ERROR [RHQ Agent Registration Thread] (enterprise.communications.command.client.ClientCommandSenderTask)- {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[registerAgent], targetInterfaceName=org.rhq.core.clientapi.server.core.CoreServerService}]]. Cause: org.jboss.remoting.CannotConnectException:Can not connect http client invoker. -> java.net.ConnectException:Connection refused. Cause: org.jboss.remoting.CannotConnectException: Can not connect http client invoker.
      10:27:31,354 WARN [RHQ Agent Registration Thread] (org.rhq.enterprise.agent.AgentMain)- {AgentMain.agent-registration-failure}Agent failed to register with the server. retry=[true], retry interval=[60,000]. Cause: org.jboss.remoting.CannotConnectException:Can not connect http client invoker. -> java.net.ConnectException:Connection refused. Cause: org.jboss.remoting.CannotConnectException: Can not connect http client invoker.
      10:28:31,364 ERROR [RHQ Agent Registration Thread] (enterprise.communications.command.client.ClientCommandSenderTask)- {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[registerAgent], targetInterfaceName=org.rhq.core.clientapi.server.core.CoreServerService}]]. Cause: org.jboss.remoting.CannotConnectException:Can not connect http client invoker. -> java.net.ConnectException:Connection refused. Cause: org.jboss.remoting.CannotConnectException: Can not connect http client invoker.
      10:28:31,365 WARN [RHQ Agent Registration Thread] (org.rhq.enterprise.agent.AgentMain)- {AgentMain.agent-registration-failure}Agent failed to register with the server. retry=[true], retry interval=[60,000]. Cause: org.jboss.remoting.CannotConnectException:Can not connect http client invoker. -> java.net.ConnectException:Connection refused. Cause: org.jboss.remoting.CannotConnectException: Can not connect http client invoker.
      10:29:31,373 ERROR [RHQ Agent Registration Thread] (enterprise.communications.command.client.ClientCommandSenderTask)- {ClientCommandSenderTask.send-failed}Failed to send command [Command: type=[remotepojo]; cmd-in-response=[false]; config=[{rhq.send-throttle=true}]; params=[{invocation=NameBasedInvocation[registerAgent], targetInterfaceName=org.rhq.core.clientapi.server.core.CoreServerService}]]. Cause: org.jboss.remoting.CannotConnectException:Can not connect http client invoker. -> java.net.ConnectException:Connection refused. Cause: org.jboss.remoting.CannotConnectException: Can not connect http client invoker.
      10:29:31,374 WARN [RHQ Agent Registration Thread] (org.rhq.enterprise.agent.AgentMain)- {AgentMain.agent-registration-failure}Agent failed to register with the server. retry=[true], retry interval=[60,000]. Cause: org.jboss.remoting.CannotConnectException:Can not connect http client invoker. -> java.net.ConnectException:Connection refused. Cause: org.jboss.remoting.CannotConnectException: Can not connect http client invoker.
      




      The agent doesn't appear on my dashboard, someone may help me? Cause i'm not enought familiar with Jboss and Jopr, and... I'm lost.

      Thank you.

        • 1. Re: Agent callback failled
          mazz

          This is something that happens to me *all* the time.

          What happens here (at least when it happens to me) is when I start my server, it defaults to a public endpoint that is its hostname. The server does a lookup for its own name and uses that. It looks like your server found this: althea-desktop.

          Even though you may have configured your agent to talk to the server over a different hostname (perhaps a FQDN) or a different IP, after registration, the agent will ask the server for its "failover list" (this is true whether or not you have multiple servers in the cloud - what we call "HA"). The server will hand down this failover list that will contain the public endpoints of all the servers in the cloud. Which means the agent will start to talk to althea-desktop.

          Take a look at your agent's failover-list.dat in the /data directory and you'll see this.

          Now the agent is getting:

          Caused by: java.net.UnknownHostException: althea-desktop

          which probably just means that's a non-routable hostname from the point of view of the agent. That is not a valid hostname that your agent's DNS can find, hence the errors.

          To configure your server's public endpoint, go to the GUI - Administration>ListServers (or ManageServers - the name of the link changed recently). From there, click your server and edit its public endpoint to an address or IP that the agent can connect to.

          Now restart your agent. It should work.

          • 2. Re: Agent callback failled
            mazz

            Some other important details from #jopr chat room that I thought was appropriate to post here to capture the information for others to see:

            (8:20:00 AM) mazz: the point to remember is, the server's endpoint must be resolvable by all agents in the network
            (8:20:41 AM) KRz-Yamaneko: hum... i don't use the name "althea-desktop" to the agent i feed him the IP address :$
            (8:21:10 AM) mazz: right - read my post and I'll explain why the change occurs (it has to do with the failover list the agent gets from the server)
            (8:21:25 AM) KRz-Yamaneko: k
            (8:21:38 AM) mazz: we call the IP you feed the agent at the setup prompts the "registration server" because its only used when the agent registers
            (8:21:56 AM) mazz: after that, the server tells the agent what hostnames/IPs to start talking to (via the failover list)
            (8:23:03 AM) mazz: this allows our customers with large numbers of agents to have a single agent-configuration.xml that all their agents can be preconfigured with - but after they all get onto the Jopr network, they are evenly split across multiple servers in the cloud (to spread the load)

            • 3. Re: Agent callback failled
              krz-yamaneko

              Thank you again dude, it's work now.

              • 4. Re: Agent callback failled
                spolti

                Hi guys, today i had a similar issue with the jon client. -.-

                follows they log:

                 

                2011-03-29 11:00:42,393 WARN  [RHQ Server Polling Thread] (org.rhq.enterprise.agent.FailoverFailureCallback)- {AgentMain.too-many-failover-attempts}Too many failover attempts have been made [1]. Exception that triggered the failover: [org.jboss.remoting.CannotConnectException: Can not connect http client invoker. jon.]

                2011-03-29 11:00:42,393 ERROR [RHQ Server Polling Thread] (enterprise.communications.command.client.JBossRemotingRemoteCommunicator)- {JBossRemotingRemoteCommunicator.init-callback-failed}The initialize callback has failed. It will be tried again. Cause: org.jboss.remoting.CannotConnectException:Can not connect http client invoker. jon. -> java.net.UnknownHostException:jon

                org.jboss.remoting.CannotConnectException: Can not connect http client invoker. jon.

                        at org.jboss.remoting.transport.http.HTTPClientInvoker.useHttpURLConnection(HTTPClientInvoker.java:348)

                        at org.jboss.remoting.transport.http.HTTPClientInvoker.transport(HTTPClientInvoker.java:137)

                        at org.jboss.remoting.MicroRemoteClientInvoker.invoke(MicroRemoteClientInvoker.java:122)

                        at org.jboss.remoting.Client.invoke(Client.java:1634)

                        at org.jboss.remoting.Client.invoke(Client.java:548)

                        at org.rhq.enterprise.communications.command.client.JBossRemotingRemoteCommunicator.rawSend(JBossRemotingRemoteCommunicator.java:514)

                        at org.rhq.enterprise.communications.command.client.JBossRemotingRemoteCommunicator.sendWithoutCallbacks(JBossRemotingRemoteCommunicator.java:456)

                        at org.rhq.enterprise.communications.command.client.JBossRemotingRemoteCommunicator.sendWithoutInitializeCallback(JBossRemotingRemoteCommunicator.java:475)

                        at org.rhq.enterprise.agent.AgentMain.sendConnectRequestToServer(AgentMain.java:1947)

                        at org.rhq.enterprise.agent.ConnectAgentInitializeCallback.sendingInitialCommand(ConnectAgentInitializeCallback.java:43)

                        at org.rhq.enterprise.communications.command.client.JBossRemotingRemoteCommunicator.invokeInitializeCallbackIfNeeded(JBossRemotingRemoteCommunicator.java:578)

                        at org.rhq.enterprise.communications.command.client.JBossRemotingRemoteCommunicator.send(JBossRemotingRemoteCommunicator.java:491)

                        at org.rhq.enterprise.communications.command.client.AbstractCommandClient.invoke(AbstractCommandClient.java:143)

                        at org.rhq.enterprise.communications.command.client.ClientCommandSender.send(ClientCommandSender.java:1087)

                        at org.rhq.enterprise.communications.command.client.ServerPollingThread.run(ServerPollingThread.java:115)

                Caused by: java.net.UnknownHostException: jon

                        at java.net.PlainSocketImpl.connect(PlainSocketImpl.java:177)

                        at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:366)

                        at java.net.Socket.connect(Socket.java:529)

                        at sun.net.NetworkClient.doConnect(NetworkClient.java:158)

                        at sun.net.www.http.HttpClient.openServer(HttpClient.java:394)

                        at sun.net.www.http.HttpClient.openServer(HttpClient.java:529)

                        at sun.net.www.http.HttpClient.<init>(HttpClient.java:233)

                        at sun.net.www.http.HttpClient.New(HttpClient.java:306)

                        at sun.net.www.http.HttpClient.New(HttpClient.java:323)

                        at sun.net.www.protocol.http.HttpURLConnection.getNewHttpClient(HttpURLConnection.java:975)

                        at sun.net.www.protocol.http.HttpURLConnection.plainConnect(HttpURLConnection.java:916)

                        at sun.net.www.protocol.http.HttpURLConnection.connect(HttpURLConnection.java:841)

                        at sun.net.www.protocol.http.HttpURLConnection.getOutputStream(HttpURLConnection.java:1019)

                        at org.jboss.remoting.transport.http.HTTPClientInvoker.useHttpURLConnection(HTTPClientInvoker.java:277)

                        ... 14 more

                 

                 

                 

                Thank you.

                • 5. Re: Agent callback failled
                  spolti

                  Hi again guysss..

                   

                  I solved this problem by just adding the server ip and hostname on /etc/hosts file.

                   

                  omg..

                   

                  Thanks.