0 Replies Latest reply on Jun 20, 2007 9:57 AM by Silviu Necula

    cluster messaging error

    Silviu Necula Newbie

      I have 2 JB servers set up as described in the guide for a development cluster.
      I have a scheduler running every minute that I deploy using farming and have set up using the example-cluster-service.xml in the farm folder.
      When I stop one server I get the error below but he scheduler still runs. My impression is that the timer the scheduler is using is the cause of this.
      Any suggestions ?

      ERROR org.jboss.remoting.transport.socket.SocketClientInvoker handleException - Got marshalling exception, exiting
      java.net.SocketException: Connection reset
      at java.net.SocketInputStream.read(SocketInputStream.java:168)
      at java.io.BufferedInputStream.fill(BufferedInputStream.java:218)
      at java.io.BufferedInputStream.read(BufferedInputStream.java:235)
      at java.io.FilterInputStream.read(FilterInputStream.java:66)
      at org.jboss.remoting.transport.socket.MicroSocketClientInvoker.readVersion(MicroSocketClientInvoker.java:986)
      at org.jboss.remoting.transport.socket.MicroSocketClientInvoker.transport(MicroSocketClientInvoker.java:572)
      at org.jboss.remoting.transport.bisocket.BisocketClientInvoker.transport(BisocketClientInvoker.java:339)
      at org.jboss.remoting.MicroRemoteClientInvoker.invoke(MicroRemoteClientInvoker.java:122)
      at org.jboss.remoting.ConnectionValidator.doCheckConnection(ConnectionValidator.java:126)
      at org.jboss.remoting.ConnectionValidator.run(ConnectionValidator.java:195)
      at java.util.TimerThread.mainLoop(Timer.java:512)
      at java.util.TimerThread.run(Timer.java:462)
      2007-06-20 09:45:24,234 DEBUG org.jboss.remoting.ConnectionValidator doCheckConnection - ConnectionValidator failed to ping via SocketClientInvoker[14d3343, bisocket://127.0.0.1:4458]
      java.rmi.MarshalException: Failed to communicate. Problem during marshalling/unmarshalling; nested exception is:
      java.net.SocketException: Connection reset
      at org.jboss.remoting.transport.socket.SocketClientInvoker.handleException(SocketClientInvoker.java:122)
      at org.jboss.remoting.transport.socket.MicroSocketClientInvoker.transport(MicroSocketClientInvoker.java:654)
      at org.jboss.remoting.transport.bisocket.BisocketClientInvoker.transport(BisocketClientInvoker.java:339)
      at org.jboss.remoting.MicroRemoteClientInvoker.invoke(MicroRemoteClientInvoker.java:122)
      at org.jboss.remoting.ConnectionValidator.doCheckConnection(ConnectionValidator.java:126)
      at org.jboss.remoting.ConnectionValidator.run(ConnectionValidator.java:195)
      at java.util.TimerThread.mainLoop(Timer.java:512)
      at java.util.TimerThread.run(Timer.java:462)
      Caused by: java.net.SocketException: Connection reset
      at java.net.SocketInputStream.read(SocketInputStream.java:168)
      at java.io.BufferedInputStream.fill(BufferedInputStream.java:218)
      at java.io.BufferedInputStream.read(BufferedInputStream.java:235)
      at java.io.FilterInputStream.read(FilterInputStream.java:66)
      at org.jboss.remoting.transport.socket.MicroSocketClientInvoker.readVersion(MicroSocketClientInvoker.java:986)
      at org.jboss.remoting.transport.socket.MicroSocketClientInvoker.transport(MicroSocketClientInvoker.java:572)
      ... 6 more
      2007-06-20 09:45:24,234 DEBUG org.jboss.remoting.ConnectionValidator run - ConnectionValidator[SocketClientInvoker[14d3343, bisocket://127.0.0.1:4458], pingPeriod=2000 ms]'s connections is invalid
      2007-06-20 09:45:24,234 DEBUG org.jboss.remoting.InvokerRegistry decrementClientInvokerCounter - decremented SocketClientInvoker[14d3343, bisocket://127.0.0.1:4458]'s count, current count 21
      2007-06-20 09:45:24,234 DEBUG org.jboss.remoting.ConnectionValidator doStop - ConnectionValidator[SocketClientInvoker[14d3343, bisocket://127.0.0.1:4458], pingPeriod=2000 ms] stopped, returning true
      2007-06-20 09:45:24,234 DEBUG org.jboss.jms.client.remoting.ConsolidatedRemotingConnectionListener handleConnectionException - ConnectionState[414].ConsolidatedListener forwarding remoting failure "java.lang.Exception: Could not connect to server!" to ConnectionFailureListener[FailoverCommandCenter[ConnectionState[414]]]
      2007-06-20 09:45:24,234 DEBUG org.jboss.jms.client.container.ConnectionFailureListener handleConnectionException - ConnectionFailureListener[FailoverCommandCenter[ConnectionState[414]]] is being notified of connection failure: java.lang.Exception: Could not connect to server!
      2007-06-20 09:45:24,234 DEBUG org.jboss.jms.client.FailoverCommandCenter failureDetected - failure detected by ConnectionFailureListener[FailoverCommandCenter[ConnectionState[414]]]
      2007-06-20 09:45:24,234 DEBUG org.jboss.remoting.ServerInvoker handleInternalInvocation - ServerInvoker (SocketServerInvoker[UNINITIALIZED]) removing client callback handler with session id of 2s2v15l-hesbpq-f35uvn63-1-f35ux8x3-6e+2s2v15l-hesbpq-f35uvn63-1-f35ux8xy-6g.
      2007-06-20 09:45:24,250 DEBUG org.jboss.remoting.InvokerRegistry decrementClientInvokerCounter - removed org.jboss.remoting.transport.local.LocalClientInvoker@4ce898 from registry
      2007-06-20 09:45:24,250 DEBUG org.jboss.remoting.ServerInvoker stop - SocketServerInvoker[UNINITIALIZED] stopped
      2007-06-20 09:45:24,250 DEBUG org.jboss.remoting.LeasePinger setDisconnectTimeout - LeasePinger[SocketClientInvoker[1613fe7, bisocket://127.0.0.1:4458](2s2v15l-hesbpq-f35uvn63-1-f35uwwun-8)] setting disconnect timeout to: 0
      2007-06-20 09:45:24,250 DEBUG org.jboss.remoting.InvokerRegistry decrementClientInvokerCounter - decremented SocketClientInvoker[1613fe7, bisocket://127.0.0.1:4458]'s count, current count 21
      2007-06-20 09:45:24,250 DEBUG org.jboss.jms.client.FailoverCommandCenter failureDetected - FailoverCommandCenter[ConnectionState[414]] starting client-side failover
      2007-06-20 09:45:24,250 DEBUG org.jboss.jms.client.remoting.JMSRemotingConnection - JMSRemotingConnection[bisocket://127.0.0.1:4459/?clientMaxPoolSize=50&clientSocketClass=org.jboss.jms.client.remoting.ClientSocketWrapper&dataType=jms&marshaller=org.jboss.jms.wireformat.JMSWireFormat&numberOfCallRetries=1&numberOfRetries=1&socket.check_connection=false&timeout=0&unmarshaller=org.jboss.jms.wireformat.JMSWireFormat] created
      2007-06-20 09:45:24,250 DEBUG org.jboss.jms.client.remoting.JMSRemotingConnection addInvokerCallbackHandler - JMSRemotingConnection[bisocket://127.0.0.1:4459/?clientMaxPoolSize=50&clientSocketClass=org.jboss.jms.client.remoting.ClientSocketWrapper&dataType=jms&marshaller=org.jboss.jms.wireformat.JMSWireFormat&numberOfCallRetries=1&numberOfRetries=1&socket.check_connection=false&timeout=0&unmarshaller=org.jboss.jms.wireformat.JMSWireFormat] is doing push callbacks
      2007-06-20 09:45:24,250 DEBUG org.jboss.remoting.transport.socket.MicroSocketClientInvoker transport - SocketClientInvoker[1613fe7, bisocket://127.0.0.1:4458] got SocketException java.net.SocketException: Connection reset
      2007-06-20 09:45:24,250 DEBUG org.jboss.remoting.transport.socket.SocketWrapper close - ClientSocketWrapper[Socket[addr=127.0.0.1/127.0.0.1,port=4458,localport=2802].13a87a0] closing
      2007-06-20 09:45:24,250 ERROR org.jboss.remoting.transport.socket.SocketClientInvoker handleException - Got marshalling exception, exiting
      java.net.SocketException: Connection reset
      at java.net.SocketInputStream.read(SocketInputStream.java:168)
      at java.io.BufferedInputStream.fill(BufferedInputStream.java:218)
      at java.io.BufferedInputStream.read(BufferedInputStream.java:235)
      at java.io.FilterInputStream.read(FilterInputStream.java:66)
      at org.jboss.remoting.transport.socket.MicroSocketClientInvoker.readVersion(MicroSocketClientInvoker.java:986)
      at org.jboss.remoting.transport.socket.MicroSocketClientInvoker.transport(MicroSocketClientInvoker.java:572)
      at org.jboss.remoting.transport.bisocket.BisocketClientInvoker.transport(BisocketClientInvoker.java:339)
      at org.jboss.remoting.MicroRemoteClientInvoker.invoke(MicroRemoteClientInvoker.java:122)
      at org.jboss.remoting.LeasePinger.sendClientPing(LeasePinger.java:283)
      at org.jboss.remoting.LeasePinger.access$000(LeasePinger.java:20)
      at org.jboss.remoting.LeasePinger$LeaseTimerTask.run(LeasePinger.java:315)
      at java.util.TimerThread.mainLoop(Timer.java:512)
      at java.util.TimerThread.run(Timer.java:462)
      2007-06-20 09:45:24,250 DEBUG org.jboss.remoting.LeasePinger sendClientPing - LeasePinger[SocketClientInvoker[1613fe7, bisocket://127.0.0.1:4458](2s2v15l-hesbpq-f35uvn63-1-f35uwwun-8)] failed to ping to server
      java.rmi.MarshalException: Failed to communicate. Problem during marshalling/unmarshalling; nested exception is:
      java.net.SocketException: Connection reset
      at org.jboss.remoting.transport.socket.SocketClientInvoker.handleException(SocketClientInvoker.java:122)
      at org.jboss.remoting.transport.socket.MicroSocketClientInvoker.transport(MicroSocketClientInvoker.java:654)
      at org.jboss.remoting.transport.bisocket.BisocketClientInvoker.transport(BisocketClientInvoker.java:339)
      at org.jboss.remoting.MicroRemoteClientInvoker.invoke(MicroRemoteClientInvoker.java:122)
      at org.jboss.remoting.LeasePinger.sendClientPing(LeasePinger.java:283)
      at org.jboss.remoting.LeasePinger.access$000(LeasePinger.java:20)
      at org.jboss.remoting.LeasePinger$LeaseTimerTask.run(LeasePinger.java:315)
      at java.util.TimerThread.mainLoop(Timer.java:512)
      at java.util.TimerThread.run(Timer.java:462)
      Caused by: java.net.SocketException: Connection reset
      at java.net.SocketInputStream.read(SocketInputStream.java:168)
      at java.io.BufferedInputStream.fill(BufferedInputStream.java:218)
      at java.io.BufferedInputStream.read(BufferedInputStream.java:235)
      at java.io.FilterInputStream.read(FilterInputStream.java:66)
      at org.jboss.remoting.transport.socket.MicroSocketClientInvoker.readVersion(MicroSocketClientInvoker.java:986)
      at org.jboss.remoting.transport.socket.MicroSocketClientInvoker.transport(MicroSocketClientInvoker.java:572)
      ... 7 more
      2007-06-20 09:45:24,250 WARN org.jboss.remoting.LeasePinger sendClientPing - LeasePinger[SocketClientInvoker[1613fe7, bisocket://127.0.0.1:4458](2s2v15l-hesbpq-f35uvn63-1-f35uwwun-8)] failed to ping to server: Failed to communicate. Problem during marshalling/unmarshalling; nested exception is:
      java.net.SocketException: Connection reset
      2007-06-20 09:45:24,265 DEBUG org.jboss.remoting.Client addListener - starting callback Connector: InvokerLocator [bisocket://100.103.1.201:95781665/callback?serverSocketClass=org.jboss.jms.server.remoting.ServerSocketWrapper&isCallbackServer=true&callbackServerProtocol=bisocket&datatype=jms&guid=2s2v15l-hesbpq-f35uvn63-1-f35uy7tm-el&callbackServerHost=100.103.1.201&callbackServerPort=95781665&onewayThreadPool=org.jboss.jms.server.remoting.DirectThreadPool&clientSocketClass=org.jboss.jms.client.remoting.ClientSocketWrapper]
      2007-06-20 09:45:24,281 DEBUG org.jboss.remoting.ServerInvoker createServerSocketFactory - SocketServerInvoker[UNINITIALIZED] did not find server socket factory configuration as mbean service or classname. Creating default server socket factory.
      2007-06-20 09:45:24,281 DEBUG org.jboss.remoting.ServerInvoker createServerSocketFactory - SocketServerInvoker[UNINITIALIZED] created server socket factory javax.net.DefaultServerSocketFactory@6504bc
      2007-06-20 09:45:24,281 DEBUG org.jboss.remoting.transport.Connector start - org.jboss.remoting.transport.Connector@16bc363 started
      2007-06-20 09:45:24,281 DEBUG org.jboss.remoting.ServerInvoker handleInternalInvocation - ServerInvoker (SocketServerInvoker[UNINITIALIZED]) added client callback handler CallbackManager[19e3bdb] with session id of 2s2v15l-hesbpq-f35uvn63-1-f35uy7tm-ek+2s2v15l-hesbpq-f35uvn63-1-f35uy7uh-em and callback handle object of null.
      2007-06-20 09:45:24,281 DEBUG org.jboss.remoting.InvokerRegistry decrementClientInvokerCounter - removed org.jboss.remoting.transport.local.LocalClientInvoker@1e8d4ce from registry
      2007-06-20 09:45:24,281 DEBUG org.jboss.remoting.callback.DefaultCallbackErrorHandler setServerInvoker - DefaultCallbackErrorHandler[UNITIALIZED] setting server invoker to SocketServerInvoker[127.0.0.1:4459]
      2007-06-20 09:45:24,281 DEBUG org.jboss.remoting.callback.DefaultCallbackErrorHandler setCallbackHandler - DefaultCallbackErrorHandler[SocketServerInvoker[127.0.0.1:4459]] setting callback handler to ServerInvokerCallbackHandler[2s2v15l-hesbpq-f35uvn63-1-f35uy7tm-ek+2s2v15l-hesbpq-f35uvn63-1-f35uy7uh-em]
      2007-06-20 09:45:24,281 DEBUG org.jboss.remoting.callback.ServerInvokerCallbackHandler init - Session id for callback handler is 2s2v15l-hesbpq-f35uvn63-1-f35uy7tm-ek+2s2v15l-hesbpq-f35uvn63-1-f35uy7uh-em
      2007-06-20 09:45:24,281 DEBUG org.jboss.jms.server.remoting.JMSServerInvocationHandler addListener - adding callback handler ServerInvokerCallbackHandler[2s2v15l-hesbpq-f35uvn63-1-f35uy7tm-ek+2s2v15l-hesbpq-f35uvn63-1-f35uy7uh-em]
      2007-06-20 09:45:24,281 DEBUG org.jboss.jms.client.remoting.JMSRemotingConnection start - JMSRemotingConnection[bisocket://127.0.0.1:4459/?clientMaxPoolSize=50&clientSocketClass=org.jboss.jms.client.remoting.ClientSocketWrapper&dataType=jms&marshaller=org.jboss.jms.wireformat.JMSWireFormat&numberOfCallRetries=1&numberOfRetries=1&socket.check_connection=false&timeout=0&unmarshaller=org.jboss.jms.wireformat.JMSWireFormat] started
      2007-06-20 09:45:24,281 DEBUG org.jboss.jms.server.remoting.JMSServerInvocationHandler invoke - found calllback handler for remoting session ...-f35uy7tm-ek
      2007-06-20 09:45:24,281 DEBUG org.jboss.jms.server.endpoint.ServerConnectionFactoryEndpoint createConnectionDelegate - ConnectionFactoryEndpoint[1] received client-side failover request. Creating failover connection to replace connection to failed node 0
      2007-06-20 09:45:24,281 DEBUG org.jboss.jms.server.ServerPeer waitForFailover - ServerPeer[1] waiting for server-side failover for failed node 0 to complete
      2007-06-20 09:45:24,281 DEBUG org.jboss.jms.server.ServerPeer waitForFailover - ServerPeer[1] blocking on the failover lock, waiting for failover to start
      2007-06-20 09:45:24,750 WARN org.jboss.remoting.transport.bisocket.BisocketServerInvoker run - org.jboss.remoting.transport.bisocket.BisocketServerInvoker$ControlMonitorTimerTask@eb5cdc: detected failure on control connection Thread[control: Socket[addr=127.0.0.1/127.0.0.1,port=3583,localport=2797],5,]: requesting new control connection
      2007-06-20 09:45:24,750 WARN org.jboss.remoting.transport.bisocket.BisocketServerInvoker run - org.jboss.remoting.transport.bisocket.BisocketServerInvoker$ControlMonitorTimerTask@1d7417: detected failure on control connection Thread[control: Socket[addr=127.0.0.1/127.0.0.1,port=3583,localport=2783],5,]: requesting new control connection
      2007-06-20 09:45:24,843 DEBUG org.jboss.remoting.transport.bisocket.BisocketClientInvoker getSecondaryLocator - getting secondary locator
      2007-06-20 09:45:24,843 DEBUG org.jboss.remoting.transport.bisocket.BisocketClientInvoker getSecondaryLocator - getting secondary locator
      2007-06-20 09:45:25,187 DEBUG org.jboss.remoting.transport.socket.MicroSocketClientInvoker getConnection - SocketClientInvoker[14d3343, bisocket://127.0.0.1:4458] got Exception java.net.ConnectException: Connection refused: connect, creation attempt took 953 ms
      2007-06-20 09:45:25,187 DEBUG org.jboss.remoting.ConnectionValidator doCheckConnection - ConnectionValidator failed to ping via SocketClientInvoker[14d3343, bisocket://127.0.0.1:4458]
      org.jboss.remoting.CannotConnectException: Can not get connection to server. Problem establishing socket connection for InvokerLocator [bisocket://127.0.0.1:4458/?clientMaxPoolSize=50&clientSocketClass=org.jboss.jms.client.remoting.ClientSocketWrapper&dataType=jms&marshaller=org.jboss.jms.wireformat.JMSWireFormat&numberOfCallRetries=1&numberOfRetries=1&socket.check_connection=false&timeout=0&unmarshaller=org.jboss.jms.wireformat.JMSWireFormat]
      at org.jboss.remoting.transport.socket.MicroSocketClientInvoker.transport(MicroSocketClientInvoker.java:530)
      at org.jboss.remoting.transport.bisocket.BisocketClientInvoker.transport(BisocketClientInvoker.java:339)
      at org.jboss.remoting.MicroRemoteClientInvoker.invoke(MicroRemoteClientInvoker.java:122)
      at org.jboss.remoting.ConnectionValidator.doCheckConnection(ConnectionValidator.java:126)
      at org.jboss.remoting.ConnectionValidator.run(ConnectionValidator.java:195)
      at java.util.TimerThread.mainLoop(Timer.java:512)
      at java.util.TimerThread.run(Timer.java:462)
      Caused by: java.net.ConnectException: Connection refused: connect
      at java.net.PlainSocketImpl.socketConnect(Native Method)
      at java.net.PlainSocketImpl.doConnect(PlainSocketImpl.java:333)
      at java.net.PlainSocketImpl.connectToAddress(PlainSocketImpl.java:195)
      at java.net.PlainSocketImpl.connect(PlainSocketImpl.java:182)
      at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:364)
      at java.net.Socket.connect(Socket.java:507)
      at org.jboss.remoting.transport.socket.SocketClientInvoker.createSocket(SocketClientInvoker.java:187)
      at org.jboss.remoting.transport.bisocket.BisocketClientInvoker.createSocket(BisocketClientInvoker.java:346)
      at org.jboss.remoting.transport.socket.MicroSocketClientInvoker.getConnection(MicroSocketClientInvoker.java:801)
      at org.jboss.remoting.transport.socket.MicroSocketClientInvoker.transport(MicroSocketClientInvoker.java:526)
      ... 6 more