4 Replies Latest reply on Oct 1, 2007 10:45 AM by ivanrc

    Correct Java Client  Behaviour when JBoss Messaging cluster

    ivanrc

      Hi,

      I want to know if this is the correct behaviour of Java client when node of Jboss messaging Cluster crashes.

      In both, Topics o Queues, when my java client is in source line ...

      TextMessage message = (TextMessage)consumer.receive(1000);

      when consumer is MessageConsumer and then the JBoss Messaging cluster node in which is connected crashes.

      I loss the connection, and by console I obtain several exceptions...

      I think that Jboss Messaging implements transparent failover, but in this case that doesn´t happen.

      thanks.


      attachment of console lines when node crashes happen:

      ---------------------

      MessageID:ID:JBM-88515
      08:40:06,992 ERROR @Timer-0 [SocketClientInvoker] Got marshalling exception, exiting
      java.net.SocketException: Connection reset
      at java.net.SocketInputStream.read(SocketInputStream.java:168)
      at java.io.BufferedInputStream.fill(BufferedInputStream.java:218)
      at java.io.BufferedInputStream.read(BufferedInputStream.java:235)
      at java.io.FilterInputStream.read(FilterInputStream.java:66)
      at org.jboss.remoting.transport.socket.MicroSocketClientInvoker.readVersion(MicroSocketClientInvoker.java:986)
      at org.jboss.remoting.transport.socket.MicroSocketClientInvoker.transport(MicroSocketClientInvoker.java:572)
      at org.jboss.remoting.transport.bisocket.BisocketClientInvoker.transport(BisocketClientInvoker.java:353)
      at org.jboss.remoting.MicroRemoteClientInvoker.invoke(MicroRemoteClientInvoker.java:122)
      at org.jboss.remoting.LeasePinger.sendClientPing(LeasePinger.java:283)
      at org.jboss.remoting.LeasePinger.access$000(LeasePinger.java:20)
      at org.jboss.remoting.LeasePinger$LeaseTimerTask.run(LeasePinger.java:315)
      at java.util.TimerThread.mainLoop(Timer.java:512)
      at java.util.TimerThread.run(Timer.java:462)
      08:40:06,992 WARN @Timer-0 [LeasePinger] LeasePinger[SocketClientInvoker[a9c09e, bisocket://127.0.0.1:4557](4sv3c5y-y83mkd-f78lwtvn-1-f78lwuba-3)] failed to ping to server: Failed to communicate. Problem during marshalling/unmarshalling; nested exception is:
      java.net.SocketException: Connection reset
      08:40:07,039 ERROR @Timer-2 [SocketClientInvoker] Got marshalling exception, exiting
      java.net.SocketException: Connection reset
      at java.net.SocketInputStream.read(SocketInputStream.java:168)
      at java.io.BufferedInputStream.fill(BufferedInputStream.java:218)
      at java.io.BufferedInputStream.read(BufferedInputStream.java:235)
      at java.io.FilterInputStream.read(FilterInputStream.java:66)
      at org.jboss.remoting.transport.socket.MicroSocketClientInvoker.readVersion(MicroSocketClientInvoker.java:986)
      at org.jboss.remoting.transport.socket.MicroSocketClientInvoker.transport(MicroSocketClientInvoker.java:572)
      at org.jboss.remoting.transport.bisocket.BisocketClientInvoker.transport(BisocketClientInvoker.java:353)
      at org.jboss.remoting.MicroRemoteClientInvoker.invoke(MicroRemoteClientInvoker.java:122)
      at org.jboss.remoting.ConnectionValidator.doCheckConnection(ConnectionValidator.java:126)
      at org.jboss.remoting.ConnectionValidator.run(ConnectionValidator.java:195)
      at java.util.TimerThread.mainLoop(Timer.java:512)
      at java.util.TimerThread.run(Timer.java:462)
      08:40:08,132 WARN @Thread-42 [ClientClusteredConnectionFactoryDelegate] org.jboss.remoting.CannotConnectException: Can not get connection to server. Problem establishing socket connection for InvokerLocator [bisocket://127.0.0.1:4557/?clientMaxPoolSize=50&clientSocketClass=org.jboss.jms.client.remoting.ClientSocketWrapper&dataType=jms&marshaller=org.jboss.jms.wireformat.JMSWireFormat&numberOfCallRetries=1&numberOfRetries=1&socket.check_connection=false&timeout=0&unmarshaller=org.jboss.jms.wireformat.JMSWireFormat]
      org.jboss.remoting.CannotConnectException: Can not get connection to server. Problem establishing socket connection for InvokerLocator [bisocket://127.0.0.1:4557/?clientMaxPoolSize=50&clientSocketClass=org.jboss.jms.client.remoting.ClientSocketWrapper&dataType=jms&marshaller=org.jboss.jms.wireformat.JMSWireFormat&numberOfCallRetries=1&numberOfRetries=1&socket.check_connection=false&timeout=0&unmarshaller=org.jboss.jms.wireformat.JMSWireFormat]
      at org.jboss.remoting.transport.socket.MicroSocketClientInvoker.transport(MicroSocketClientInvoker.java:530)
      at org.jboss.remoting.transport.bisocket.BisocketClientInvoker.transport(BisocketClientInvoker.java:353)
      at org.jboss.remoting.MicroRemoteClientInvoker.invoke(MicroRemoteClientInvoker.java:122)
      at org.jboss.remoting.Client.invoke(Client.java:1550)
      at org.jboss.remoting.Client.invoke(Client.java:530)
      at org.jboss.jms.client.delegate.ClientClusteredConnectionFactoryDelegate.removeCallback(ClientClusteredConnectionFactoryDelegate.java:144)
      at org.jboss.jms.client.delegate.ClientClusteredConnectionFactoryDelegate.closeCallback(ClientClusteredConnectionFactoryDelegate.java:160)
      at org.jboss.jms.client.delegate.ClientClusteredConnectionFactoryDelegate.establishCallback(ClientClusteredConnectionFactoryDelegate.java:91)
      at org.jboss.jms.client.remoting.ConnectionFactoryCallbackHandler$CallbackConnectionListener.handleConnectionException(ConnectionFactoryCallbackHandler.java:105)
      at org.jboss.remoting.ConnectionValidator$1.run(ConnectionValidator.java:346)
      Caused by: java.net.ConnectException: Connection refused: connect
      at java.net.PlainSocketImpl.socketConnect(Native Method)
      at java.net.PlainSocketImpl.doConnect(PlainSocketImpl.java:333)
      at java.net.PlainSocketImpl.connectToAddress(PlainSocketImpl.java:195)
      at java.net.PlainSocketImpl.connect(PlainSocketImpl.java:182)
      at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:366)
      at java.net.Socket.connect(Socket.java:516)
      at org.jboss.remoting.transport.socket.SocketClientInvoker.createSocket(SocketClientInvoker.java:187)
      at org.jboss.remoting.transport.bisocket.BisocketClientInvoker.createSocket(BisocketClientInvoker.java:360)
      at org.jboss.remoting.transport.socket.MicroSocketClientInvoker.getConnection(MicroSocketClientInvoker.java:801)
      at org.jboss.remoting.transport.socket.MicroSocketClientInvoker.transport(MicroSocketClientInvoker.java:526)
      ... 9 more
      08:40:09,164 WARN @Thread-42 [Client] unable to remove remote callback handler: Can not get connection to server. Problem establishing socket connection for InvokerLocator [bisocket://127.0.0.1:4557/?clientMaxPoolSize=50&clientSocketClass=org.jboss.jms.client.remoting.ClientSocketWrapper&dataType=jms&marshaller=org.jboss.jms.wireformat.JMSWireFormat&numberOfCallRetries=1&numberOfRetries=1&socket.check_connection=false&timeout=0&unmarshaller=org.jboss.jms.wireformat.JMSWireFormat]
      08:40:10,148 WARN @Thread-42 [LeasePinger] LeasePinger[SocketClientInvoker[a9c09e, bisocket://127.0.0.1:4557](4sv3c5y-y83mkd-f78lwtvn-1-f78lwuba-3)] failed sending disconnect for client lease for client with session ID 4sv3c5y-y83mkd-f78lwtvn-1-f78lwtxe-2
      08:40:11,164 ERROR @Thread-42 [MicroRemoteClientInvoker] error shutting down lease pinger
      08:40:11,445 ERROR @Thread-43 [JMSWireFormat] Failed to write packet
      java.lang.NullPointerException
      at java.io.DataOutputStream.writeUTF(DataOutputStream.java:330)
      at java.io.DataOutputStream.writeUTF(DataOutputStream.java:306)
      at org.jboss.jms.delegate.DeliveryRecovery.write(DeliveryRecovery.java:79)
      at org.jboss.jms.wireformat.SessionRecoverDeliveriesRequest.write(SessionRecoverDeliveriesRequest.java:113)
      at org.jboss.jms.wireformat.JMSWireFormat.write(JMSWireFormat.java:237)
      at org.jboss.remoting.transport.socket.MicroSocketClientInvoker.versionedWrite(MicroSocketClientInvoker.java:969)
      at org.jboss.remoting.transport.socket.MicroSocketClientInvoker.transport(MicroSocketClientInvoker.java:557)
      at org.jboss.remoting.transport.bisocket.BisocketClientInvoker.transport(BisocketClientInvoker.java:353)
      at org.jboss.remoting.MicroRemoteClientInvoker.invoke(MicroRemoteClientInvoker.java:122)
      at org.jboss.remoting.Client.invoke(Client.java:1550)
      at org.jboss.remoting.Client.invoke(Client.java:530)
      at org.jboss.remoting.Client.invoke(Client.java:518)
      at org.jboss.jms.client.delegate.DelegateSupport.doInvoke(DelegateSupport.java:186)
      at org.jboss.jms.client.delegate.DelegateSupport.doInvoke(DelegateSupport.java:157)
      at org.jboss.jms.client.delegate.ClientSessionDelegate.org$jboss$jms$client$delegate$ClientSessionDelegate$recoverDeliveries$aop(ClientSessionDelegate.java:468)
      at org.jboss.jms.client.delegate.ClientSessionDelegate$recoverDeliveries_4196799148363996265.invokeNext(ClientSessionDelegate$recoverDeliveries_4196799148363996265.java)
      at org.jboss.jms.client.container.FailoverValveInterceptor.invoke(FailoverValveInterceptor.java:107)
      at org.jboss.aop.advice.PerInstanceInterceptor.invoke(PerInstanceInterceptor.java:105)
      at org.jboss.jms.client.delegate.ClientSessionDelegate$recoverDeliveries_4196799148363996265.invokeNext(ClientSessionDelegate$recoverDeliveries_4196799148363996265.java)
      at org.jboss.jms.client.container.ClosedInterceptor.invoke(ClosedInterceptor.java:170)
      at org.jboss.aop.advice.PerInstanceInterceptor.invoke(PerInstanceInterceptor.java:105)
      at org.jboss.jms.client.delegate.ClientSessionDelegate$recoverDeliveries_4196799148363996265.invokeNext(ClientSessionDelegate$recoverDeliveries_4196799148363996265.java)
      at org.jboss.jms.client.delegate.ClientSessionDelegate.recoverDeliveries(ClientSessionDelegate.java)
      at org.jboss.jms.client.state.SessionState.synchronizeWith(SessionState.java:344)
      at org.jboss.jms.client.delegate.ClientSessionDelegate.synchronizeWith(ClientSessionDelegate.java:122)
      at org.jboss.jms.client.state.ConnectionState.synchronizeWith(ConnectionState.java:174)
      at org.jboss.jms.client.delegate.ClientConnectionDelegate.synchronizeWith(ClientConnectionDelegate.java:125)
      at org.jboss.jms.client.FailoverCommandCenter.failureDetected(FailoverCommandCenter.java:141)
      at org.jboss.jms.client.container.ConnectionFailureListener.handleConnectionException(ConnectionFailureListener.java:62)
      at org.jboss.jms.client.remoting.ConsolidatedRemotingConnectionListener.handleConnectionException(ConsolidatedRemotingConnectionListener.java:81)
      at org.jboss.remoting.ConnectionValidator$1.run(ConnectionValidator.java:346)
      08:40:11,460 ERROR @Thread-43 [SocketClientInvoker] Got marshalling exception, exiting
      java.io.IOException
      at java.io.DataOutputStream.writeUTF(DataOutputStream.java:330)
      at java.io.DataOutputStream.writeUTF(DataOutputStream.java:306)
      at org.jboss.jms.delegate.DeliveryRecovery.write(DeliveryRecovery.java:79)
      at org.jboss.jms.wireformat.SessionRecoverDeliveriesRequest.write(SessionRecoverDeliveriesRequest.java:113)
      at org.jboss.jms.wireformat.JMSWireFormat.write(JMSWireFormat.java:237)
      at org.jboss.remoting.transport.socket.MicroSocketClientInvoker.versionedWrite(MicroSocketClientInvoker.java:969)
      at org.jboss.remoting.transport.socket.MicroSocketClientInvoker.transport(MicroSocketClientInvoker.java:557)
      at org.jboss.remoting.transport.bisocket.BisocketClientInvoker.transport(BisocketClientInvoker.java:353)
      at org.jboss.remoting.MicroRemoteClientInvoker.invoke(MicroRemoteClientInvoker.java:122)
      at org.jboss.remoting.Client.invoke(Client.java:1550)
      at org.jboss.remoting.Client.invoke(Client.java:530)
      at org.jboss.remoting.Client.invoke(Client.java:518)
      at org.jboss.jms.client.delegate.DelegateSupport.doInvoke(DelegateSupport.java:186)
      at org.jboss.jms.client.delegate.DelegateSupport.doInvoke(DelegateSupport.java:157)
      at org.jboss.jms.client.delegate.ClientSessionDelegate.org$jboss$jms$client$delegate$ClientSessionDelegate$recoverDeliveries$aop(ClientSessionDelegate.java:468)
      at org.jboss.jms.client.delegate.ClientSessionDelegate$recoverDeliveries_4196799148363996265.invokeNext(ClientSessionDelegate$recoverDeliveries_4196799148363996265.java)
      at org.jboss.jms.client.container.FailoverValveInterceptor.invoke(FailoverValveInterceptor.java:107)
      at org.jboss.aop.advice.PerInstanceInterceptor.invoke(PerInstanceInterceptor.java:105)
      at org.jboss.jms.client.delegate.ClientSessionDelegate$recoverDeliveries_4196799148363996265.invokeNext(ClientSessionDelegate$recoverDeliveries_4196799148363996265.java)
      at org.jboss.jms.client.container.ClosedInterceptor.invoke(ClosedInterceptor.java:170)
      at org.jboss.aop.advice.PerInstanceInterceptor.invoke(PerInstanceInterceptor.java:105)
      at org.jboss.jms.client.delegate.ClientSessionDelegate$recoverDeliveries_4196799148363996265.invokeNext(ClientSessionDelegate$recoverDeliveries_4196799148363996265.java)
      at org.jboss.jms.client.delegate.ClientSessionDelegate.recoverDeliveries(ClientSessionDelegate.java)
      at org.jboss.jms.client.state.SessionState.synchronizeWith(SessionState.java:344)
      at org.jboss.jms.client.delegate.ClientSessionDelegate.synchronizeWith(ClientSessionDelegate.java:122)
      at org.jboss.jms.client.state.ConnectionState.synchronizeWith(ConnectionState.java:174)
      at org.jboss.jms.client.delegate.ClientConnectionDelegate.synchronizeWith(ClientConnectionDelegate.java:125)
      at org.jboss.jms.client.FailoverCommandCenter.failureDetected(FailoverCommandCenter.java:141)
      at org.jboss.jms.client.container.ConnectionFailureListener.handleConnectionException(ConnectionFailureListener.java:62)
      at org.jboss.jms.client.remoting.ConsolidatedRemotingConnectionListener.handleConnectionException(ConsolidatedRemotingConnectionListener.java:81)
      at org.jboss.remoting.ConnectionValidator$1.run(ConnectionValidator.java:346)
      08:40:11,648 WARN @Thread-43 [ClusteringAspect] Client attempted failover, but no failover attempt has been detected on the server side. We will now try again on the original server in case there was a temporary glitch on the client--server network
      08:40:13,742 ERROR @Thread-43 [JMSWireFormat] Failed to write packet
      java.lang.NullPointerException
      at java.io.DataOutputStream.writeUTF(DataOutputStream.java:330)
      at java.io.DataOutputStream.writeUTF(DataOutputStream.java:306)
      at org.jboss.jms.delegate.DeliveryRecovery.write(DeliveryRecovery.java:79)
      at org.jboss.jms.wireformat.SessionRecoverDeliveriesRequest.write(SessionRecoverDeliveriesRequest.java:113)
      at org.jboss.jms.wireformat.JMSWireFormat.write(JMSWireFormat.java:237)
      at org.jboss.remoting.transport.socket.MicroSocketClientInvoker.versionedWrite(MicroSocketClientInvoker.java:969)
      at org.jboss.remoting.transport.socket.MicroSocketClientInvoker.transport(MicroSocketClientInvoker.java:557)
      at org.jboss.remoting.transport.bisocket.BisocketClientInvoker.transport(BisocketClientInvoker.java:353)
      at org.jboss.remoting.MicroRemoteClientInvoker.invoke(MicroRemoteClientInvoker.java:122)
      at org.jboss.remoting.Client.invoke(Client.java:1550)
      at org.jboss.remoting.Client.invoke(Client.java:530)
      at org.jboss.remoting.Client.invoke(Client.java:518)
      at org.jboss.jms.client.delegate.DelegateSupport.doInvoke(DelegateSupport.java:186)
      at org.jboss.jms.client.delegate.DelegateSupport.doInvoke(DelegateSupport.java:157)
      at org.jboss.jms.client.delegate.ClientSessionDelegate.org$jboss$jms$client$delegate$ClientSessionDelegate$recoverDeliveries$aop(ClientSessionDelegate.java:468)
      at org.jboss.jms.client.delegate.ClientSessionDelegate$recoverDeliveries_4196799148363996265.invokeNext(ClientSessionDelegate$recoverDeliveries_4196799148363996265.java)
      at org.jboss.jms.client.container.FailoverValveInterceptor.invoke(FailoverValveInterceptor.java:147)
      at org.jboss.aop.advice.PerInstanceInterceptor.invoke(PerInstanceInterceptor.java:105)
      at org.jboss.jms.client.delegate.ClientSessionDelegate$recoverDeliveries_4196799148363996265.invokeNext(ClientSessionDelegate$recoverDeliveries_4196799148363996265.java)
      at org.jboss.jms.client.container.ClosedInterceptor.invoke(ClosedInterceptor.java:170)
      at org.jboss.aop.advice.PerInstanceInterceptor.invoke(PerInstanceInterceptor.java:105)
      at org.jboss.jms.client.delegate.ClientSessionDelegate$recoverDeliveries_4196799148363996265.invokeNext(ClientSessionDelegate$recoverDeliveries_4196799148363996265.java)
      at org.jboss.jms.client.delegate.ClientSessionDelegate.recoverDeliveries(ClientSessionDelegate.java)
      at org.jboss.jms.client.state.SessionState.synchronizeWith(SessionState.java:344)
      at org.jboss.jms.client.delegate.ClientSessionDelegate.synchronizeWith(ClientSessionDelegate.java:122)
      at org.jboss.jms.client.state.ConnectionState.synchronizeWith(ConnectionState.java:174)
      at org.jboss.jms.client.delegate.ClientConnectionDelegate.synchronizeWith(ClientConnectionDelegate.java:125)
      at org.jboss.jms.client.FailoverCommandCenter.failureDetected(FailoverCommandCenter.java:141)
      at org.jboss.jms.client.container.ConnectionFailureListener.handleConnectionException(ConnectionFailureListener.java:62)
      at org.jboss.jms.client.remoting.ConsolidatedRemotingConnectionListener.handleConnectionException(ConsolidatedRemotingConnectionListener.java:81)
      at org.jboss.remoting.ConnectionValidator$1.run(ConnectionValidator.java:346)
      08:40:13,742 ERROR @Thread-43 [SocketClientInvoker] Got marshalling exception, exiting
      java.io.IOException
      at java.io.DataOutputStream.writeUTF(DataOutputStream.java:330)
      at java.io.DataOutputStream.writeUTF(DataOutputStream.java:306)
      at org.jboss.jms.delegate.DeliveryRecovery.write(DeliveryRecovery.java:79)
      at org.jboss.jms.wireformat.SessionRecoverDeliveriesRequest.write(SessionRecoverDeliveriesRequest.java:113)
      at org.jboss.jms.wireformat.JMSWireFormat.write(JMSWireFormat.java:237)
      at org.jboss.remoting.transport.socket.MicroSocketClientInvoker.versionedWrite(MicroSocketClientInvoker.java:969)
      at org.jboss.remoting.transport.socket.MicroSocketClientInvoker.transport(MicroSocketClientInvoker.java:557)
      at org.jboss.remoting.transport.bisocket.BisocketClientInvoker.transport(BisocketClientInvoker.java:353)
      at org.jboss.remoting.MicroRemoteClientInvoker.invoke(MicroRemoteClientInvoker.java:122)
      at org.jboss.remoting.Client.invoke(Client.java:1550)
      at org.jboss.remoting.Client.invoke(Client.java:530)
      at org.jboss.remoting.Client.invoke(Client.java:518)
      at org.jboss.jms.client.delegate.DelegateSupport.doInvoke(DelegateSupport.java:186)
      at org.jboss.jms.client.delegate.DelegateSupport.doInvoke(DelegateSupport.java:157)
      at org.jboss.jms.client.delegate.ClientSessionDelegate.org$jboss$jms$client$delegate$ClientSessionDelegate$recoverDeliveries$aop(ClientSessionDelegate.java:468)
      at org.jboss.jms.client.delegate.ClientSessionDelegate$recoverDeliveries_4196799148363996265.invokeNext(ClientSessionDelegate$recoverDeliveries_4196799148363996265.java)
      at org.jboss.jms.client.container.FailoverValveInterceptor.invoke(FailoverValveInterceptor.java:147)
      at org.jboss.aop.advice.PerInstanceInterceptor.invoke(PerInstanceInterceptor.java:105)
      at org.jboss.jms.client.delegate.ClientSessionDelegate$recoverDeliveries_4196799148363996265.invokeNext(ClientSessionDelegate$recoverDeliveries_4196799148363996265.java)
      at org.jboss.jms.client.container.ClosedInterceptor.invoke(ClosedInterceptor.java:170)
      at org.jboss.aop.advice.PerInstanceInterceptor.invoke(PerInstanceInterceptor.java:105)
      at org.jboss.jms.client.delegate.ClientSessionDelegate$recoverDeliveries_4196799148363996265.invokeNext(ClientSessionDelegate$recoverDeliveries_4196799148363996265.java)
      at org.jboss.jms.client.delegate.ClientSessionDelegate.recoverDeliveries(ClientSessionDelegate.java)
      at org.jboss.jms.client.state.SessionState.synchronizeWith(SessionState.java:344)
      at org.jboss.jms.client.delegate.ClientSessionDelegate.synchronizeWith(ClientSessionDelegate.java:122)
      at org.jboss.jms.client.state.ConnectionState.synchronizeWith(ConnectionState.java:174)
      at org.jboss.jms.client.delegate.ClientConnectionDelegate.synchronizeWith(ClientConnectionDelegate.java:125)
      at org.jboss.jms.client.FailoverCommandCenter.failureDetected(FailoverCommandCenter.java:141)
      at org.jboss.jms.client.container.ConnectionFailureListener.handleConnectionException(ConnectionFailureListener.java:62)
      at org.jboss.jms.client.remoting.ConsolidatedRemotingConnectionListener.handleConnectionException(ConsolidatedRemotingConnectionListener.java:81)
      at org.jboss.remoting.ConnectionValidator$1.run(ConnectionValidator.java:346)
      08:40:13,757 ERROR @Thread-43 [FailoverCommandCenter] Failover failed
      org.jboss.jms.exception.MessagingNetworkFailureException
      at org.jboss.jms.client.delegate.DelegateSupport.handleThrowable(DelegateSupport.java:221)
      at org.jboss.jms.client.delegate.DelegateSupport.doInvoke(DelegateSupport.java:202)
      at org.jboss.jms.client.delegate.DelegateSupport.doInvoke(DelegateSupport.java:157)
      at org.jboss.jms.client.delegate.ClientSessionDelegate.org$jboss$jms$client$delegate$ClientSessionDelegate$recoverDeliveries$aop(ClientSessionDelegate.java:468)
      at org.jboss.jms.client.delegate.ClientSessionDelegate$recoverDeliveries_4196799148363996265.invokeNext(ClientSessionDelegate$recoverDeliveries_4196799148363996265.java)
      at org.jboss.jms.client.container.FailoverValveInterceptor.invoke(FailoverValveInterceptor.java:147)
      at org.jboss.aop.advice.PerInstanceInterceptor.invoke(PerInstanceInterceptor.java:105)
      at org.jboss.jms.client.delegate.ClientSessionDelegate$recoverDeliveries_4196799148363996265.invokeNext(ClientSessionDelegate$recoverDeliveries_4196799148363996265.java)
      at org.jboss.jms.client.container.ClosedInterceptor.invoke(ClosedInterceptor.java:170)
      at org.jboss.aop.advice.PerInstanceInterceptor.invoke(PerInstanceInterceptor.java:105)
      at org.jboss.jms.client.delegate.ClientSessionDelegate$recoverDeliveries_4196799148363996265.invokeNext(ClientSessionDelegate$recoverDeliveries_4196799148363996265.java)
      at org.jboss.jms.client.delegate.ClientSessionDelegate.recoverDeliveries(ClientSessionDelegate.java)
      at org.jboss.jms.client.state.SessionState.synchronizeWith(SessionState.java:344)
      at org.jboss.jms.client.delegate.ClientSessionDelegate.synchronizeWith(ClientSessionDelegate.java:122)
      at org.jboss.jms.client.state.ConnectionState.synchronizeWith(ConnectionState.java:174)
      at org.jboss.jms.client.delegate.ClientConnectionDelegate.synchronizeWith(ClientConnectionDelegate.java:125)
      at org.jboss.jms.client.FailoverCommandCenter.failureDetected(FailoverCommandCenter.java:141)
      at org.jboss.jms.client.container.ConnectionFailureListener.handleConnectionException(ConnectionFailureListener.java:62)
      at org.jboss.jms.client.remoting.ConsolidatedRemotingConnectionListener.handleConnectionException(ConsolidatedRemotingConnectionListener.java:81)
      at org.jboss.remoting.ConnectionValidator$1.run(ConnectionValidator.java:346)
      Caused by: java.rmi.MarshalException: Failed to communicate. Problem during marshalling/unmarshalling; nested exception is:
      java.io.IOException
      at org.jboss.remoting.transport.socket.SocketClientInvoker.handleException(SocketClientInvoker.java:122)
      at org.jboss.remoting.transport.socket.MicroSocketClientInvoker.transport(MicroSocketClientInvoker.java:644)
      at org.jboss.remoting.transport.bisocket.BisocketClientInvoker.transport(BisocketClientInvoker.java:353)
      at org.jboss.remoting.MicroRemoteClientInvoker.invoke(MicroRemoteClientInvoker.java:122)
      at org.jboss.remoting.Client.invoke(Client.java:1550)
      at org.jboss.remoting.Client.invoke(Client.java:530)
      at org.jboss.remoting.Client.invoke(Client.java:518)
      at org.jboss.jms.client.delegate.DelegateSupport.doInvoke(DelegateSupport.java:186)
      ... 18 more
      Caused by: java.io.IOException
      at java.io.DataOutputStream.writeUTF(DataOutputStream.java:330)
      at java.io.DataOutputStream.writeUTF(DataOutputStream.java:306)
      at org.jboss.jms.delegate.DeliveryRecovery.write(DeliveryRecovery.java:79)
      at org.jboss.jms.wireformat.SessionRecoverDeliveriesRequest.write(SessionRecoverDeliveriesRequest.java:113)
      at org.jboss.jms.wireformat.JMSWireFormat.write(JMSWireFormat.java:237)
      at org.jboss.remoting.transport.socket.MicroSocketClientInvoker.versionedWrite(MicroSocketClientInvoker.java:969)
      at org.jboss.remoting.transport.socket.MicroSocketClientInvoker.transport(MicroSocketClientInvoker.java:557)
      ... 24 more
      08:40:14,210 ERROR @Thread-43 [ConnectionFailureListener] Caught exception in handling failure
      org.jboss.jms.exception.MessagingNetworkFailureException
      at org.jboss.jms.client.delegate.DelegateSupport.handleThrowable(DelegateSupport.java:221)
      at org.jboss.jms.client.delegate.DelegateSupport.doInvoke(DelegateSupport.java:202)
      at org.jboss.jms.client.delegate.DelegateSupport.doInvoke(DelegateSupport.java:157)
      at org.jboss.jms.client.delegate.ClientSessionDelegate.org$jboss$jms$client$delegate$ClientSessionDelegate$recoverDeliveries$aop(ClientSessionDelegate.java:468)
      at org.jboss.jms.client.delegate.ClientSessionDelegate$recoverDeliveries_4196799148363996265.invokeNext(ClientSessionDelegate$recoverDeliveries_4196799148363996265.java)
      at org.jboss.jms.client.container.FailoverValveInterceptor.invoke(FailoverValveInterceptor.java:147)
      at org.jboss.aop.advice.PerInstanceInterceptor.invoke(PerInstanceInterceptor.java:105)
      at org.jboss.jms.client.delegate.ClientSessionDelegate$recoverDeliveries_4196799148363996265.invokeNext(ClientSessionDelegate$recoverDeliveries_4196799148363996265.java)
      at org.jboss.jms.client.container.ClosedInterceptor.invoke(ClosedInterceptor.java:170)
      at org.jboss.aop.advice.PerInstanceInterceptor.invoke(PerInstanceInterceptor.java:105)
      at org.jboss.jms.client.delegate.ClientSessionDelegate$recoverDeliveries_4196799148363996265.invokeNext(ClientSessionDelegate$recoverDeliveries_4196799148363996265.java)
      at org.jboss.jms.client.delegate.ClientSessionDelegate.recoverDeliveries(ClientSessionDelegate.java)
      at org.jboss.jms.client.state.SessionState.synchronizeWith(SessionState.java:344)
      at org.jboss.jms.client.delegate.ClientSessionDelegate.synchronizeWith(ClientSessionDelegate.java:122)
      at org.jboss.jms.client.state.ConnectionState.synchronizeWith(ConnectionState.java:174)
      at org.jboss.jms.client.delegate.ClientConnectionDelegate.synchronizeWith(ClientConnectionDelegate.java:125)
      at org.jboss.jms.client.FailoverCommandCenter.failureDetected(FailoverCommandCenter.java:141)
      at org.jboss.jms.client.container.ConnectionFailureListener.handleConnectionException(ConnectionFailureListener.java:62)
      at org.jboss.jms.client.remoting.ConsolidatedRemotingConnectionListener.handleConnectionException(ConsolidatedRemotingConnectionListener.java:81)
      at org.jboss.remoting.ConnectionValidator$1.run(ConnectionValidator.java:346)
      Caused by: java.rmi.MarshalException: Failed to communicate. Problem during marshalling/unmarshalling; nested exception is:
      java.io.IOException
      at org.jboss.remoting.transport.socket.SocketClientInvoker.handleException(SocketClientInvoker.java:122)
      at org.jboss.remoting.transport.socket.MicroSocketClientInvoker.transport(MicroSocketClientInvoker.java:644)
      at org.jboss.remoting.transport.bisocket.BisocketClientInvoker.transport(BisocketClientInvoker.java:353)
      at org.jboss.remoting.MicroRemoteClientInvoker.invoke(MicroRemoteClientInvoker.java:122)
      at org.jboss.remoting.Client.invoke(Client.java:1550)
      at org.jboss.remoting.Client.invoke(Client.java:530)
      at org.jboss.remoting.Client.invoke(Client.java:518)
      at org.jboss.jms.client.delegate.DelegateSupport.doInvoke(DelegateSupport.java:186)
      ... 18 more
      Caused by: java.io.IOException
      at java.io.DataOutputStream.writeUTF(DataOutputStream.java:330)
      at java.io.DataOutputStream.writeUTF(DataOutputStream.java:306)
      at org.jboss.jms.delegate.DeliveryRecovery.write(DeliveryRecovery.java:79)
      at org.jboss.jms.wireformat.SessionRecoverDeliveriesRequest.write(SessionRecoverDeliveriesRequest.java:113)
      at org.jboss.jms.wireformat.JMSWireFormat.write(JMSWireFormat.java:237)
      at org.jboss.remoting.transport.socket.MicroSocketClientInvoker.versionedWrite(MicroSocketClientInvoker.java:969)
      at org.jboss.remoting.transport.socket.MicroSocketClientInvoker.transport(MicroSocketClientInvoker.java:557)
      ... 24 more