Hi Ron
Thanks for your reply.
I have tried what you suggested and many other tests with different settings with no luck. In my first test where I disconnect the link on eth0, and eth1 should take over seems to work. But when I do the test with connecting the eth0 again and disconnect eth1 I always fails and it stops receiving messages
I have attach my latest changes to remoting-bisocket-service.xml and I have tried numberOfCallRetries with the value 1, 2 and 10. I have also have attach a snippet of a TRACE out. It keep getting “Read timed out” on the socket and it never get’s out of this mode.
Any other suggestions ?
From the remoting-bisocket-service.xml :
<attribute name="socket.check_connection" isParam="true">true</attribute>
<attribute name="keepAlive" isParam="true">true</attribute>
<attribute name="numberOfCallRetries" isParam="true">10</attribute>
<attribute name="pingFrequency" isParam="true">30000</attribute>
<attribute name="pingWindowFactor" isParam="true">1</attribute>
<attribute name="generalizeSocketException" isParam="true">true</attribute>
TRACE from a test with disconnection the eth link
2013-11-12 14:56:29,759 TRACE [SocketWrapper] NEW ClientSocketWrapper[Socket[addr=/10.81.80.34,port=4457,localport=64245].4b1f5d8c] closing socket
2013-11-12 14:56:29,759 TRACE [SocketWrapper] NEW ClientSocketWrapper[Socket[addr=/10.81.80.34,port=4457,localport=64245].4b1f5d8c] closed socket
2013-11-12 14:56:29,759 TRACE [MicroSocketClientInvoker] SocketClientInvoker[46938420, bisocket://10.81.80.34:4457] reusing pooled connection: null
2013-11-12 14:56:29,759 TRACE [MicroSocketClientInvoker] SocketClientInvoker[46938420, bisocket://10.81.80.34:4457] creating socket
2013-11-12 14:56:29,760 TRACE [MicroSocketClientInvoker] SocketClientInvoker[46938420, bisocket://10.81.80.34:4457] created socket: Socket[addr=/10.81.80.34,port=4457,localport=64248]
2013-11-12 14:56:29,760 TRACE [SocketWrapper] constructing org.jboss.jms.client.remoting.ClientSocketWrapper instance for Socket[addr=/10.81.80.34,port=4457,localport=64248], using timeout 10000
2013-11-12 14:56:29,760 TRACE [SocketWrapper] NEW ClientSocketWrapper[Socket[addr=/10.81.80.34,port=4457,localport=64248].5a2bf583] setting timeout to 10000
2013-11-12 14:56:29,760 TRACE [ClientSocketWrapper] NEW ClientSocketWrapper[Socket[addr=/10.81.80.34,port=4457,localport=64248].5a2bf583] getting output stream from Socket[addr=/10.81.80.34,port=4457,localport=64248], org.jboss.jms.wireformat.JMSWireFormat@5fc02db5
2013-11-12 14:56:29,760 TRACE [SocketWrapper] NEW ClientSocketWrapper[Socket[addr=/10.81.80.34,port=4457,localport=64248].5a2bf583] setting timeout to 10000
2013-11-12 14:56:29,760 TRACE [ClientSocketWrapper] reset timeout: 10000
2013-11-12 14:56:29,760 TRACE [ClientSocketWrapper] NEW ClientSocketWrapper[Socket[addr=/10.81.80.34,port=4457,localport=64248].5a2bf583] getting output stream from Socket[addr=/10.81.80.34,port=4457,localport=64248], org.jboss.jms.wireformat.JMSWireFormat@5fc02db5
2013-11-12 14:56:29,760 TRACE [SocketWrapper] NEW ClientSocketWrapper[Socket[addr=/10.81.80.34,port=4457,localport=64248].5a2bf583] setting timeout to 10000
2013-11-12 14:56:29,760 TRACE [ClientSocketWrapper] reset timeout: 10000
2013-11-12 14:56:29,760 TRACE [MicroSocketClientInvoker] SocketClientInvoker[46938420, bisocket://10.81.80.34:4457] got socketWrapper: NEW ClientSocketWrapper[Socket[addr=/10.81.80.34,port=4457,localport=64248].5a2bf583]
2013-11-12 14:56:29,760 TRACE [MicroSocketClientInvoker] SocketClientInvoker[46938420, bisocket://10.81.80.34:4457]got outputStream: java.io.DataOutputStream@5cfe7cca
2013-11-12 14:56:29,760 TRACE [MicroSocketClientInvoker] SocketClientInvoker[46938420, bisocket://10.81.80.34:4457] writing version
2013-11-12 14:56:29,760 TRACE [MicroSocketClientInvoker] SocketClientInvoker[46938420, bisocket://10.81.80.34:4457] writing version 22 on output stream
2013-11-12 14:56:29,760 TRACE [MicroSocketClientInvoker] SocketClientInvoker[46938420, bisocket://10.81.80.34:4457] wrote version
2013-11-12 14:56:29,760 TRACE [MicroSocketClientInvoker] SocketClientInvoker[46938420, bisocket://10.81.80.34:4457] writing invocation to marshaller
2013-11-12 14:56:29,760 TRACE [MicroSocketClientInvoker] SocketClientInvoker[46938420, bisocket://10.81.80.34:4457] done writing invocation to marshaller
2013-11-12 14:56:29,760 TRACE [MicroSocketClientInvoker] SocketClientInvoker[46938420, bisocket://10.81.80.34:4457] reading version from input stream
2013-11-12 14:56:29,761 TRACE [MicroSocketClientInvoker] SocketClientInvoker[46938420, bisocket://10.81.80.34:4457] read version 22 from input stream
2013-11-12 14:56:29,761 TRACE [MicroSocketClientInvoker] SocketClientInvoker[46938420, bisocket://10.81.80.34:4457] reading response from unmarshaller
2013-11-12 14:56:29,762 TRACE [MicroSocketClientInvoker] SocketClientInvoker[46938420, bisocket://10.81.80.34:4457] returned NEW ClientSocketWrapper[Socket[addr=/10.81.80.34,port=4457,localport=64248].5a2bf583] to pool
2013-11-12 14:56:29,762 TRACE [MicroSocketClientInvoker] SocketClientInvoker[46938420, bisocket://10.81.80.34:4457] released semaphore: 50
2013-11-12 14:56:29,762 TRACE [MicroSocketClientInvoker] SocketClientInvoker[46938420, bisocket://10.81.80.34:4457] received response InvocationResponse[30f0b20e, InvocationResponse[a39ce3c, true]]
2013-11-12 14:56:29,762 TRACE [MicroRemoteClientInvoker] SocketClientInvoker[46938420, bisocket://10.81.80.34:4457] received InvocationResponse so going to return response's return value of InvocationResponse[a39ce3c, true]
2013-11-12 14:56:29,762 TRACE [ConnectionValidator] ConnectionValidator got successful ping using SocketClientInvoker[46938420, bisocket://10.81.80.34:4457]
2013-11-12 14:56:30,368 ERROR [ServerThread] WorkerThread#0[10.81.80.34:4460] exception occurred during first invocation
java.net.SocketTimeoutException: Read timed out
at java.net.SocketInputStream.socketRead0(Native Method)
at java.net.SocketInputStream.read(SocketInputStream.java:129)
at java.io.BufferedInputStream.fill(BufferedInputStream.java:218)
at java.io.BufferedInputStream.read(BufferedInputStream.java:237)
at java.io.FilterInputStream.read(FilterInputStream.java:66)
at org.jboss.remoting.transport.socket.ServerThread.readVersion(ServerThread.java:1104)
at org.jboss.remoting.transport.socket.ServerThread.processInvocation(ServerThread.java:720)
at org.jboss.remoting.transport.socket.ServerThread.dorun(ServerThread.java:548)
at org.jboss.remoting.transport.socket.ServerThread.run(ServerThread.java:234)
2013-11-12 14:56:30,368 TRACE [SocketWrapper] ServerSocketWrapper[Socket[addr=/10.81.80.34,port=4460,localport=64246].679f5191] closing socket
2013-11-12 14:56:30,368 TRACE [SocketWrapper] ServerSocketWrapper[Socket[addr=/10.81.80.34,port=4460,localport=64246].679f5191] closed socket
2013-11-12 14:56:30,368 DEBUG [ServerThread] WorkerThread#0[10.81.80.34:4460] closed socketWrapper: ServerSocketWrapper[Socket[addr=/10.81.80.34,port=4460,localport=64246].679f5191]
2013-11-12 14:56:30,368 TRACE [ServerThread] WorkerThread#0[10.81.80.34:4460] removing itself from clientpool
2013-11-12 14:56:30,368 TRACE [ServerThread] WorkerThread#0[10.81.80.34:4460] returning itself to threadpool
2013-11-12 14:56:30,368 TRACE [ServerThread] WorkerThread#0[10.81.80.34:4460] begins to wait
2013-11-12 14:56:34,701 TRACE [LeasePinger] LeasePinger[a60241m-5tpzqz-hnx66yh6-1-hnx66yly-6:SocketClientInvoker[7d4e3c21, bisocket://10.81.80.34:4457](a60241m-5tpzqz-hnx66yh6-1-hnx66yl3-5)] sending ping to server. Currently managing lease for following clients:
a60241m-5tpzqz-hnx66yh6-1-hnx66ysp-g
a60241m-5tpzqz-hnx66yh6-1-hnx66ykp-3
2013-11-12 14:56:34,701 TRACE [MicroRemoteClientInvoker] SocketClientInvoker[7d4e3c21, bisocket://10.81.80.34:4457](1) invoking InvocationRequest[5912ca62, $PING$]
2013-11-12 14:56:34,701 TRACE [MicroSocketClientInvoker] SocketClientInvoker[7d4e3c21, bisocket://10.81.80.34:4457] retryCount: 0
2013-11-12 14:56:34,701 TRACE [MicroSocketClientInvoker] SocketClientInvoker[7d4e3c21, bisocket://10.81.80.34:4457] obtained semaphore: 199
2013-11-12 14:56:34,701 TRACE [SocketClientInvoker] Couldn't reuse connection from pool
java.io.IOException: remote endpoint has closed
at org.jboss.jms.client.remoting.ClientSocketWrapper.checkOpenConnection(ClientSocketWrapper.java:108)
at org.jboss.remoting.transport.socket.SocketClientInvoker.getPooledConnection(SocketClientInvoker.java:226)
at org.jboss.remoting.transport.socket.MicroSocketClientInvoker.getConnection(MicroSocketClientInvoker.java:1141)
at org.jboss.remoting.transport.socket.MicroSocketClientInvoker.transport(MicroSocketClientInvoker.java:816)
at org.jboss.remoting.transport.bisocket.BisocketClientInvoker.transport(BisocketClientInvoker.java:470)
at org.jboss.remoting.MicroRemoteClientInvoker.invoke(MicroRemoteClientInvoker.java:169)
at org.jboss.remoting.LeasePinger.sendClientPing(LeasePinger.java:416)
at org.jboss.remoting.LeasePinger.access$000(LeasePinger.java:19)
at org.jboss.remoting.LeasePinger$LeaseTimerTask.run(LeasePinger.java:446)
at java.util.TimerThread.mainLoop(Timer.java:512)
at java.util.TimerThread.run(Timer.java:462)
2013-11-12 14:56:34,701 TRACE [SocketWrapper] NEW ClientSocketWrapper[Socket[addr=/10.81.80.34,port=4457,localport=64244].79e9ac03] closing socket
2013-11-12 14:56:34,701 TRACE [SocketWrapper] NEW ClientSocketWrapper[Socket[addr=/10.81.80.34,port=4457,localport=64244].79e9ac03] closed socket
2013-11-12 14:56:34,701 TRACE [MicroSocketClientInvoker] SocketClientInvoker[7d4e3c21, bisocket://10.81.80.34:4457] reusing pooled connection: null
2013-11-12 14:56:34,701 TRACE [MicroSocketClientInvoker] SocketClientInvoker[7d4e3c21, bisocket://10.81.80.34:4457] creating socket
2013-11-12 14:56:34,702 TRACE [MicroSocketClientInvoker] SocketClientInvoker[7d4e3c21, bisocket://10.81.80.34:4457] created socket: Socket[addr=/10.81.80.34,port=4457,localport=64249]