JGroups exception over bridged network
yelin666 Nov 23, 2010 2:30 PMI am running two cache instances - one on a VM managed by VMware, and the other on the host machine, and they are connected by the bridged network. Is this supported by JGroups? I am using TCP configuration with attached JGroups config files. I tested with different scenarios, and got the following observations. Please suggest what's the possible problems:
1. If I start the CacheManager and get a cache on the host machine first, then when I start the CacheManager and get a cache on the VM, get the following warning & exception on the VM instance:
2010-11-23 11:52:43,166 -0500 | INFO | JChannel | apache.log4j.Category | 849 | JGroups version: 2.10.0.GA
2010-11-23 11:52:45,577 -0500 | WARN | FD_SOCK | apache.log4j.Category | 846 | I (localhost-47185) was suspected by T00696119-61197; ignoring the SUSPECT message
2010-11-23 11:52:45,629 -0500 | INFO | GlobalComponentRegistry | apache.log4j.Category | 849 | Infinispan version: Infinispan 'Radegast' 4.1.0.FINAL
2010-11-23 11:52:45,741 -0500 | WARN | STREAMING_STATE_TRANSFER | apache.log4j.Category | 846 | State reader socket thread spawned abnormaly
java.net.SocketException: No such device
at java.net.PlainSocketImpl.socketConnect(Native Method)
at java.net.PlainSocketImpl.doConnect(PlainSocketImpl.java:333)
at java.net.PlainSocketImpl.connectToAddress(PlainSocketImpl.java:195)
at java.net.PlainSocketImpl.connect(PlainSocketImpl.java:182)
at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:366)
at java.net.Socket.connect(Socket.java:529)
at org.jgroups.util.Util.connect(Util.java:247)
at org.jgroups.protocols.pbcast.STREAMING_STATE_TRANSFER.connectToStateProvider(STREAMING_STATE_TRANSFER.java:512)
at org.jgroups.protocols.pbcast.STREAMING_STATE_TRANSFER.handleStateRsp(STREAMING_STATE_TRANSFER.java:464)
at org.jgroups.protocols.pbcast.STREAMING_STATE_TRANSFER.up(STREAMING_STATE_TRANSFER.java:225)
at org.jgroups.protocols.FRAG2.up(FRAG2.java:188)
at org.jgroups.protocols.FC.up(FC.java:474)
at org.jgroups.protocols.pbcast.GMS.up(GMS.java:888)
at org.jgroups.protocols.pbcast.STABLE.up(STABLE.java:234)
at org.jgroups.protocols.UNICAST.handleDataReceived(UNICAST.java:614)
at org.jgroups.protocols.UNICAST.up(UNICAST.java:294)
at org.jgroups.protocols.pbcast.NAKACK.up(NAKACK.java:707)
at org.jgroups.protocols.VERIFY_SUSPECT.up(VERIFY_SUSPECT.java:132)
at org.jgroups.protocols.FD.up(FD.java:266)
at org.jgroups.protocols.FD_SOCK.up(FD_SOCK.java:270)
at org.jgroups.protocols.MERGE2.up(MERGE2.java:210)
at org.jgroups.protocols.Discovery.up(Discovery.java:281)
at org.jgroups.protocols.TP.passMessageUp(TP.java:1009)
at org.jgroups.protocols.TP.access$100(TP.java:56)
at org.jgroups.protocols.TP$IncomingPacket.handleMyMessage(TP.java:1549)
at org.jgroups.protocols.TP$IncomingPacket.run(TP.java:1531)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:619)
2010-11-23 11:52:45,744 -0500 | WARN | STREAMING_STATE_TRANSFER | apache.log4j.Category | 846 | Could not connect to state provider. Closing socket...
2. If I start the CacheManager and get a cache on the VM first, then start the CacheManager and get a cache on the host machine, sometimes the data replication works. However, even when it works, I get the following warnings:
2010-11-23 10:38:59,005 -0500 | WARN | NAKACK | apache.log4j.Category | 846 | T00696119-29820: dropped message from localhost-39737 (not in xmit_table), keys are [T00696119-29820], view=[T00696119-29820|0] [T00696119-29820] (on host machine)
2010-11-23 10:38:55,348 -0500 | WARN | NAKACK | apache.log4j.Category | 846 | localhost-39737: dropped message from T00696119-29820 (not in xmit_table), keys are [localhost-39737], view=[localhost-39737|0] [localhost-39737] (on VM)
Sometimes replication failed with the following exception on the VM:
2010-11-23 10:21:13,464 -0500 | WARN | TCPConnectionMap | apache.log4j.Category | 846 | Could not accept connection from peer
java.net.SocketException: Connection reset
at java.net.SocketInputStream.read(SocketInputStream.java:168)
at java.io.BufferedInputStream.fill(BufferedInputStream.java:218)
at java.io.BufferedInputStream.read1(BufferedInputStream.java:258)
at java.io.BufferedInputStream.read(BufferedInputStream.java:317)
at java.io.DataInputStream.readFully(DataInputStream.java:178)
at org.jgroups.blocks.TCPConnectionMap$TCPConnection.readPeerAddress(TCPConnectionMap.java:485)
at org.jgroups.blocks.TCPConnectionMap$TCPConnection.<init>(TCPConnectionMap.java:375)
at org.jgroups.blocks.TCPConnectionMap$ConnectionAcceptor.run(TCPConnectionMap.java:246)
at java.lang.Thread.run(Thread.java:619)
2010-11-23 10:21:17,405 -0500 | WARN | NAKACK | apache.log4j.Category | 846 | localhost-28269: dropped message from T00696119-15155 (not in xmit_table), keys are [localhost-28269], view=[localhost-28269|0] [localhost-28269]
2010-11-23 10:21:17,405 -0500 | WARN | NAKACK | apache.log4j.Category | 846 | localhost-28269: dropped message from T00696119-15155 (not in xmit_table), keys are [localhost-28269], view=[localhost-28269|0] [localhost-28269]
2010-11-23 10:22:26,575 -0500 | WARN | NAKACK | apache.log4j.Category | 846 | localhost-28269: dropped message from T00696119-15155 (not in xmit_table), keys are [localhost-28269], view=[localhost-28269|0] [localhost-28269]
2010-11-23 10:22:29,862 -0500 | WARN | NAKACK | apache.log4j.Category | 846 | localhost-28269: dropped message from T00696119-15155 (not in xmit_table), keys are [localhost-28269], view=[localhost-28269|0] [localhost-28269]
2010-11-23 10:22:45,702 -0500 | WARN | TCPConnectionMap | apache.log4j.Category | 846 | Could not accept connection from peer
java.net.SocketException: ConnectionTable.Connection.readPeerAddress(): cookie read by 3.96.157.49:7800 does not match own cookie; terminating connection
at org.jgroups.blocks.TCPConnectionMap$TCPConnection.readPeerAddress(TCPConnectionMap.java:487)
at org.jgroups.blocks.TCPConnectionMap$TCPConnection.<init>(TCPConnectionMap.java:375)
at org.jgroups.blocks.TCPConnectionMap$ConnectionAcceptor.run(TCPConnectionMap.java:246)
at java.lang.Thread.run(Thread.java:619)
2010-11-23 10:23:26,832 -0500 | WARN | TCPConnectionMap | apache.log4j.Category | 846 | Could not accept connection from peer
java.net.SocketException: ConnectionTable.Connection.readPeerAddress(): cookie read by 3.96.157.49:7800 does not match own cookie; terminating connection
at org.jgroups.blocks.TCPConnectionMap$TCPConnection.readPeerAddress(TCPConnectionMap.java:487)
at org.jgroups.blocks.TCPConnectionMap$TCPConnection.<init>(TCPConnectionMap.java:375)
at org.jgroups.blocks.TCPConnectionMap$ConnectionAcceptor.run(TCPConnectionMap.java:246)
at java.lang.Thread.run(Thread.java:619)
(10.96.157.49 is the VM IP address)
-
jgroups-tcp_VM.xml 2.4 KB
-
jgroups-tcp_Host.xml 2.4 KB