Encryption related issue
Hi!
I'm using Infinispan 4.2.1 Final with JGroups 2.12.0.CR5. My cluster is using JGroups encrypction (simple option - http://www.jgroups.org/javadoc/org/jgroups/protocols/ENCRYPT.html Option 1). When keys are the same on both nodes everything is ok. But I've made simple rainy day scenario - I've changed cluster key on one node. In this situation it looks like node whose are joning cluster don't know about fact that he has wrong key. It is trying to retrive state but due to wrong key after 40s there is exception thrown:
2011-05-18 14:00:56,191 ERROR [org.infinispan.remoting.transport.jgroups.JGroupsTransport] Incoming-1,MyCluster,NodeB-2809: Caught while requesting or applying state org.infinispan.statetransfer.StateTransferException: org.infinispan.util.concurrent.TimeoutException: Timed out after 40 seconds waiting for a response from NodeA-64528 at org.infinispan.statetransfer.StateTransferManagerImpl.applyState(StateTransferManagerImpl.java:332) at org.infinispan.remoting.InboundInvocationHandlerImpl.applyState(InboundInvocationHandlerImpl.java:199) at org.infinispan.remoting.transport.jgroups.JGroupsTransport.setState(JGroupsTransport.java:595) at org.jgroups.blocks.MessageDispatcher$ProtocolAdapter.handleUpEvent(MessageDispatcher.java:711) at org.jgroups.blocks.MessageDispatcher$ProtocolAdapter.up(MessageDispatcher.java:771) at org.jgroups.JChannel.up(JChannel.java:1441) at org.jgroups.stack.ProtocolStack.up(ProtocolStack.java:1074) at org.jgroups.protocols.pbcast.STREAMING_STATE_TRANSFER.connectToStateProvider(STREAMING_STATE_TRANSFER.java:523) at org.jgroups.protocols.pbcast.STREAMING_STATE_TRANSFER.handleStateRsp(STREAMING_STATE_TRANSFER.java:462) at org.jgroups.protocols.pbcast.STREAMING_STATE_TRANSFER.up(STREAMING_STATE_TRANSFER.java:223) at org.jgroups.protocols.pbcast.FLUSH.up(FLUSH.java:429) at org.jgroups.protocols.FRAG2.up(FRAG2.java:189) at org.jgroups.protocols.FlowControl.up(FlowControl.java:418) at org.jgroups.protocols.FlowControl.up(FlowControl.java:400) at org.jgroups.protocols.pbcast.GMS.up(GMS.java:891) at org.jgroups.protocols.pbcast.STABLE.up(STABLE.java:246) at org.jgroups.protocols.UNICAST.handleDataReceived(UNICAST.java:613) at org.jgroups.protocols.UNICAST.up(UNICAST.java:294) at org.jgroups.protocols.pbcast.NAKACK.up(NAKACK.java:703) at org.jgroups.protocols.VERIFY_SUSPECT.up(VERIFY_SUSPECT.java:133) at org.jgroups.protocols.FD.up(FD.java:275) at org.jgroups.protocols.FD_SOCK.up(FD_SOCK.java:275) at org.jgroups.protocols.MERGE2.up(MERGE2.java:209) at org.jgroups.protocols.Discovery.up(Discovery.java:291) at org.jgroups.protocols.PING.up(PING.java:66) at org.jgroups.protocols.MPING.up(MPING.java:176) at org.jgroups.protocols.ENCRYPT.passItUp(ENCRYPT.java:455) at org.jgroups.protocols.ENCRYPT.up(ENCRYPT.java:440) at org.jgroups.protocols.TP.passMessageUp(TP.java:1102) at org.jgroups.protocols.TP$IncomingPacket.handleMyMessage(TP.java:1658) at org.jgroups.protocols.TP$IncomingPacket.run(TP.java:1640) at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) at java.lang.Thread.run(Thread.java:619) Caused by: org.infinispan.util.concurrent.TimeoutException: Timed out after 40 seconds waiting for a response from WSGnode28-64528 at org.infinispan.remoting.transport.jgroups.CommandAwareRpcDispatcher$ReplicationTask.call(CommandAwareRpcDispatcher.java:274) at org.infinispan.remoting.transport.jgroups.CommandAwareRpcDispatcher.invokeRemoteCommands(CommandAwareRpcDispatcher.java:116) at org.infinispan.remoting.transport.jgroups.JGroupsTransport.invokeRemotely(JGroupsTransport.java:407) at org.infinispan.remoting.rpc.RpcManagerImpl.invokeRemotely(RpcManagerImpl.java:102) at org.infinispan.remoting.rpc.RpcManagerImpl.invokeRemotely(RpcManagerImpl.java:126) at org.infinispan.statetransfer.StateTransferManagerImpl.mimicPartialFlushViaRPC(StateTransferManagerImpl.java:300) at org.infinispan.statetransfer.StateTransferManagerImpl.applyTransactionLog(StateTransferManagerImpl.java:253) at org.infinispan.statetransfer.StateTransferManagerImpl.applyState(StateTransferManagerImpl.java:321) ... 34 more
Moreover I've noticed that NodeB (this one with wrong key) hangs up after that... He is waiting for notify signal at:
stateLock.wait();
in org.infinispan.remoting.transport.jgroups.StateTransferMonitor:28
On second node I can see in log files that he knows that something is wrong with received encrypted messages from joining node:
2011-05-17 23:01:53,034 WARN [org.jgroups.protocols.ENCRYPT] OOB-14,MyCluster,NodeA-64528: attempting to use stored cipher as message does not uses current encryption version 2011-05-17 23:01:53,034 WARN [org.jgroups.protocols.ENCRYPT] OOB-14,MyCluster,NodeA-64528: Unable to find a matching cipher in previous key map 2011-05-17 23:01:53,034 WARN [org.jgroups.protocols.ENCRYPT] OOB-14,MyCluster,NodeA-64528: Unrecognised cipher discarding message
Is there any possibility to check if key is correct (communication could be established between nodes)? In my situation, joining node don't know that he has wrong key...
Regards,
Juliusz
