Wildfly 10 (KeyCloak) HA on AWS EC2 with docker - cluster is up but login fails
haimp Aug 17, 2016 5:02 AMHi,
We are trying to set Wildfly 10 (KeyCloak 1.9.3) with HA on AWS EC2 with docker, the cluster is up without errors however the login fails with the below error:
WARN [org.keycloak.events] (default task-10) type=LOGIN_ERROR, realmId=master, clientId=null, userId=null, ipAddress=172.30.200.171, error=invalid_code
we have followed this (http://lists.jboss.org/pipermail/keycloak-user/2016-February/004940.html ) post but used S3_PING instead of JDBC_PING.
It seems that the nodes detect each other:
INFO [org.infinispan.remoting.transport.jgroups.JGroupsTransport] (Incoming-2,ee,6dbce1e2a05a) ISPN000094: Received new cluster view for channel keycloak: [6dbce1e2a05a|1] (2) [6dbce1e2a05a, 75f2b2e98cfd]
We suspect that the nodes doesn't communicate with each other, when we queried the jboss mbean "jboss.as.expr:subsystem=jgroups,channel=ee" the result was:
jgroups,channel=ee = [6dbce1e2a05a|1] (2) [6dbce1e2a05a, 75f2b2e98cfd]
jgroups,channel=ee receivedMessages = 0
jgroups,channel=ee sentMessages = 0
And for the second node:
jgroups,channel=ee = [6dbce1e2a05a|1] (2) [6dbce1e2a05a, 75f2b2e98cfd]
jgroups,channel=ee receivedMessages = 0
jgroups,channel=ee sentMessages = 5
We also verified that the TCP ports 57600 and 7600 are open.
Any idea what might cause it ?
Here is the relevant standalone-ha.xml configuration and below is that startup command:
<subsystem xmlns="urn:jboss:domain:jgroups:4.0">
<channels default="ee">
<channel name="ee" stack="tcp"/>
</channels>
<stacks>
<stack name="udp">
<transport type="UDP" socket-binding="jgroups-udp"/>
<protocol type="PING"/>
<protocol type="MERGE3"/>
<protocol type="FD_SOCK" socket-binding="jgroups-udp-fd"/>
<protocol type="FD_ALL"/>
<protocol type="VERIFY_SUSPECT"/>
<protocol type="pbcast.NAKACK2"/>
<protocol type="UNICAST3"/>
<protocol type="pbcast.STABLE"/>
<protocol type="pbcast.GMS"/>
<protocol type="UFC"/>
<protocol type="MFC"/>
<protocol type="FRAG2"/>
</stack>
<stack name="tcp">
<transport type="TCP" socket-binding="jgroups-tcp">
<property name="external_addr">200.129.4.189</property>
</transport>
<protocol type="S3_PING">
<property name="access_key">AAAAAAAAAAAAAA</property>
<property name="secret_access_key">BBBBBBBBBBBBBB</property>
<property name="location">CCCCCCCCCCCCCCCCCCCC</property>
</protocol>
<protocol type="MERGE3"/>
<protocol type="FD_SOCK" socket-binding="jgroups-tcp-fd">
<property name="external_addr">200.129.4.189</property>
</protocol>
<protocol type="FD"/>
<protocol type="VERIFY_SUSPECT"/>
<protocol type="pbcast.NAKACK2"/>
<protocol type="UNICAST3"/>
<protocol type="pbcast.STABLE"/>
<protocol type="pbcast.GMS"/>
<protocol type="MFC"/>
<protocol type="FRAG2"/>
</stack>
</stacks>
</subsystem>
<socket-binding name="jgroups-tcp" interface="public" port="7600"/>
<socket-binding name="jgroups-tcp-fd" interface="public" port="57600"/>
And we start the server using the below ($INTERNAL_HOST_IP is the container internal IP address):
- standalone.sh -c=standalone-ha.xml -b=$INTERNAL_HOST_IP -bmanagement=$INTERNAL_HOST_IP -bprivate=$INTERNAL_HOST_IP
Any help will be appreciated.
Thanks,
Haim.