3 Replies Latest reply on Mar 27, 2018 12:08 PM by max_lan

Slave failed to authenticate to Master: ERROR JBREM000200 (Wildfly 9)

bayuanggoro Feb 14, 2017 2:37 AM

Hi guys, I get some problem when authorizing Slave host controller to Master. I've been looking around Internet for the solution but end up there isn't appropriate way. In the way of making Wildfly cluster I follow this tutorial https://docs.jboss.org/author/display/WFLY9/WildFly+9+Cluster+Howto.

I run two Wildfly 9.0.2 Final above two separate instances of AWS EC2 (RHEL 7.3). With that environment each RHELs have two IP Addresses: private and public.

Master private IP Address : 172.31.20.13

Master public IP Address : 54.254.224.244

Slave private IP Address : 172.31.18.88

Slave public IP Address : 54.255.192.29

I open necessary ports in AWS EC2 Network Security group so those instances can communicate.

Here the configuration for Master and Slave.

MASTER configuration ({JBOSS_HOME}/domain/configuration/host.xml)

<?xml version='1.0' encoding='UTF-8'?>


<host name="master" xmlns="urn:jboss:domain:3.0">
    <extensions>
        <extension module="org.jboss.as.jmx"/>
    </extensions>


    <management>
        <security-realms>
            <security-realm name="ManagementRealm">
                <authentication>
                    <local default-user="$local" skip-group-loading="true"/>
                    <properties path="mgmt-users.properties" relative-to="jboss.domain.config.dir"/>
                </authentication>
                <authorization map-groups-to-roles="false">
                    <properties path="mgmt-groups.properties" relative-to="jboss.domain.config.dir"/>
                </authorization>
            </security-realm>
            <security-realm name="ApplicationRealm">
                <authentication>
                    <local default-user="$local" allowed-users="*" skip-group-loading="true"/>
                    <properties path="application-users.properties" relative-to="jboss.domain.config.dir"/>
                </authentication>
                <authorization>
                    <properties path="application-roles.properties" relative-to="jboss.domain.config.dir"/>
                </authorization>
            </security-realm>
        </security-realms>
        <audit-log>
            <formatters>
                <json-formatter name="json-formatter"/>
            </formatters>
            <handlers>
                <file-handler name="host-file" formatter="json-formatter" relative-to="jboss.domain.data.dir" path="audit-log.log"/>
                <file-handler name="server-file" formatter="json-formatter" relative-to="jboss.server.data.dir" path="audit-log.log"/>
            </handlers>
            <logger log-boot="true" log-read-only="false" enabled="false">
                <handlers>
                    <handler name="host-file"/>
                </handlers>
            </logger>
            <server-logger log-boot="true" log-read-only="false" enabled="false">
                <handlers>
                    <handler name="server-file"/>
                </handlers>
            </server-logger>
        </audit-log>
        <management-interfaces>
            <native-interface security-realm="ManagementRealm">
                <socket interface="management" port="${jboss.management.native.port:9999}"/>
            </native-interface>
            <http-interface security-realm="ManagementRealm" http-upgrade-enabled="true">
                <socket interface="management" port="${jboss.management.http.port:9990}"/>
            </http-interface>
        </management-interfaces>
    </management>


    <domain-controller>
        <local/>
        <!-- Alternative remote domain controller configuration with a host and port -->
        <!-- <remote protocol="remote" host="${jboss.domain.master.address}" port="${jboss.domain.master.port:9999}" security-realm="ManagementRealm"/> -->
    </domain-controller>


    <interfaces>
        <interface name="management">
            <inet-address value="${jboss.bind.address.management:172.31.20.13}"/>
        </interface>
        <interface name="public">
            <inet-address value="${jboss.bind.address:172.31.20.13}"/>
        </interface>
        <interface name="unsecure">
            <!-- Used for IIOP sockets in the standard configuration.
                 To secure JacORB you need to setup SSL -->
            <inet-address value="${jboss.bind.address.unsecure:172.31.20.13}"/>
        </interface>
    </interfaces>


    <jvms>
        <jvm name="default">
            <heap size="64m" max-size="256m"/>
            <permgen size="256m" max-size="256m"/>
            <jvm-options>
                <option value="-server"/>
            </jvm-options>
        </jvm>
    </jvms>


    <servers>
        <server name="server-one" group="main-server-group">
            <!-- Remote JPDA debugging for a specific server
            <jvm name="default">
              <jvm-options>
                <option value="-agentlib:jdwp=transport=dt_socket,address=8787,server=y,suspend=n"/>
              </jvm-options>
           </jvm>
           -->
        </server>
        <server name="server-two" group="main-server-group" auto-start="true">
            <!-- server-two avoids port conflicts by incrementing the ports in
                 the default socket-group declared in the server-group -->
            <socket-bindings port-offset="150"/>
        </server>
        <server name="server-three" group="other-server-group" auto-start="true">
            <!-- server-three avoids port conflicts by incrementing the ports in
                 the default socket-group declared in the server-group -->
            <socket-bindings port-offset="250"/>
        </server>
    </servers>
    
    <profile>
        <subsystem xmlns="urn:jboss:domain:jmx:1.3">
            <expose-resolved-model/>
            <expose-expression-model/>
            <remoting-connector/>
        </subsystem>
    </profile>
</host>

SLAVE configuration ({JBOSS_HOME}/domain/configuration/host.xml)

<?xml version='1.0' encoding='UTF-8'?>


<host name="slave" xmlns="urn:jboss:domain:3.0">
    <extensions>
        <extension module="org.jboss.as.jmx"/>
    </extensions>


    <management>
        <security-realms>
            <security-realm name="ManagementRealm">
                <authentication>
  <local default-user="$local" skip-group-loading="true" />      
  <properties path="mgmt-users.properties" relative-to="jboss.domain.config.dir"/>
                </authentication>
  <server-identities>
            <secret value="UEBzc3cwcmQ" />
  <!-- <ssl>
  <keystore path="server.keystore" relative-to="jboss.domain.config.dir" keystore-password="jbossas" alias="jboss" key-password="jbossas" />
  </ssl> -->
  </server-identities>
            </security-realm>
            <security-realm name="ApplicationRealm">
                <authentication>
                    <local default-user="$local" allowed-users="*" skip-group-loading="true"/>
                    <properties path="application-users.properties" relative-to="jboss.domain.config.dir"/>
                </authentication>
                <authorization>
                    <properties path="application-roles.properties" relative-to="jboss.domain.config.dir"/>
                </authorization>
            </security-realm>
        </security-realms>
        <audit-log>
            <formatters>
                <json-formatter name="json-formatter"/>
            </formatters>
            <handlers>
                <file-handler name="host-file" formatter="json-formatter" relative-to="jboss.domain.data.dir" path="audit-log.log"/>
                <file-handler name="server-file" formatter="json-formatter" relative-to="jboss.server.data.dir" path="audit-log.log"/>
            </handlers>
            <logger log-boot="true" log-read-only="false" enabled="false">
                <handlers>
                    <handler name="host-file"/>
                </handlers>
            </logger>
            <server-logger log-boot="true" log-read-only="false" enabled="false">
                <handlers>
                    <handler name="server-file"/>
                </handlers>
            </server-logger>
        </audit-log>
        <management-interfaces>
            <native-interface security-realm="ManagementRealm">
                <socket interface="management" port="${jboss.management.native.port:9999}"/>
            </native-interface>
            <http-interface security-realm="ManagementRealm" http-upgrade-enabled="true">
                <socket interface="management" port="${jboss.management.http.port:9990}"/>
            </http-interface>
        </management-interfaces>
    </management>


    <domain-controller>
        
        <!-- Alternative remote domain controller configuration with a host and port -->
        <remote host="172.31.20.13" port="9999" security-realm="ManagementRealm"/>
  <!-- <remote security-realm="ManagementRealm">
  <discovery-options>
  <static-discovery name="master-native" protocol="remote" host="54.254.224.244" port="9999"/>
  <static-discovery name="master-https" protocol="https-remoting" host="54.254.224.244" port="9993"/>
  <static-discovery name="master-http" protocol="http-remoting" host="54.254.224.244" port="9990"/>
  </discovery-options> -->
    </domain-controller>


    <interfaces>
        <interface name="management">
            <inet-address value="${jboss.bind.address.management:172.31.18.88}"/>
        </interface>
        <interface name="public">
            <inet-address value="${jboss.bind.address:172.31.18.88}"/>
        </interface>
        <interface name="unsecure">
            <!-- Used for IIOP sockets in the standard configuration.
                 To secure JacORB you need to setup SSL -->
            <inet-address value="${jboss.bind.address.unsecure:172.31.18.88}"/>
        </interface>
    </interfaces>


    <jvms>
        <jvm name="default">
            <heap size="64m" max-size="256m"/>
            <permgen size="256m" max-size="256m"/>
            <jvm-options>
                <option value="-server"/>
            </jvm-options>
        </jvm>
    </jvms>


    <servers>
        <server name="server-one" group="main-server-group">
            <!-- Remote JPDA debugging for a specific server
            <jvm name="default">
              <jvm-options>
                <option value="-agentlib:jdwp=transport=dt_socket,address=8787,server=y,suspend=n"/>
              </jvm-options>
           </jvm>
           -->
        </server>
        <server name="server-two" group="main-server-group" auto-start="true">
            <!-- server-two avoids port conflicts by incrementing the ports in
                 the default socket-group declared in the server-group -->
            <socket-bindings port-offset="150"/>
        </server>
        <server name="server-three" group="other-server-group" auto-start="true">
            <!-- server-three avoids port conflicts by incrementing the ports in
                 the default socket-group declared in the server-group -->
            <socket-bindings port-offset="250"/>
        </server>
    </servers>
    
    <profile>
        <subsystem xmlns="urn:jboss:domain:jmx:1.3">
            <expose-resolved-model/>
            <expose-expression-model/>
            <remoting-connector/>
        </subsystem>
    </profile>
</host>

SLAVE host controller log ({JBOSS_HOME}/domain/log/host-controller.log)

2017-02-13 08:14:14,859 INFO  [org.jboss.modules] (main) JBoss Modules version 1.4.3.Final
2017-02-13 08:14:15,171 INFO  [org.jboss.msc] (main) JBoss MSC version 1.2.6.Final
2017-02-13 08:14:15,222 INFO  [org.jboss.as] (MSC service thread 1-2) WFLYSRV0049: WildFly Full 9.0.2.Final (WildFly Core 1.0.2.Final) starting
2017-02-13 08:14:15,950 INFO  [org.xnio] (MSC service thread 1-2) XNIO version 3.3.1.Final
2017-02-13 08:14:15,958 INFO  [org.xnio.nio] (MSC service thread 1-2) XNIO NIO Implementation Version 3.3.1.Final
2017-02-13 08:14:15,973 INFO  [org.jboss.as.host.controller] (Controller Boot Thread) WFLYHC0003: Creating http management service using network interface (management) port (9990) securePort (-1)
2017-02-13 08:14:16,050 INFO  [org.jboss.remoting] (MSC service thread 1-2) JBoss Remoting version 4.0.9.Final
2017-02-13 08:14:16,167 INFO  [org.jboss.as.remoting] (MSC service thread 1-2) WFLYRMT0001: Listening on 172.31.18.88:9999
2017-02-13 08:14:16,920 ERROR [org.jboss.remoting.remote.connection] (Remoting "slave:MANAGEMENT" I/O-1) JBREM000200: Remote connection failed: javax.security.sasl.SaslException: Authentication failed: all available authentication mechanisms failed:
   DIGEST-MD5: Server rejected authentication
2017-02-13 08:14:16,923 WARN  [org.jboss.as.host.controller] (Controller Boot Thread) WFLYHC0001: Could not connect to remote domain controller remote://172.31.20.13:9999 -- java.lang.IllegalStateException: WFLYHC0043: Unable to connect due to authentication failure.
2017-02-13 08:14:16,924 WARN  [org.jboss.as.host.controller] (Controller Boot Thread) WFLYHC0147: No domain controller discovery options remain.
2017-02-13 08:14:16,924 ERROR [org.jboss.as.host.controller] (Controller Boot Thread) WFLYHC0002: Could not connect to master. Aborting. Error was: java.lang.IllegalStateException: WFLYHC0120: Tried all domain controller discovery option(s) but unable to connect
2017-02-13 08:14:16,948 INFO  [org.jboss.as] (MSC service thread 1-2) WFLYSRV0050: WildFly Full 9.0.2.Final (WildFly Core 1.0.2.Final) stopped in 18ms

SLAVE process controller log ({JBOSS_HOME}/domain/log/process-controller.log)

2017-02-13 08:14:13,941 INFO  [org.jboss.modules] (main) JBoss Modules version 1.4.3.Final
2017-02-13 08:14:14,080 INFO  [org.jboss.as.process.Host Controller.status] (main) WFLYPC0018: Starting process 'Host Controller'
2017-02-13 08:14:17,267 INFO  [org.jboss.as.process.Host Controller.status] (reaper for Host Controller) WFLYPC0011: Process 'Host Controller' finished with an exit status of 99
2017-02-13 08:14:17,269 INFO  [org.jboss.as.process] (Thread-8) WFLYPC0017: Shutting down process controller
2017-02-13 08:14:17,270 INFO  [org.jboss.as.process] (Thread-8) WFLYPC0016: All processes finished; exiting

Really appreciate any feedbacks from you guys. Thank you in advance.

1. Re: Slave failed to authenticate to Master: ERROR JBREM000200 (Wildfly 9)

wesssel May 9, 2017 4:21 AM (in response to bayuanggoro)

I am encountering the same issue with WildFly 10.0.1.0. It works when connecting from the same host, but not from a remote host. Might this have something to do with stricter JDK security policies?
Actions
2. Re: Slave failed to authenticate to Master: ERROR JBREM000200 (Wildfly 9)

kmarc Jun 27, 2017 4:22 AM (in response to wesssel)

I'm also facing this very same problem. Did you find any solution?
Actions
3. Re: Slave failed to authenticate to Master: ERROR JBREM000200 (Wildfly 9)

max_lan Mar 27, 2018 12:08 PM (in response to bayuanggoro)
I was having a very similar problem. I could start a slave locally but not remotely.
I experimented with using the jboss-cli.sh to connect from the slave to the master and could login as admin with my supplied password.
Eventually I found that I could change the name of the user making the connection from $local to "admin"
<domain-controller> <remote username="admin" security-realm="ManagementRealm"> <discovery-options> <static-discovery name="primary" protocol="${jboss.domain.master.protocol:remote}" host="${jboss.domain.master.address:127.0.0.1}" port="${jboss.domain.master.port:9999}"/> </discovery-options> </remote> </domain-controller>

And it all connects fine. Even though, the $local username is supposed to mean that the remote server uses it's local default username, if I understand the docs. I'm not even convinced it needs to be set.
Apologies for being almost a year late but hopefully this will help someone else in future.
Actions

Go to original post