1 2 Previous Next 22 Replies Latest reply on Sep 12, 2018 7:11 AM by George Labuschagne

    Cannot connect to remote WildFly management console.

    George Labuschagne Newbie

      Hi all

       

      I cannot connect to a remote running WildFly management console. I can however open the console if logged in locally or view the management web site.

       

      Here is my startup parameters:

       

      ./standalone.sh -b=0.0.0.0 -bmanagement=0.0.0.0 --debug

       

      I also tried with:

       

      ./standalone.sh -b=192.168.0.80 -bmanagement=192.168.0.80 --debug

      The last lines of my log says:

       

      13:17:37,393 INFO  [org.jboss.as.server] (Controller Boot Thread) WFLYSRV0212: Resuming server
      13:17:37,397 INFO  [org.jboss.as] (Controller Boot Thread) WFLYSRV0060: Http management interface listening on http://192.168.0.80:9990/management
      13:17:37,398 INFO  [org.jboss.as] (Controller Boot Thread) WFLYSRV0051: Admin console listening on http://192.168.0.80:9990
      13:17:37,398 INFO  [org.jboss.as] (Controller Boot Thread) WFLYSRV0025: WildFly Full 14.0.0.Final (WildFly Core 6.0.1.Final) started in 48453ms - Started 1784 of 1957 services (331 services are lazy, passive or on-demand)

       

      The command I use to attempt to connect from a local WildFly 14 - same as server - bin directory on my local machine is:

       

      ./jboss-cli.sh --connect --controller=remote+http://192.168.0.80:9990

       

      All required ports are opened on the server (else the web site would not work either).

       

      I get a request to authenticate against:

       

      ./jboss-cli.sh jboss-cli.sh --connect --controller=remote+http://192.168.0.80:9990
      Picked up _JAVA_OPTIONS: -Xms512m -Xmx2g
      Authenticating against security realm: ManagementRealm
      Username: myuser
      Password: 
      Failed to connect to the controller: The controller is not available at 192.168.0.80:9990: java.net.ConnectException: WFLYPRT0023: Could not connect to remote+http://192.168.0.80:9990. The connection timed out: WFLYPRT0023: Could not connect to remote+http://192.168.0.80:9990. The connection timed out

       

      The error I get is:

       

      Failed to connect to the controller: The controller is not available at 192.168.0.80:9990: java.net.ConnectException: WFLYPRT0023: Could not connect to remote+http://192.168.0.80:9990. The connection timed out: WFLYPRT0023: Could not connect to remote+http://192.168.0.80:9990. The connection timed out

       

      I need to be able to connect to this remote management console for remote debugging via IntelliJ.

       

      This is nmap reports, first one from the server running WildFly, second from devbox trying to connect to WildFly:

       

      Host is up (0.0021s latency).
      Not shown: 990 closed ports
      PORT     STATE SERVICE
      22/tcp   open  ssh
      25/tcp   open  smtp
      80/tcp   open  http
      111/tcp  open  rpcbind
      443/tcp  open  https
      873/tcp  open  rsync
      3306/tcp open  mysql
      5432/tcp open  postgresql
      8080/tcp open  http-proxy
      8443/tcp open  https-alt
      
      
      Nmap done: 1 IP address (1 host up) scanned in 0.13 seconds

       

      Host is up (0.0040s latency).
      Not shown: 994 filtered ports
      PORT     STATE  SERVICE
      22/tcp   open   ssh
      80/tcp   open   http
      873/tcp  open   rsync
      8009/tcp closed ajp13
      8080/tcp open   http-proxy
      8443/tcp open   https-alt
      
      
      Nmap done: 1 IP address (1 host up) scanned in 16.17 seconds

       

      This also appears in the server log when trying to remotely connect:

       

      WARN  [org.jboss.as.domain.management.security] (default I/O-6) WFLYDM0113: Generated self signed certificate at /home/edsadmin/devbuild/wildfly-14.0.0.Final/standalone/configuration/application.keystore. Please note that self signed certificates are not secure, and should only be used for testing purposes. Do not use this self signed certificate in production.
      SHA-1 fingerprint of the generated key is 37:a5:dd:3a:1b:0e:1b:ba:48:89:23:be:e6:d6:21:14:61:85:18:90
      SHA-256 fingerprint of the generated key is 47:a2:5b:e4:73:d0:54:62:90:8b:a2:0f:e1:03:f2:c7:02:c6:fe:04:6f:ab:fe:85:31:35:99:31:11:b4:3c:95

       

      OS is CentOS in both cases.

       

      Any ideas please?

        • 1. Re: Cannot connect to remote WildFly management console.
          Claudio Miranda Expert

          Can you try the status with netcat ? NMAP requires you to inform the 9990 port.

           

          $ nc -zv -w2 192.168.25.220 9990

          Ncat: Version 7.60 ( https://nmap.org/ncat )

          Ncat: Connected to 192.168.25.220:9990.

          Ncat: 0 bytes sent, 0 bytes received in 0.01 seconds.



          $ sudo nmap -sT 192.168.25.220 -p9990

           

          Starting Nmap 7.60 ( https://nmap.org ) at 2018-09-06 12:05 -03

          Nmap scan report for uranus (192.168.25.220)

          Host is up (0.000046s latency).

           

          PORT     STATE SERVICE

          9990/tcp open  osm-appsrvr

           

          • 2. Re: Cannot connect to remote WildFly management console.
            Martin Choma Master

            When connecting locally JBOSS_LOCAL authentication is used. When connecting remotely you have to have user myuser in ManagementRealm. You can add that by bin/add-user.sh for example. I don't see 9990 port open in your list.

            • 3. Re: Cannot connect to remote WildFly management console.
              George Labuschagne Newbie

              Good day Claudio claudio4j

               

              Thanks for the response.

               

              The output from the two commands ran on the client are:

               

              [georgel@devbox bin]$ nc -zv -w2 192.168.0.80 9990
              Ncat: Version 7.50 ( https://nmap.org/ncat )
              Ncat: Connected to 192.168.0.80:9990.
              Ncat: 0 bytes sent, 0 bytes received in 0.01 seconds.

               

              and

               

              [georgel@devbox bin]$ sudo nmap -sT 192.168.25.220 -p9990
              [sudo] password for georgel:
              
              
              Starting Nmap 6.40 ( http://nmap.org ) at 2018-09-07 07:06 SAST
              Note: Host seems down. If it is really up, but blocking our ping probes, try -Pn
              Nmap done: 1 IP address (0 hosts up) scanned in 0.46 seconds
              [georgel@devbox bin]$

               

              If I use the -Pn parameter nmap is able to connect, is this causing the timeout perhaps? If so how to resolve?

               

              [georgel@devbox bin]$ sudo nmap -sT -Pn 192.168.25.220 -p9990
              [sudo] password for georgel: 
              
              
              Starting Nmap 6.40 ( http://nmap.org ) at 2018-09-07 07:12 SAST
              Nmap scan report for 192.168.25.220
              Host is up.
              PORT     STATE    SERVICE
              9990/tcp filtered osm-appsrvr
              
              
              Nmap done: 1 IP address (1 host up) scanned in 13.07 seconds
              [georgel@devbox bin]$ 

               

              So it seems netcat is able to connect to the port but nmap can't ping the port. I can ping the machine however:

               

              [georgel@devbox bin]$ ping 192.168.0.80 -c 5
              PING 192.168.0.80 (192.168.0.80) 56(84) bytes of data.
              64 bytes from 192.168.0.80: icmp_seq=1 ttl=63 time=2.03 ms
              64 bytes from 192.168.0.80: icmp_seq=2 ttl=64 time=1.68 ms
              64 bytes from 192.168.0.80: icmp_seq=3 ttl=64 time=1.71 ms
              64 bytes from 192.168.0.80: icmp_seq=4 ttl=64 time=4.27 ms
              64 bytes from 192.168.0.80: icmp_seq=5 ttl=64 time=1.70 ms
              
              
              --- 192.168.0.80 ping statistics ---
              5 packets transmitted, 5 received, 0% packet loss, time 4009ms
              rtt min/avg/max/mdev = 1.686/2.282/4.274/1.006 ms
              [georgel@devbox bin]$ 

               

              netstat on the server shows that the server is listening:

               

              [edsadmin@bugzilla ~]$ netstat -at
              Active Internet connections (servers and established)
              Proto Recv-Q Send-Q Local Address           Foreign Address         State      
              tcp        0      0 0.0.0.0:smtp            0.0.0.0:*               LISTEN     
              tcp        0      0 0.0.0.0:pcsync-https    0.0.0.0:*               LISTEN     
              tcp        0      0 0.0.0.0:osm-appsrvr     0.0.0.0:*               LISTEN     
              tcp        0      0 0.0.0.0:rsync           0.0.0.0:*               LISTEN     
              tcp        0      0 0.0.0.0:mysql           0.0.0.0:*               LISTEN     
              tcp        0      0 0.0.0.0:sunrpc          0.0.0.0:*               LISTEN     
              tcp        0      0 0.0.0.0:webcache        0.0.0.0:*               LISTEN     
              tcp        0      0 0.0.0.0:ssh             0.0.0.0:*               LISTEN     
              tcp        0      0 0.0.0.0:postgres        0.0.0.0:*               LISTEN     
              tcp        0      0 bugzilla.headoffi:39116 webgateway.hea:webcache TIME_WAIT  
              tcp        0      0 bugzilla.headoffice:ssh 192.168.41.111:49466    ESTABLISHED
              tcp        0      0 localhost:postgres      localhost:46840         ESTABLISHED
              tcp        0      0 bugzilla.headoffice:ssh 192.168.41.111:49472    ESTABLISHED
              tcp        0      0 localhost:46840         localhost:postgres      ESTABLISHED
              tcp6       0      0 [::]:https              [::]:*                  LISTEN     
              tcp6       0      0 [::]:rsync              [::]:*                  LISTEN     
              tcp6       0      0 [::]:sunrpc             [::]:*                  LISTEN     
              tcp6       0      0 [::]:http               [::]:*                  LISTEN     
              tcp6       0      0 [::]:ssh                [::]:*                  LISTEN     
              tcp6       0      0 [::]:postgres           [::]:*                  LISTEN   
              • 4. Re: Cannot connect to remote WildFly management console.
                George Labuschagne Newbie

                Good day Martin mchoma

                 

                I already have a user with a similar username on the server and client, both as ManagementRealm users.

                 

                I created on both now as well a user literately called 'myuser', added it to the ManagementRealm and to the 'public' group on both machine.

                 

                However the timeout after I enter the password still persist:

                 

                [georgel@devbox bin]$ ./add-user.sh
                Picked up _JAVA_OPTIONS: -Xms512m -Xmx2g
                
                
                What type of user do you wish to add?
                a) Management User (mgmt-users.properties)
                b) Application User (application-users.properties)
                (a): a
                
                
                Enter the details of the new user to add.
                Using realm 'ManagementRealm' as discovered from the existing property files.
                Username : myuser
                Password recommendations are listed below. To modify these restrictions edit the add-user.properties configuration file.
                - The password should be different from the username
                - The password should not be one of the following restricted values {root, admin, administrator}
                - The password should contain at least 8 characters, 1 alphabetic character(s), 1 digit(s), 1 non-alphanumeric symbol(s)
                Password :
                Re-enter Password :
                What groups do you want this user to belong to? (Please enter a comma separated list, or leave blank for none)[  ]: public
                About to add user 'myuser' for realm 'ManagementRealm'
                Is this correct yes/no? yes
                Added user 'myuser' to file '/home/georgel/opt/wildfly-14.0.0.Final/standalone/configuration/mgmt-users.properties'
                Added user 'myuser' to file '/home/georgel/opt/wildfly-14.0.0.Final/domain/configuration/mgmt-users.properties'
                Added user 'myuser' with groups public to file '/home/georgel/opt/wildfly-14.0.0.Final/standalone/configuration/mgmt-groups.properties'
                Added user 'myuser' with groups public to file '/home/georgel/opt/wildfly-14.0.0.Final/domain/configuration/mgmt-groups.properties'
                Is this new user going to be used for one AS process to connect to another AS process?
                e.g. for a slave host controller connecting to the master or for a Remoting connection for server to server EJB calls.
                yes/no? no

                 

                and then

                 

                [georgel@devbox bin]$ ./jboss-cli.sh --connect --controller=192.168.0.80:9990
                Picked up _JAVA_OPTIONS: -Xms512m -Xmx2g
                Authenticating against security realm: ManagementRealm
                Username: myuser
                Password:
                Failed to connect to the controller: The controller is not available at 192.168.0.80:9990: java.net.ConnectException: WFLYPRT0023: Could not connect to remote+http://192.168.0.80:9990. The connection timed out: WFLYPRT0023: Could not connect to remote+http://192.168.0.80:9990. The connection timed out
                [georgel@devbox bin]$
                • 5. Re: Cannot connect to remote WildFly management console.
                  George Labuschagne Newbie

                  Some additional info

                   

                  When no username is supplied the following appears:

                   

                  [georgel@devbox bin]$ ./jboss-cli.sh --connect --controller=192.168.0.80:9990
                  Authenticating against security realm: ManagementRealm
                  Username: 
                  Failed to connect to the controller: Unable to authenticate against controller at 192.168.0.80:9990: Authentication failed: all available authentication mechanisms failed:
                     DIGEST-MD5: javax.security.sasl.SaslException: ELY05053: Callback handler failed for unknown reason [Caused by javax.security.sasl.SaslException: No username supplied.]
                  [georgel@devbox bin]$ 

                   

                  When an incorrect password is supplied:

                   

                  georgel@devbox bin]$ ./jboss-cli.sh --connect --controller=192.168.0.80:9990
                  Authenticating against security realm: ManagementRealm
                  Username: edsadmin
                  Password: 
                  Failed to connect to the controller: Unable to authenticate against controller at 192.168.0.80:9990: Authentication failed: all available authentication mechanisms failed:
                     DIGEST-MD5: javax.security.sasl.SaslException: DIGEST-MD5: Server rejected authentication
                  [georgel@devbox bin]$ 
                  • 6. Re: Cannot connect to remote WildFly management console.
                    Prem Yadav Newbie

                    hi is --controller=192.168.0.80:9990  is a domain controller ?

                    • 7. Re: Cannot connect to remote WildFly management console.
                      George Labuschagne Newbie

                      Good day Prem yadavprem

                       

                      No it is not. It is another standalone installation in a testing environment that I want to connect to for debugging purposes via IntelliJ

                      • 8. Re: Cannot connect to remote WildFly management console.
                        Wolfgang Mayer Master

                        This seems to be an issue with your firewall.

                        Did you check this also with Wireshark?

                        • 9. Re: Cannot connect to remote WildFly management console.
                          George Labuschagne Newbie

                          Good day Wolfgang mayerw01

                           

                          Thanks for the reply.

                           

                          I disabled the firewalld and SELINUX on both machines:

                           

                          [root@devbox ~]# systemctl status firewalld
                          ● firewalld.service - firewalld - dynamic firewall daemon
                            Loaded: loaded (/usr/lib/systemd/system/firewalld.service; disabled; vendor preset: enabled)
                            Active: inactive (dead)
                              Docs: man:firewalld(1)
                          [root@devbox ~]#

                           

                          and

                           

                          [root@devbox ~]# sestatus 
                          SELinux status:                 disabled
                          [root@devbox ~]#

                           

                          still get the timeout after authentication.

                          • 10. Re: Cannot connect to remote WildFly management console.
                            Wolfgang Mayer Master

                            Did you check this also with a network analyzer like Wireshark?

                            • 11. Re: Cannot connect to remote WildFly management console.
                              Claudio Miranda Expert

                              Enable security logging on server as

                              <logger category="org.wildfly.security" use-parent-handlers="true">

                                  <level name="ALL"/>

                              </logger>

                              And do a tail -f on server.log

                               

                              Also, related to nmap, the port state is filtered, which means nmap cannot determine whether the port is open because packet filtering prevents its probes from reaching the port. You should double check if there are firewall rules on 9990 port.

                               

                              Port Scanning Basics | Nmap Network Scanning

                              • 12. Re: Cannot connect to remote WildFly management console.
                                George Labuschagne Newbie

                                Good day claudio4j

                                 

                                I completely disabled firewalld and SELINUX on both machines.

                                 

                                I still get the timeout after attempting to authenticate.

                                 

                                Below is the output from the log as requested. I definitely can reach the server based on this log file, but something then goes wrong with the authentication it seems.

                                 

                                [edsadmin@bugzilla log]$ tail -f on server.log
                                tail: cannot open ‘on’ for reading: No such file or directory
                                ==> server.log <==
                                2018-09-12 08:02:53,629 INFO  [org.jboss.as.server] (Controller Boot Thread) WFLYSRV0010: Deployed "postgresql-42.2.5.jar" (runtime-name : "postgresql-42.2.5.jar")
                                2018-09-12 08:02:53,635 INFO  [org.jboss.as.server] (ServerService Thread Pool -- 42) WFLYSRV0010: Deployed "edsnext.war" (runtime-name : "edsnext.war")
                                2018-09-12 08:02:53,819 INFO  [org.jboss.as.server] (Controller Boot Thread) WFLYSRV0212: Resuming server
                                2018-09-12 08:02:53,822 INFO  [org.jboss.as] (Controller Boot Thread) WFLYSRV0060: Http management interface listening on http://0.0.0.0:9990/management
                                2018-09-12 08:02:53,823 INFO  [org.jboss.as] (Controller Boot Thread) WFLYSRV0051: Admin console listening on http://0.0.0.0:9990
                                2018-09-12 08:02:53,823 INFO  [org.jboss.as] (Controller Boot Thread) WFLYSRV0025: WildFly Full 14.0.0.Final (WildFly Core 6.0.1.Final) started in 29326ms - Started 1782 of 1956 services (332 services are lazy, passive or on-demand)
                                2018-09-12 08:04:34,509 TRACE [org.wildfly.security] (management I/O-2) Handling MechanismInformationCallback type='SASL' name='DIGEST-MD5' host-name='bugzilla.headoffice.megchem.co.za' protocol='remote'
                                2018-09-12 08:04:34,511 TRACE [org.wildfly.security] (management I/O-2) Handling AvailableRealmsCallback: realms = [ManagementRealm]
                                2018-09-12 08:04:34,520 TRACE [org.wildfly.security] (management I/O-2) Creating SaslServer [org.wildfly.security.sasl.digest.DigestSaslServer@574f31a4] for mechanism [DIGEST-MD5] and protocol [remote]
                                2018-09-12 08:04:34,523 TRACE [org.wildfly.security] (management I/O-2) Created SaslServer [org.wildfly.security.sasl.util.SecurityIdentitySaslServerFactory$1@27646241->org.wildfly.security.sasl.util.AuthenticationTimeoutSaslServerFactory$DelegatingTimeoutSaslServer@2c4151b3->org.wildfly.security.sasl.util.AuthenticationCompleteCallbackSaslServerFactory$1@58b7e66d->org.wildfly.security.sasl.digest.DigestSaslServer@574f31a4] for mechanism [DIGEST-MD5]
                                2018-09-12 08:05:07,204 TRACE [org.wildfly.security] (management task-1) Handling RealmCallback: selected = [ManagementRealm]
                                2018-09-12 08:05:07,205 TRACE [org.wildfly.security] (management task-1) Handling NameCallback: authenticationName = edsadmin
                                2018-09-12 08:05:07,210 TRACE [org.wildfly.security] (management task-1) Principal assigning: [edsadmin], pre-realm rewritten: [edsadmin@ManagementRealm], realm name: [DIGEST], post-realm rewritten: [edsadmin@ManagementRealm], realm rewritten: [edsadmin@ManagementRealm]
                                2018-09-12 08:05:07,228 TRACE [org.wildfly.security] (management task-1) Handling CredentialCallback: obtained credential for correct realm "ManagementRealm"
                                2018-09-12 08:05:07,228 TRACE [org.wildfly.security] (management task-1) Handling CredentialCallback: obtained credential: org.wildfly.security.credential.PasswordCredential@bebfb5b9
                                2018-09-12 08:05:07,230 TRACE [org.wildfly.security] (management task-1) Role mapping: principal [edsadmin@ManagementRealm] -> decoded roles [] -> realm mapped roles [] -> domain mapped roles []
                                2018-09-12 08:05:07,231 TRACE [org.wildfly.security] (management task-1) Authorizing principal edsadmin.
                                2018-09-12 08:05:07,232 TRACE [org.wildfly.security] (management task-1) Authorizing against the following attributes: [] => []
                                2018-09-12 08:05:07,233 TRACE [org.wildfly.security] (management task-1) Permission mapping: identity [edsadmin@ManagementRealm] with roles [] implies ("org.wildfly.security.auth.permission.LoginPermission" "") = true
                                2018-09-12 08:05:07,234 TRACE [org.wildfly.security] (management task-1) Authorization succeed
                                2018-09-12 08:05:07,236 TRACE [org.wildfly.security] (management task-1) RunAs authorization succeed - the same identity
                                2018-09-12 08:05:07,236 TRACE [org.wildfly.security] (management task-1) Handling AuthorizeCallback: authenticationID = edsadmin  authorizationID = edsadmin  authorized = true
                                2018-09-12 08:05:07,237 TRACE [org.wildfly.security] (management task-1) Handling AuthenticationCompleteCallback: succeed
                                2018-09-12 08:05:07,237 TRACE [org.wildfly.security] (management task-1) Handling SecurityIdentityCallback: identity = SecurityIdentity{principal=edsadmin@ManagementRealm, securityDomain=org.wildfly.security.auth.server.SecurityDomain@17b8d842, authorizationIdentity=EMPTY, realmInfo=RealmInfo{name='DIGEST', securityRealm=org.jboss.as.domain.management.security.SecurityRealmService$SharedStateSecurityRealm@20c1cb10}, creationTime=2018-09-12T06:05:07.230Z}
                                
                                
                                • 13. Re: Cannot connect to remote WildFly management console.
                                  Prem Yadav Newbie

                                  did you entered the base64 (password encoded) into host.xml of the machine from which you want to connect

                                  • 14. Re: Cannot connect to remote WildFly management console.
                                    Darran Lofthouse Master

                                    The example being discussed here is just using standalone.xml so the host.xml is not used in this situation.

                                    1 2 Previous Next