Accessing JMX with jconsole over SSL on EAP 7.0 or Wildfly 10+

[dgilbert]

I am trying to setup secure jmx connection with ssl in DOMAIN mode. I have been able to get it to work in standalone mode (See my steps below). However in domain mode I am missing something. I was able to find this knowledge base article for EAP 6 which worked fine for me in both standalone and domain mode. In EAP 7 there is no more remoting port 4447 that existed in EAP 6. I can connect to my servers over JMX in non-ssl mode with no issues.

Accessing JMX with jconsole over SSL on EAP 6 - Red Hat Customer Portal

This is what I did so far for both standalone and domain mode:

This is being done on a dev setup so exact production quality practices is not an issue

right now.

In standalone mode.

1) Created keystore

keytool -genkey -alias jboss -keyalg RSA -keystore eap7console.jks -storepass changeit

2) Added to standalone/configuration/standalone.xml, the ssl config to ManagementRealm

    <management>
        <security-realms>
            <security-realm name="ManagementRealm">
                <server-identities>
                    <ssl>
                        <keystore path="eap7console.jks" relative-

to="jboss.server.config.dir" keystore-password="changeit" alias="jboss"/>
                    </ssl>
                </server-identities>
                <authentication>
                    <local default-user="$local" skip-group-loading="true"/>
                    <properties path="mgmt-users.properties" relative-

to="jboss.server.config.dir"/>
                </authentication>
                <authorization map-groups-to-roles="false">
                    <properties path="mgmt-groups.properties" relative-

to="jboss.server.config.dir"/>
                </authorization>
            </security-realm>

...
    </management>

3) Set management-https binding on Management interface

<management-interfaces>
          
            <http-interface security-realm="ManagementRealm" http-upgrade-enabled="true">
                <socket-binding https="management-https"/>
            </http-interface>
        </management-interfaces>

4) Modify the jconsole.sh script in $JBOSS_HOME/bin directory. For simplicity I created a

copy of jconsole.sh script for ssl connection. I called my copy jconsole_ssl.sh. Within the

new jconsole_ssl.sh script, I modified the startup parameters to include the truststore and

password.

$JAVA_HOME/bin/jconsole -J-Djava.class.path="$CLASSPATH" -J-Djavax.net.ssl.trustStorePassword=changeit -J-Djavax.net.ssl.trustStore=standalone/configuration/eap7console.jks -J-Djavax.net.debug=ssl "$@"

4) When jconsole starts up, I selected Remote Process and entered the following for the

service url:

service:jmx:remote+https://<hostname>:9993

The username and password for the management user created through the add-user.sh script

port 9993 is defined in the management-https socket binding.

5) The jconsole output window appears to show ssl connection succeeded based on the debug putput because I specified J-Djavax.net.debug=ssl on startup

===============================================================================

However domain mode is totally different problem.

1) In the domain/configuration/domain.xml file

I enabled the support of management of the server. Just for completeness I enabled it on

all profiles even though my server is using full profile.

            <subsystem xmlns="urn:jboss:domain:jmx:1.3">
                <expose-resolved-model/>
                <expose-expression-model/>
                <remoting-connector use-management-endpoint="false"/>
            </subsystem>

2) In the domain/configuration/host.xml file

I add ssl config to the ApplicationRealm since that is how the server are accessed in

domain mode using jmx management.

<security-realm name="ApplicationRealm">
                <server-identities>
                    <ssl protocol="TLSv1">
                         <keystore path="eap7console.jks" relative-

to="jboss.domain.config.dir" keystore-password="changeit" alias="jboss"/>
                    </ssl>
                 </server-identities>
                <authentication>
                    <local default-user="$local" allowed-users="*" skip-group-

loading="true"/>
                    <properties path="application-users.properties" relative-

to="jboss.domain.config.dir"/>
                </authentication>
                <authorization>
                    <properties path="application-roles.properties" relative-

to="jboss.domain.config.dir"/>
                </authorization>
            </security-realm>
        </security-realms>

3) Modify the jconsole.sh script in $JBOSS_HOME/bin directory. For simplicity I created a

copy of jconsole.sh script for ssl connection. I called my copy jconsole_ssl_domain.sh.

Within the new jconsole_ssl.sh script, I modified the startup parameters to include the

truststore and password.

$JAVA_HOME/bin/jconsole -J-Djava.class.path="$CLASSPATH" -J-

Djavax.net.ssl.trustStorePassword=changeit -J-

Djavax.net.ssl.trustStore=domain/configuration/eap7console.jks -J-Djavax.net.debug=ssl "$@"

4) Started jconsole script jconsole_ssl_domain.sh

5) This is what I tried in the jconsole dialog:

- Select Remote Process
- enter service url - service:jmx:remote+https://<hostname>:8443
- the user name and password is the application user created through the add-user.sh script

I am using the https port of the server

The connection fails.

=============================================

If I connect in non-ssl mode, I have no issues connecting to the following service url in jconsole:

service:jmx:remote+http://<hostname>:8080

application username and password

Currently I have been running my tests on wildfly 10.0.0 final. I have tried in EAP 7 also. I need to work in both wildfly and EAP versions. It should be identical procedure for setup in both.

[pjhavariotis]

Have you tried to use the native remoting connector like EAP 6?

By default EAP 7 is configured to use http upgrade and the connector is http-remoting-connector.

If you want to try the native remoting connector, you must make the appropriate configuration changes:

<subsystem xmlns="urn:jboss:domain:remoting:3.0">

    <endpoint/>

    <!-- this is the default http-remoting connector configuration -->

    <http-connector name="http-remoting-connector" connector-ref="default" security-realm="ApplicationRealm"/>

    <!-- this is the native remoting connector, note that you need to add a socket-binding also!  -->

    <connector name="remoting-connector" socket-binding="remoting" security-realm="ApplicationRealm"/>

</subsystem>

<socket-binding-group ...>

    ...

    <socket-binding name="remoting" port="4447" />

    ...

</socket-binding-group>

[dgilbert]

Even though this works I am questioning if this is the proper way to set this up.  From EAP 6.x to EAP 7.0 the remoting connector at port 4447 was removed and now you can access the servers at the app server port i.e. 8080. It seems odd to me that in a non-SSL setup that you can access the server using JMX with a service url of 'service:jmx:remote+http://<hostname>:<app server port> where app server port is the typically port the application server is running on i.e 8080. Using the above method,  a new port has to be created which does not to be guaranteed it will always exist. This approach is basically reverted to the EAP 6.0 config by adding a the remoting connector at port 4447.

My issue is that my application needs a consistent and known JMX protocol and port to connect to. I need the service URL in order to connect to each server to gather JMX metrics.

In non-ssl mode I know that I can use the 'remote+http' jmx service protocol and the port will be the HTTP application server port i.e. 8080. However with the above procedure for SSL it does not seem like I can rely on that the port is going to exist. I can assume there is going to be remoting socket binding but that is not going to work for me.

I still feel that has to be a way to use something like 'remote+https' and connect to the HTTPS application server port.

[dgilbert]

Not sure if solved my own problem. Here is what I did that seems to work.

1) In domain.xml file

Obviously I did the basics to enable JMX subsystem with <remoting-connector use-management-endpoint="false"/> .

<subsystem xmlns="urn:jboss:domain:jmx:1.3">

                <expose-resolved-model/>

                <expose-expression-model/>

                <remoting-connector use-management-endpoint="false"/>

            </subsystem>

This is the part below is what seemed to get things to work as expected.

NOTE: I modified the following on all profiles that were in use.

I changed the http-connector on remoting subsystem. I changed the name to 'https-remoting-connector' and set connector-ref to 'default-https'

<subsystem xmlns="urn:jboss:domain:remoting:3.0">

      <endpoint/>

      <http-connector name="https-remoting-connector" connector-ref="default-https" security-realm="ApplicationRealm"/>

</subsystem>

I also changed the ejb subsystem remote-connector to use https-remoting-connector.

<subsystem xmlns="urn:jboss:domain:ejb3:4.0">

      ....

    <remote connector-ref="https-remoting-connector" thread-pool-name="default"/>

     ....

</subsystem>

2) In host.xml file

I added SSL keystore to the ApplicationRealm

             <security-realm name="ApplicationRealm">

                <server-identities>

                    <ssl protocol="TLSv1">

                        <keystore path="eap7console.jks" relative-to="jboss.domain.config.dir" keystore-password="changeit" alias="jboss"/>

                     </ssl>

                </server-identities>

                <authentication>

                    <properties path="application-users.properties" relative-to="jboss.domain.config.dir"/>

                </authentication>

                <authorization>

                    <properties path="application-roles.properties" relative-to="jboss.domain.config.dir"/>

                </authorization>

            </security-realm>

3) Within my modified jconsole.sh script (I found it easier to create a copy of the script so I would have to keep specified command line args) which I modified to specify the following startup parameters.

$JAVA_HOME/bin/jconsole -J-Djava.class.path="$CLASSPATH" -J-Djavax.net.ssl.trustStorePassword=changeit -J-Djavax.net.ssl.trustStore=domain/configuration/eap7console.jks -J-Djavax.net.debug=ssl "$@"

In the Jconsole dialog

- Select Remote Process
- enter the following service url - service:jmx:remote+https://<hostname>:8443

- the user name and password is the application user created that is part of the ApplicationRealm

Others servers are accessible at https port + port offset

This solution seems to work for me. I have no idea if this is proper solution. If anyone with more experience or knowledge in this area can help or confirm this is correct approach, then feel free to chime in. Right now I am going to go with this solution.

[dgilbert]

Looks like I forgot one important part that is needed.

<subsystem xmlns="urn:jboss:domain:undertow:3.1">

     <https-listener name="default-https" socket-binding="https" security-realm="ApplicationRealm"/>

</subsystem>

This is tied in the http-connector on the remoting subsystem

I also found this article at http://middlewaremagic.com/jboss/?p=2783 after I did my implementation. Even though the article is about EJB SSL config, some of the info still applies.  See section on 'Configuring Wildfly 10 to use SSL'. This basically seems to be very similar to my steps. I probably could have create a separate security realm. I may try that out later today. The article also uses the CLI commands which is better than my manually modifying the domain.xml and host.xml directly.