1 2 Previous Next 18 Replies Latest reply on Sep 6, 2013 6:41 AM by bernd.koecke

    How can an EJB from one WildFly server call another one with remoting and SSL?

    bernd.koecke

      I have two WildFly server, one is the service provider, the other one the service consumer. Both have an EAR with EJBs deployed. Now I would like the service consumer bean let call the service provider bean over a SSL secured connection. But I can't get it working. The provider listens for connections with an undertow https-listener and the consumer uses a remote outgoing connection. I can call both beans from a JavaSE client with SSL, without any problems. But when the two server should talk to each other the communication stops during SSL handshake. And the server (the provider) throws an exception after both printed out the ignored/unused cipher suites:

       

      javax.net.ssl.SSLException: Unrecognized SSL message, plaintext connection?

       

      The JavaSE client shows the same behavior when I set SSL_STARTTLS=true on the client side. It works, when this option isn't set at all. My problem is that in class org.jboss.as.remoting.RemoteOutboundConnectionService the endpoint is always configured with SSL_STARTTLS=true. It overwrites all what is set through the CLI.

       

      The result is that the consumer starts sending data not encrypted. On the provider side the https-listener (org.wildfly.extension.undertow.HttpsListenerService) doesn't allow me to configure the connection. In method startListening is the OptionMap hardcoded. The result is that the provider WildFly server expects encryption right from the beginning. The exception is right, the consumer sends plain data and the provider expects encrypted data. Do you know a way how they could understand each other?

       

      May be I should use a http-listener and use the upgrade from http to remoting for switching to encryption, but I can't get it working either.

       

      For completeness here are some parts of my configuration:

       

      Both WildFly server are from github, ALPHA4-SNAPSHOT from this morning.

       

      WildFly provider config:

       

      subsystem undertow, default-server:

       

      <https-listener name="defaults" socket-binding="https" security-realm="HttpsRealm"/>

       

      subsystem remoting

      <http-connector name="https-remoting-connector" connector-ref="defaults" security-realm="ProviderAppRealm"/>

       

      subsystem ejb3:

      <remote connector-ref="https-remoting-connector" thread-pool-name="default"/>

       

      Output on the service provider side when the communication stops:

       

      [stdout] (default I/O-3) Using SSLEngineImpl.

      [stdout] (default I/O-3) Ignoring unavailable cipher suite: TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA

      *** 13 lines more ***

      [stdout] (default I/O-3) Allow unsafe renegotiation: false

      [stdout] (default I/O-3) Allow legacy hello messages: true

      [stdout] (default I/O-3) Is initial handshake: true

      [stdout] (default I/O-3) Is secure renegotiation: false

      [stdout] (default I/O-3) default I/O-3, fatal error: 80: problem unwrapping net record

      [stdout] (default I/O-3) javax.net.ssl.SSLException: Unrecognized SSL message, plaintext connection?

      [stdout] (default I/O-3) default I/O-3, SEND TLSv1 ALERT:  fatal, description = internal_error

      [stdout] (default I/O-3) default I/O-3, WRITE: TLSv1 Alert, length = 2

      [stdout] (default I/O-3) default I/O-3, called closeOutbound()

      [stdout] (default I/O-3) default I/O-3, closeOutboundInternal()

      [stdout] (default I/O-3) [Raw write]: length = 7

      [stdout] (default I/O-3) 0000: 15 03 01 00 02 02 50                               ......P

       

      WildFly consumer config

       

      subsystem remoting, outbound connections:

      <remote-outbound-connection name="provider-one-connection" outbound-socket-binding-ref="provider-ejb" security-realm="ProviderOneRealm" protocol="https-remoting">

        <properties>

          <property name="SASL_POLICY_NOANONYMOUS" value="false"/>

        </properties>

      </remote-outbound-connection>

       

      The realms contain the key- and truststores and I got no error messages that a key is missing or not trusted or a password is wrong. I tried this for the working JavaSE client and I got dedicated error messages when one of the values don't fit.

       

      Can you tell me, what I did wrong? I think I miss something really basic, but I can't find it. Thanks a lot in advance for your help!

       

      Best regards

      Bernd

        • 1. Re: How can an EJB from one WildFly server call another one with remoting and SSL?
          jaikiran

          Is this still an issue against the just released WildFly 8.0.0.Alpha4?

          • 2. Re: How can an EJB from one WildFly server call another one with remoting and SSL?
            bernd.koecke

            I'm sorry, yes, I did a pull at around 14:20 UTC today, got a 8.0.0.Beta1-SNAPSHOT, build and run it and got the same result.

            • 3. Re: How can an EJB from one WildFly server call another one with remoting and SSL?
              bernd.koecke

              I think my last post could be misunderstood. What I did was to take the 8.0.0.Beta1-SNAPSHOT and the issue is still there. I can't call an EJB from another WildFly server over a SSL secured socket with the configuration of my inital post. I think, because of the defaults in the source code, it isn't possible with this setup.

               

              One solution could be to use the unencrypted http connection and do the upgrade to TLS during the SASL capability exchange. But I don't have an idea how to configure this.

              • 4. Re: How can an EJB from one WildFly server call another one with remoting and SSL?
                jaikiran

                Thanks for the update. I'll take a look later today.

                • 5. Re: How can an EJB from one WildFly server call another one with remoting and SSL?
                  jaikiran

                  By the way, I don't see the hardcoding you mention. Can you point me to the code?

                  • 6. Re: How can an EJB from one WildFly server call another one with remoting and SSL?
                    bernd.koecke

                    May be I'm looking at the wrong place, but anyway here is what I found:

                     

                    The service consumer (or client) WildFly has a remote-outbound-connection in the remoting subsystem config which results in an org.jboss.as.remoting.RemoteOutboundConnectionService. There is a connect() method which contains the lines:

                     

                        OptionMap.Builder builder = OptionMap.builder();

                        builder.addAll(this.connectionCreationOptions);

                        builder.set(SASL_POLICY_NOANONYMOUS, Boolean.FALSE);

                        builder.set(SASL_POLICY_NOPLAINTEXT, Boolean.FALSE);

                        builder.set(Options.SASL_DISALLOWED_MECHANISMS, Sequence.of(JBOSS_LOCAL_USER));

                        builder.set(Options.SSL_ENABLED, true);

                        builder.set(Options.SSL_STARTTLS, true);

                     

                        return endpoint.connect(uri, builder.getMap(), callbackHandler, sslContext);

                     

                    My configuration comes from "this.connectionOptions" and is overwritten by the defaults. The service consumer will open the connection with SSL_STARTTLS==true.

                     

                    The service provider (or server) WildFly uses an https-listener from the undertow subsystem. Which generates an org.wildfly.extension.undertow.HttpsListenerService. Method "startListening(XnioWorker worker,...)" has the following code:

                     

                    protected void startListening(XnioWorker worker, InetSocketAddress socketAddress, ChannelListener<AcceptingChannel<StreamConnection>> acceptListener) throws IOException {

                        SSLContext sslContext = securityRealm.getValue().getSSLContext();

                        Builder builder = OptionMap.builder().addAll(SERVER_OPTIONS);

                        builder.set(UndertowOptions.MAX_ENTITY_SIZE, maxUploadSize);

                        if (securityRealm.getValue().getSupportedAuthenticationMechanisms().contains(AuthMechanism.CLIENT_CERT)) {

                            builder.set(SSL_CLIENT_AUTH_MODE, REQUESTED);

                        }

                        builder.set(Options.USE_DIRECT_BUFFERS,true);

                        OptionMap combined = builder.getMap();

                     

                        XnioSsl xnioSsl = new JsseXnioSsl(worker.getXnio(), combined, sslContext);

                        sslServer = xnioSsl.createSslConnectionServer(worker, socketAddress, (ChannelListener) acceptListener, combined);

                        sslServer.resumeAccepts();

                     

                        UndertowLogger.ROOT_LOGGER.listenerStarted("HTTPS", getName(), socketAddress);

                    }

                     

                    The OptionMap "combined" is used for the JsseXnioSsl and I can't see a way how to add my own configuration options. The result is that SSL_STARTTLS is undefined and in JsseXnioSsl has a method "connectSsl" which calls "openSslConnection". There is an event handler build which calls:

                     

                    new JsseSslStreamConnection(connection, JsseSslUtils.createSSLEngine(sslContext, optionMap, destination), socketBufferPool, applicationBufferPool, optionMap.get(Options.SSL_STARTTLS, false));

                     

                    This sets the flag "startTls" of the constructor to false and the member variable "tls" in org.xnio.ssl.JsseSslStreamConnection to true. And when I understand it right will then the consumer start unencrypted and will only switch to TLS when it is told todo so. But the service provider expectes, because of "tls==true", an encrypted connection right from the beginning. And this generates the above error message/exception. I can reproduce this by using a JavaSE client to call the service provider. This works well when I don't set SSL_STARTTLS on the client side. But when I set it to "true" I get the same behavior like when the client is another WildFly instance.

                     

                    Thanks in advance for looking into it! By the way I will not be in the office for a few days, only for the case that my answer will take a little bit longer.

                    • 7. Re: How can an EJB from one WildFly server call another one with remoting and SSL?
                      ctomc

                      Bernd Koecke wrote:

                       

                      May be I'm looking at the wrong place, but anyway here is what I found:

                       

                      The service consumer (or client) WildFly has a remote-outbound-connection in the remoting subsystem config which results in an org.jboss.as.remoting.RemoteOutboundConnectionService. There is a connect() method which contains the lines:

                       

                          OptionMap.Builder builder = OptionMap.builder();

                          builder.addAll(this.connectionCreationOptions);

                          builder.set(SASL_POLICY_NOANONYMOUS, Boolean.FALSE);

                          builder.set(SASL_POLICY_NOPLAINTEXT, Boolean.FALSE);

                          builder.set(Options.SASL_DISALLOWED_MECHANISMS, Sequence.of(JBOSS_LOCAL_USER));

                          builder.set(Options.SSL_ENABLED, true);

                          builder.set(Options.SSL_STARTTLS, true);

                       

                          return endpoint.connect(uri, builder.getMap(), callbackHandler, sslContext);

                       

                      My configuration comes from "this.connectionOptions" and is overwritten by the defaults. The service consumer will open the connection with SSL_STARTTLS==true.

                       

                      You are right, this is a bug. the addAll should be callled last.

                      1 of 1 people found this helpful
                      • 8. Re: How can an EJB from one WildFly server call another one with remoting and SSL?
                        jaikiran

                        Please create a JIRA for this.

                        1 of 1 people found this helpful
                        • 9. Re: How can an EJB from one WildFly server call another one with remoting and SSL?
                          bernd.koecke

                          I added the issue. It took a little bit longer, because I checked it with a new WildFly version from this morning. Only to make sure to file an issue for something which is still there.

                          • 10. Re: Re: How can an EJB from one WildFly server call another one with remoting and SSL?
                            grzegorz.lojek

                            Hello,

                             

                            I would like to add my bit to the thread. I need to create an EJB from another EJB running in different server using dynamic JNDI context lookup (feature added recently, see this: Re: ejb access from one server to multiple servers ).

                            It was working fine with unencrypted connection, however, I need to run it with SSL.

                             

                            I configured WildFly in the similar way as Bernd did (added HttpsRealm, changed remoting subsystem to use https-remoting-connector). The client calls the following code to create InitialContext:

                            Properties props = new Properties();

                            String connectionName = "c1";

                            props.put("org.jboss.ejb.client.scoped.context", true);

                            props.put(Context.URL_PKG_PREFIXES, "org.jboss.ejb.client.naming");

                            props.put("remote.connectionprovider.create.options.org.xnio.Options.SSL_ENABLED", "true");

                            props.put("remote.connections", connectionName);

                            props.put("remote.connection." + connectionName + ".host", host);

                            props.put("remote.connection." + connectionName + ".port", port);

                            props.put("remote.connection." + connectionName + ".username", user);

                            props.put("remote.connection." + connectionName + ".password", pass);

                            props.put("remote.connection." + connectionName + ".connect.options.org.xnio.Options.SSL_STARTTLS", "true") ;

                            props.put("remote.connection." + connectionName + ".connect.options.org.xnio.Options.SASL_POLICY_NOPLAINTEXT","false") ;

                            InitialContext ic = new InitialContext(props);

                             

                             

                            When I run it, the clients reports the following error:

                            [org.jboss.ejb.client.EJBClientPropertiesLoader]: Looking for jboss-ejb-client.properties using classloader sun.misc.Launcher$AppClassLoader@6fd7bd04

                            [org.jboss.ejb.client.PropertiesBasedEJBClientConfiguration]: endpoint.create.options. has the following options {}

                            [org.jboss.ejb.client.PropertiesBasedEJBClientConfiguration]: remote.connectionprovider.create.options. has the following options {org.xnio.Options.SSL_ENABLED=>true}

                            [org.jboss.ejb.client.PropertiesBasedEJBClientConfiguration]: remote.connection.c1.connect.options. has the following options {org.xnio.Options.SSL_STARTTLS=>true,org.xnio.Options.SASL_POLICY_NOPLAINTEXT=>false}

                            [org.jboss.ejb.client.PropertiesBasedEJBClientConfiguration]: remote.connection.c1.channel.options. has the following options {}

                            [org.jboss.ejb.client.PropertiesBasedEJBClientConfiguration]: Connection org.jboss.ejb.client.PropertiesBasedEJBClientConfiguration$RemotingConnectionConfigurationImpl@590864d2 successfully created for connection named c1

                            [org.jboss.ejb.client.PropertiesBasedEJBClientConfiguration]: No clusters configured in properties

                            [org.xnio]: XNIO Version 3.1.0.CR3

                            [org.xnio.nio]: XNIO NIO Implementation Version 3.1.0.CR3

                            [org.jboss.remoting]: JBoss Remoting version (unknown)

                            [org.xnio.nio]: Started channel thread 'Remoting "config-based-ejb-client-endpoint" I/O-1', selector sun.nio.ch.EPollSelectorImpl@386cbe05

                            [org.jboss.ejb.client.remoting.ConfigBasedEJBClientContextSelector]: Could not register a EJB receiver for connection to localhost:8543

                            java.lang.RuntimeException: Operation failed with status WAITING

                                at org.jboss.ejb.client.remoting.IoFutureHelper.get(IoFutureHelper.java:93)

                                at org.jboss.ejb.client.remoting.ConnectionPool.getConnection(ConnectionPool.java:77)

                                at org.jboss.ejb.client.remoting.RemotingConnectionManager.getConnection(RemotingConnectionManager.java:51)

                                at org.jboss.ejb.client.remoting.ConfigBasedEJBClientContextSelector.setupEJBReceivers(ConfigBasedEJBClientContextSelector.java:147)

                                at org.jboss.ejb.client.remoting.ConfigBasedEJBClientContextSelector.getCurrent(ConfigBasedEJBClientContextSelector.java:115)

                                at org.jboss.ejb.client.naming.ejb.EjbNamingContext.createIdentifiableEjbClientContext(EjbNamingContext.java:258)

                                at org.jboss.ejb.client.naming.ejb.EjbNamingContext.setupScopedEjbClientContextIfNeeded(EjbNamingContext.java:123)

                                at org.jboss.ejb.client.naming.ejb.EjbNamingContext.<init>(EjbNamingContext.java:98)

                                at org.jboss.ejb.client.naming.ejb.ejbURLContextFactory.getObjectInstance(ejbURLContextFactory.java:38)

                                at javax.naming.spi.NamingManager.getURLObject(NamingManager.java:601)

                                at javax.naming.spi.NamingManager.getURLContext(NamingManager.java:550)

                                at javax.naming.InitialContext.getURLOrDefaultInitCtx(InitialContext.java:339)

                                at javax.naming.InitialContext.lookup(InitialContext.java:411)

                                at common.EJBConnectorJBoss8.lookup(EJBConnectorJBoss8.java:86)

                                at client.TestPhysicalCluster.main(TestPhysicalCluster.java:10)

                            [org.jboss.ejb.client.remoting.ConfigBasedEJBClientContextSelector]: Registered a reconnect handler in EJB client context org.jboss.ejb.client.EJBClientContext@2016aa19 for remote://localhost:8543

                            [org.jboss.ejb.client.remoting.ConfigBasedEJBClientContextSelector]: Registered 0 remoting EJB receivers for EJB client context org.jboss.ejb.client.EJBClientContext@2016aa19

                             

                            On server side I can see:

                            Using SSLEngineImpl.

                            Allow unsafe renegotiation: false

                            Allow legacy hello messages: true

                            Is initial handshake: true

                            Is secure renegotiation: false

                            Ignoring unsupported cipher suite: TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA384 for SSLv2Hello

                            (27 similar lines, with different cipher suites)

                            default I/O-3, fatal error: 80: problem unwrapping net record

                            javax.net.ssl.SSLException: Unrecognized SSL message, plaintext connection?

                            SEND TLSv1 ALERT:  fatal, description = internal_error

                            WRITE: TLSv1 Alert, length = 2

                            called closeOutbound()

                            closeOutboundInternal()

                             

                            Could you please let me know if the way of creating InitialContext is OK ?( it worked when I connected to unencrypted port).

                            Do you think it will be supported soon?

                            I tried to patch WildFly moving line with "builder.addAll(this.connectionCreationOptions);" to the end, but it did not help.

                             

                            Thanks for taking a look!

                            • 11. Re: Re: How can an EJB from one WildFly server call another one with remoting and SSL?
                              bernd.koecke

                              As you can see from the linked JIRA issue, it is resolved by Jaikiran and I will try the solution today. It is still available upstream.

                               

                              Your patch should work, but your options are setting it back to the not working defaults. Did you tried it without the SSL_STARTTLS option? The only other difference is, that I don't use SASL_POLICY_NOPLAINTEXT. I have SASL_POLICY_NOANONYMOUS set to false. But I haven't checked if this makes a difference.

                              • 12. Re: Re: How can an EJB from one WildFly server call another one with remoting and SSL?
                                grzegorz.lojek

                                In fact, I just realized that I'm not testing EJB to EJB connections yet! I'm testing standalone app to EJB (EJB to EJB will be the next step).

                                You mentioned that connection from standalone application was working for you, right?

                                 

                                For me it does not matter what options I set. I think I tried all the combinations, and every time I have the same error - looking at server logs seems like my client always created plaintext connection:

                                problem unwrapping net record

                                javax.net.ssl.SSLException: Unrecognized SSL message, plaintext connection?

                                 

                                I'm using jboss-client.jar from WildFly, otherwise would not be able to use this dynamic JNDI lookup (which is really cool feature as I don't need to setup any remote outgoing/inbound connections between servers).

                                • 13. Re: How can an EJB from one WildFly server call another one with remoting and SSL?
                                  bernd.koecke

                                  Ah, sorry I missed that you use a standalone app. You have a standalone app which should call an EJB on a WildFly server and you want to do this through an SSL connection, right? For a standalone app you don't need the discussed patch. It is only necessary for the calling WildFly server. For the standalone app the properties for the JNDI context should be enough. But I had to configure the SSLContext before I created the JNDI context. A minimal setup needs a truststore with the imported X509 certificate of the server. Then I had to create a SSLContext, configure it with the truststore  and set it as default with SSLContext.setDefault() method. If you want to use client certificates, this setup does only work, if your application has got one identity, valid for all calls to all servers. Because you can only have one default SSL context for the whole JVM.

                                   

                                  I hope that I now understood your issue right. Please let me know, if you need some examples on how to configure the SSLContext.

                                  • 14. Re: Re: How can an EJB from one WildFly server call another one with remoting and SSL?
                                    grzegorz.lojek

                                    I was not aware of configuring SSLContext. On JBoss 7.1.1 I was able to setup SSL connection successfully, just with specifying path and password to truststore via JVM properties. It would be great if you could provide me with some examples how to configure the SSLContext.

                                    Thanks!

                                    1 2 Previous Next