Dialog between wildfly and haproxy - Invalid PROXY protocol header

[guigou]

Hello,

I have one linux machine with 2 Wildfly servers listening on 2 différents https ports.

I have one domain and 2 sub-domain: aa.mydomain.fr et bb.mydomain.fr that i redirect to my 2 wildlfy servers using a Haproxy (i didn't find other solutions to redirect 2 sub-domain in dealing with 2 different https ports and one linux server IP)

Haproxy v1.8.8

Wildfly v15.0.1

• My HapProxy server configuration (for aa.mydomain.fr only):

global
    log 127.0.0.1:514 local0 info
    daemon
    maxconn 4096
    tune.ssl.default-dh-param 1024
    ssl-default-bind-options ssl-min-ver TLSv1.2

defaults
    mode http
    timeout connect 5000ms
    timeout client 50000ms
    timeout server 50000ms
    log global
    option httplog
    option forwardfor

frontend http-in
    bind linux_server_ip:80
    acl is_demo_site hdr_end(host) aa.mydomain.fr
    use_backend demo_site if is_demo_site

frontend https-in
    bind linux_server_ip:443 ssl crt /etc/haproxy/cert/mycert.pem
    acl is_demo_https_site hdr_end(host) aa.mydomain.fr
    use_backend demo_https_site if is_demo_https_site

backend demo_site
    server s1 linux_server_ip:8xxx maxconn 32

backend demo_https_site
    server s3 linux_server_ip:8yyy maxconn 32
    http-request set-header X-Forwarded-Proto https

• My wildfly server conf for sub-domain aa.mydomain.fr:

<subsystem xmlns="urn:jboss:domain:undertow:8.0" default-server="default-server" default-virtual-host="default-host" default-servlet-container="default" default-security-domain="other">

            <buffer-cache name="default"/>

            <server name="default-server">

                <http-listener name="default" socket-binding="http" redirect-socket="https" proxy-address-forwarding="true" enable-http2="true"/>

                <https-listener name="https" socket-binding="https" security-realm="ApplicationRealm" enable-http2="true" proxy-protocol="true"/>

                <host name="default-host" alias="localhost">

                    <location name="/" handler="welcome-content"/>

                    <access-log pattern="%a %t %H %p %U %s %S %T" directory="${jboss.home.dir}/standalone/log" prefix="access_"/>

                    <http-invoker security-realm="ApplicationRealm"/>

                </host>

            </server>

            <servlet-container name="default">

                <jsp-config/>

                <websockets/>

            </servlet-container>

            <handlers>

                <file name="welcome-content" path="${jboss.home.dir}/welcome-content"/>

            </handlers>

</subsystem>

<socket-binding-group name="standard-sockets" default-interface="public" port-offset="${jboss.socket.binding.port-offset:0}">
...
        <socket-binding name="http" port="${jboss.http.port:8xxx}"/>
        <socket-binding name="https" port="${jboss.https.port:8yyy}"/>
...
  </socket-binding-group>

The http redirection works fine but not the https one which return an 502 error code bad Gateway and i have this error message in my wildfly server log:

2019-09-10 10:47:11,746 TRACE [org.xnio.nio] (default I/O-2) Running task org.xnio.nio.QueuedNioTcpServer$1@7b85bf52
2019-09-10 10:47:11,746 TRACE [org.xnio.nio] (default I/O-2) Running task org.xnio.nio.NioHandle$1@dd77838
2019-09-10 10:47:11,746 DEBUG [io.undertow.request.io] (default I/O-2) UT005013: An IOException occurred: java.io.IOException: UT000179: Invalid PROXY protocol header
at [e-mail]//io.undertow.server.protocol.proxy.ProxyProtocolReadListener.handleEvent(ProxyProtocolReadListener.java:90)
at [e-mail]//io.undertow.server.protocol.proxy.ProxyProtocolReadListener.handleEvent(ProxyProtocolReadListener.java:34)
at [e-mail]//org.xnio.ChannelListeners.invokeChannelListener(ChannelListeners.java:92)
at [e-mail]//org.xnio.conduits.ReadReadyHandler$ChannelListenerHandler.readReady(ReadReadyHandler.java:66)
at [e-mail]//org.xnio.nio.NioSocketConduit.handleReady(NioSocketConduit.java:89)
at [e-mail]//org.xnio.nio.NioHandle$1.run(NioHandle.java:50)
at [e-mail]//org.xnio.nio.WorkerThread.safeRun(WorkerThread.java:612)
at [e-mail]//org.xnio.nio.WorkerThread.run(WorkerThread.java:479)

2019-09-10 10:47:11,747 TRACE [org.xnio.nio] (default I/O-2) Cancelling key channel=java.nio.channels.SocketChannel[connected local=/linux_server_ip:8xxx remote=/linux_server_ip:49866], selector=sun.nio.ch.EPollSelectorImpl@4a7d8873, interestOps=1, readyOps=0 of java.nio.channels.SocketChannel[connected local=/linux_server_ip:8xxx remote=/linux_server_ip:49866] (same thread)
2019-09-10 10:47:11,747 TRACE [org.xnio.nio] (default I/O-2) Added task org.xnio.nio.QueuedNioTcpServer$2@1939a2a9

Details of the error :

    private static final byte[] NAME = "PROXY ".getBytes(StandardCharsets.US_ASCII);

..

    public void handleEvent(StreamSourceChannel streamSourceChannel) {
        PooledByteBuffer buffer = bufferPool.allocate();
        boolean freeBuffer = true;
        try {
            for (; ; ) {
                int res = streamSourceChannel.read(buffer.getBuffer());
                if (res == -1) {
                    IoUtils.safeClose(streamConnection);
                    return;
                } else if (res == 0) {
                    return;
                } else {
                    buffer.getBuffer().flip();
                    while (buffer.getBuffer().hasRemaining()) {
                        char c = (char) buffer.getBuffer().get();
                        if (byteCount < NAME.length) {
                            //first we verify that we have the correct protocol
                            if (c != NAME[byteCount]) {
                                throw UndertowMessages.MESSAGES.invalidProxyHeader();

...

Notes:

1. I use a "Let's encrypt" SSL certificat.

2. I get the same error code if i remove the "option forwardfor" in the Haproxy conf.

3. If i add "accept-proxy" in frontend https-in section and "send-proxy" in backend demo_https_site, i get the Following message in haproxy.log: "Received something which does not look like a PROXY protocol header".

4. When i monitor the header request with FF monitor tools, i don't see X-Forwarded detail...

I don't know if the issue come from my wildfly conf or my haproxy conf, can somebody suggest idea or fix please ?

Best regards.