UT005032: Listener not making progress with EJB over HTTP
I'm trying to upgrade an application from WildFly 10.1.0.Final to 16.0.0.Final, and as part of that I need to implement EJB over HTTP to support load balancing in AWS.
I have two WildFly 16 standalone servers. Server A calls a stateless session EJB running in server B. Some calls to the EJB succeed, and others fail with "UT005032: Listener not making progress with EJB over HTTP". This error is reported on server A, and immediately after that server B fails with "WFHTTP000006: Failed to write exception: java.nio.channels.ClosedChannelException".
The call that fails works fine when I run it in a standalone client, but it fails consistently when I run it inside server A. If I put server B in debug with a breakpoint on the method that's being called I can see that the call never actually makes it to the EJB (it never hits the breakpoint when server A calls it, but it does when the standalone client calls it). There's another method on the EJB that server A calls and it works consistently.
Does anyone have any suggestions on logging I can enable in server B to help track down where the call from server A is getting lost, i.e., why it's not getting into the EJB?
Server A log output
2019-04-23 12:31:37,008-0400 TRACE [org.xnio.listener] (XNIO-1 I/O-1) Invoking listener org.wildfly.httpclient.common.WildflyClientOutputStream$1@38a31d63 on channel io.undertow.protocols.http2.Http2HeadersStreamSinkChannel@6288941 2019-04-23 12:31:37,008-0400 TRACE [org.xnio.nio] (XNIO-1 I/O-1) Added task io.undertow.server.protocol.framed.AbstractFramedStreamSinkChannel$1@3b8f9bf7 2019-04-23 12:31:37,008-0400 TRACE [org.xnio.nio] (XNIO-1 I/O-1) Not polling, no wakeup 2019-04-23 12:31:37,008-0400 TRACE [org.xnio.nio] (XNIO-1 I/O-1) Running task io.undertow.server.protocol.framed.AbstractFramedStreamSinkChannel$1@3b8f9bf7 2019-04-23 12:31:37,008-0400 TRACE [org.xnio.listener] (XNIO-1 I/O-1) Invoking listener org.wildfly.httpclient.common.WildflyClientOutputStream$1@38a31d63 on channel io.undertow.protocols.http2.Http2HeadersStreamSinkChannel@6288941 2019-04-23 12:31:37,008-0400 TRACE [org.xnio.nio] (XNIO-1 I/O-1) Added task io.undertow.server.protocol.framed.AbstractFramedStreamSinkChannel$1@3b8f9bf7 2019-04-23 12:31:37,008-0400 TRACE [org.xnio.nio] (XNIO-1 I/O-1) Not polling, no wakeup 2019-04-23 12:31:37,008-0400 TRACE [org.xnio.nio] (XNIO-1 I/O-1) Running task io.undertow.server.protocol.framed.AbstractFramedStreamSinkChannel$1@3b8f9bf7 2019-04-23 12:31:37,008-0400 ERROR [io.undertow] (XNIO-1 I/O-1) UT005032: Listener not making progress on framed channel, closing channel to prevent infinite loop 2019-04-23 12:31:37,009-0400 TRACE [org.xnio.safe-close] (XNIO-1 I/O-1) Closing resource io.undertow.protocols.http2.Http2HeadersStreamSinkChannel@6288941 2019-04-23 12:31:37,010-0400 TRACE [org.xnio.nio] (XNIO-1 I/O-1) Added task Channel listener task for io.undertow.protocols.http2.Http2HeadersStreamSinkChannel@6288941 -> org.wildfly.httpclient.common.WildflyClientOutputStream$1@38a31d63 2019-04-23 12:31:37,010-0400 TRACE [org.xnio.nio] (XNIO-1 I/O-1) Not polling, no wakeup 2019-04-23 12:31:37,010-0400 TRACE [org.xnio.nio] (XNIO-1 I/O-1) Added task Channel listener task for io.undertow.protocols.http2.Http2HeadersStreamSinkChannel@6288941 -> org.wildfly.httpclient.common.WildflyClientOutputStream$1@38a31d63 2019-04-23 12:31:37,010-0400 TRACE [org.xnio.nio] (XNIO-1 I/O-1) Not polling, no wakeup 2019-04-23 12:31:37,010-0400 TRACE [org.xnio.nio] (XNIO-1 I/O-1) Running task Channel listener task for io.undertow.protocols.http2.Http2HeadersStreamSinkChannel@6288941 -> org.wildfly.httpclient.common.WildflyClientOutputStream$1@38a31d63 2019-04-23 12:31:37,010-0400 TRACE [org.xnio.listener] (XNIO-1 I/O-1) Invoking listener org.wildfly.httpclient.common.WildflyClientOutputStream$1@38a31d63 on channel io.undertow.protocols.http2.Http2HeadersStreamSinkChannel@6288941 2019-04-23 12:31:37,010-0400 TRACE [org.xnio.nio] (XNIO-1 I/O-1) Running task Channel listener task for io.undertow.protocols.http2.Http2HeadersStreamSinkChannel@6288941 -> org.wildfly.httpclient.common.WildflyClientOutputStream$1@38a31d63 2019-04-23 12:31:37,010-0400 TRACE [org.xnio.listener] (XNIO-1 I/O-1) Invoking listener org.wildfly.httpclient.common.WildflyClientOutputStream$1@38a31d63 on channel io.undertow.protocols.http2.Http2HeadersStreamSinkChannel@6288941 2019-04-23 12:31:37,011-0400 TRACE [org.xnio.nio.selector] (XNIO-1 I/O-1) Beginning select on sun.nio.ch.WindowsSelectorImpl@142ed837 (with timeout) 2019-04-23 12:31:37,011-0400 TRACE [org.xnio.nio] (XNIO-1 I/O-1) Select, queue is empty 2019-04-23 12:31:37,013-0400 TRACE [org.xnio.nio.selector] (XNIO-1 I/O-1) Selected on sun.nio.ch.WindowsSelectorImpl@142ed837
Server B log output (I added a log statement that produces lines 79-164 below so that I could see what exception it was trying to send):
2019-04-23 15:51:35,310+0000 TRACE [org.xnio.nio] (default I/O-2) Select, queue is empty
2019-04-23 15:51:35,368+0000 TRACE [org.xnio.nio] (default task-1) Added task io.undertow.server.protocol.framed.AbstractFramedChannel$1@39c44788
2019-04-23 15:51:35,369+0000 TRACE [org.xnio.nio.selector] (default I/O-2) Selected on sun.nio.ch.WindowsSelectorImpl@6b192ae
2019-04-23 15:51:35,369+0000 TRACE [org.xnio.nio] (default I/O-2) Running task io.undertow.server.protocol.framed.AbstractFramedChannel$1@39c44788
2019-04-23 15:51:35,370+0000 TRACE [org.xnio.listener] (default task-1) Setting channel listener to Flushing channel listener -> io.undertow.io.AsyncSenderImpl$1@5bfdd54d
2019-04-23 15:51:35,374+0000 TRACE [org.xnio.nio] (default I/O-2) Added task Channel listener task for io.undertow.protocols.http2.Http2HeadersStreamSinkChannel@4d218016 -> org.xnio.conduits.StreamSinkChannelWrappingConduit$1@76f68f6
2019-04-23 15:51:35,374+0000 TRACE [org.xnio.nio] (default I/O-2) Not polling, no wakeup
2019-04-23 15:51:35,375+0000 TRACE [org.xnio.nio] (default I/O-2) Running task Channel listener task for io.undertow.protocols.http2.Http2HeadersStreamSinkChannel@4d218016 -> org.xnio.conduits.StreamSinkChannelWrappingConduit$1@76f68f6
2019-04-23 15:51:35,375+0000 TRACE [org.xnio.listener] (default I/O-2) Invoking listener org.xnio.conduits.StreamSinkChannelWrappingConduit$1@76f68f6 on channel io.undertow.protocols.http2.Http2HeadersStreamSinkChannel@4d218016
2019-04-23 15:51:35,375+0000 TRACE [org.xnio.listener] (default I/O-2) Invoking listener Setter delegating channel listener -> Simple channel listener setter (currently=Flushing channel listener -> io.undertow.io.AsyncSenderImpl$1@5bfdd54d) on channel org.xnio.conduits.ConduitStreamSinkChannel@55ceaf2a
2019-04-23 15:51:35,376+0000 TRACE [org.xnio.listener] (default I/O-2) Invoking listener Flushing channel listener -> io.undertow.io.AsyncSenderImpl$1@5bfdd54d on channel io.undertow.server.HttpServerExchange$WriteDispatchChannel@28498171
2019-04-23 15:51:35,376+0000 TRACE [org.xnio.listener] (default I/O-2) Invoking listener io.undertow.server.protocol.http2.Http2ReceiveListener$4@33f50e96 on channel io.undertow.protocols.http2.Http2HeadersStreamSinkChannel@4d218016
2019-04-23 15:51:35,379+0000 TRACE [org.xnio.listener] (default I/O-2) Setting channel listener to io.undertow.io.AsyncSenderImpl$1@5bfdd54d
2019-04-23 15:51:35,379+0000 TRACE [org.xnio.listener] (default I/O-2) Invoking listener io.undertow.io.AsyncSenderImpl$1@5bfdd54d on channel io.undertow.server.HttpServerExchange$WriteDispatchChannel@28498171
2019-04-23 15:51:35,380+0000 TRACE [org.xnio.nio.selector] (default I/O-2) Beginning select on sun.nio.ch.WindowsSelectorImpl@6b192ae (with timeout)
2019-04-23 15:51:35,380+0000 TRACE [org.xnio.nio] (default I/O-2) Select, queue is empty
2019-04-23 15:51:35,410+0000 TRACE [org.xnio.nio.selector] (default I/O-2) Selected on sun.nio.ch.WindowsSelectorImpl@6b192ae
2019-04-23 15:51:35,410+0000 TRACE [org.xnio.nio.selector] (default I/O-2) Selected key sun.nio.ch.SelectionKeyImpl@494b6633 for java.nio.channels.SocketChannel[connected local=/10.39.146.56:8080 remote=/10.39.146.56:64153]
2019-04-23 15:51:35,410+0000 TRACE [org.xnio.nio.selector] (default I/O-2) Calling handleReady key 1 for java.nio.channels.SocketChannel[connected local=/10.39.146.56:8080 remote=/10.39.146.56:64153]
2019-04-23 15:51:35,410+0000 TRACE [org.xnio.listener] (default I/O-2) Invoking listener io.undertow.server.protocol.framed.AbstractFramedChannel$FrameReadListener@1719b766 on channel org.xnio.conduits.ConduitStreamSourceChannel@7a4899b1
2019-04-23 15:51:35,410+0000 TRACE [org.xnio.listener] (default I/O-2) Invoking listener io.undertow.server.protocol.http2.Http2ReceiveListener@62e56c17 on channel Http2Channel peer /10.39.146.56:64153 local /10.39.146.56:8080[ No Receiver [] -- [] -- []]
2019-04-23 15:51:35,412+0000 TRACE [org.xnio.listener] (default I/O-2) Setting channel listener to org.xnio.conduits.StreamSinkChannelWrappingConduit$1@76a8331f
2019-04-23 15:51:35,412+0000 TRACE [org.xnio.listener] (default I/O-2) Setting channel listener to org.xnio.conduits.StreamSourceChannelWrappingConduit$1@54116c50
2019-04-23 15:51:35,413+0000 TRACE [org.xnio.nio] (default I/O-2) Added task io.undertow.server.protocol.framed.AbstractFramedChannel$1@39c44788
2019-04-23 15:51:35,413+0000 TRACE [org.xnio.nio] (default I/O-2) Not polling, no wakeup
2019-04-23 15:51:35,413+0000 TRACE [org.xnio.nio] (default I/O-2) Running task io.undertow.server.protocol.framed.AbstractFramedChannel$1@39c44788
2019-04-23 15:51:35,413+0000 TRACE [org.xnio.listener] (default I/O-2) Invoking listener io.undertow.server.protocol.framed.AbstractFramedChannel$FrameReadListener@1719b766 on channel org.xnio.conduits.ConduitStreamSourceChannel@7a4899b1
2019-04-23 15:51:35,413+0000 TRACE [org.xnio.listener] (default I/O-2) Invoking listener io.undertow.server.protocol.http2.Http2ReceiveListener@62e56c17 on channel Http2Channel peer /10.39.146.56:64153 local /10.39.146.56:8080[ No Receiver [] -- [] -- []]
2019-04-23 15:51:35,413+0000 TRACE [org.xnio.nio.selector] (default I/O-2) Beginning select on sun.nio.ch.WindowsSelectorImpl@6b192ae (with timeout)
2019-04-23 15:51:35,413+0000 TRACE [org.xnio.nio] (default I/O-2) Select, queue is empty
2019-04-23 15:51:35,486+0000 TRACE [org.xnio.listener] (default task-2) Invoking listener io.undertow.server.protocol.http2.Http2ReceiveListener$2@1a20dd4d on channel Http2StreamSourceChannel{headers={accept=[application/x-wf-ejb-response;version=1,application/x-wf-jbmar-exception;version=1], http2-settings=[], authorization=[Digest username="ejb", uri="http://usronjsipher6:8080/wildfly-services/ejb/v1/invoke/wmserver/wmserverclient_ejb/wmserveroltp/TransactionServiceSO/-/com.ssaglobal.scm.wms.service.exeprocessmanager.TransactionServiceSORemoteHome/create", realm="ApplicationRealm", nc=00000001, cnonce="OC4jT68NXCPdZH8QmvAysIKwiV4lk2Tdysxchq2G", algorithm=MD5, nonce="AAAAAQAB3xLr106x8NcAh/f1uPe1xmt6rB8x0MlKi+GczR6p3mpcBWyzfZg=", opaque="00000000000000000000000000000000", qop=auth, response="58ed8a222759252b834b41fdedacbff7"], content-type=[application/x-wf-ejb-jbmar-invocation;version=1], Host=[usronjsipher6:8080]}}
2019-04-23 15:51:35,515+0000 INFO [org.jboss.ejb.client] (default task-1) JBoss EJB Client version 4.0.14.Final
2019-04-23 15:51:35,577+0000 TRACE [org.xnio.nio] (default task-1) Added task io.undertow.server.protocol.framed.AbstractFramedChannel$1@39c44788
2019-04-23 15:51:35,577+0000 TRACE [org.xnio.nio.selector] (default I/O-2) Selected on sun.nio.ch.WindowsSelectorImpl@6b192ae
2019-04-23 15:51:35,577+0000 TRACE [org.xnio.nio] (default I/O-2) Running task io.undertow.server.protocol.framed.AbstractFramedChannel$1@39c44788
2019-04-23 15:51:35,577+0000 TRACE [org.xnio.nio.selector] (default I/O-2) Beginning select on sun.nio.ch.WindowsSelectorImpl@6b192ae (with timeout)
2019-04-23 15:51:35,577+0000 TRACE [org.xnio.listener] (default task-1) Invoking listener io.undertow.server.protocol.http2.Http2ReceiveListener$4@26713e11 on channel io.undertow.protocols.http2.Http2HeadersStreamSinkChannel@5aa0ff15
2019-04-23 15:51:35,578+0000 TRACE [org.xnio.nio] (default I/O-2) Select, queue is empty
2019-04-23 15:51:35,685+0000 TRACE [org.xnio.nio.selector] (default I/O-2) Selected on sun.nio.ch.WindowsSelectorImpl@6b192ae
2019-04-23 15:51:35,686+0000 TRACE [org.xnio.nio.selector] (default I/O-2) Selected key sun.nio.ch.SelectionKeyImpl@494b6633 for java.nio.channels.SocketChannel[connected local=/10.39.146.56:8080 remote=/10.39.146.56:64153]
2019-04-23 15:51:35,686+0000 TRACE [org.xnio.nio.selector] (default I/O-2) Calling handleReady key 1 for java.nio.channels.SocketChannel[connected local=/10.39.146.56:8080 remote=/10.39.146.56:64153]
2019-04-23 15:51:35,686+0000 TRACE [org.xnio.listener] (default I/O-2) Invoking listener io.undertow.server.protocol.framed.AbstractFramedChannel$FrameReadListener@1719b766 on channel org.xnio.conduits.ConduitStreamSourceChannel@7a4899b1
2019-04-23 15:51:35,686+0000 TRACE [org.xnio.listener] (default I/O-2) Invoking listener io.undertow.server.protocol.http2.Http2ReceiveListener@62e56c17 on channel Http2Channel peer /10.39.146.56:64153 local /10.39.146.56:8080[ No Receiver [] -- [] -- []]
2019-04-23 15:51:35,686+0000 TRACE [org.xnio.listener] (default I/O-2) Setting channel listener to org.xnio.conduits.StreamSinkChannelWrappingConduit$1@10ab5ef5
2019-04-23 15:51:35,687+0000 TRACE [org.xnio.listener] (default I/O-2) Setting channel listener to org.xnio.conduits.StreamSourceChannelWrappingConduit$1@325ee5d2
2019-04-23 15:51:35,687+0000 TRACE [org.xnio.nio.selector] (default I/O-2) Beginning select on sun.nio.ch.WindowsSelectorImpl@6b192ae (with timeout)
2019-04-23 15:51:35,687+0000 TRACE [org.xnio.nio] (default I/O-2) Select, queue is empty
2019-04-23 15:51:36,001+0000 TRACE [org.xnio.nio.selector] (default I/O-2) Selected on sun.nio.ch.WindowsSelectorImpl@6b192ae
2019-04-23 15:51:36,001+0000 TRACE [org.xnio.nio] (default I/O-2) Running task io.undertow.util.DateUtils$2@1e75baba
2019-04-23 15:51:36,001+0000 TRACE [org.xnio.nio.selector] (default I/O-2) Beginning select on sun.nio.ch.WindowsSelectorImpl@6b192ae (with timeout)
2019-04-23 15:51:36,001+0000 TRACE [org.xnio.nio] (default I/O-2) Select, queue is empty
2019-04-23 15:51:39,646+0000 TRACE [org.xnio.nio.selector] (default I/O-2) Selected on sun.nio.ch.WindowsSelectorImpl@6b192ae
2019-04-23 15:51:39,646+0000 TRACE [org.xnio.nio.selector] (default I/O-2) Selected key sun.nio.ch.SelectionKeyImpl@494b6633 for java.nio.channels.SocketChannel[connected local=/10.39.146.56:8080 remote=/10.39.146.56:64153]
2019-04-23 15:51:39,646+0000 TRACE [org.xnio.nio.selector] (default I/O-2) Calling handleReady key 1 for java.nio.channels.SocketChannel[connected local=/10.39.146.56:8080 remote=/10.39.146.56:64153]
2019-04-23 15:51:39,646+0000 TRACE [org.xnio.listener] (default I/O-2) Invoking listener io.undertow.server.protocol.framed.AbstractFramedChannel$FrameReadListener@1719b766 on channel org.xnio.conduits.ConduitStreamSourceChannel@7a4899b1
2019-04-23 15:51:39,646+0000 TRACE [org.xnio.listener] (default I/O-2) Invoking listener io.undertow.server.protocol.http2.Http2ReceiveListener@62e56c17 on channel Http2Channel peer /10.39.146.56:64153 local /10.39.146.56:8080[ No Receiver [] -- [] -- []]
2019-04-23 15:51:39,647+0000 TRACE [org.xnio.safe-close] (default I/O-2) Closing resource io.undertow.protocols.http2.Http2RstStreamStreamSourceChannel@3b92ef71
2019-04-23 15:51:39,647+0000 TRACE [org.xnio.safe-close] (default I/O-2) Closing resource io.undertow.protocols.http2.Http2HeadersStreamSinkChannel@71044200
2019-04-23 15:51:39,647+0000 TRACE [org.xnio.listener] (default I/O-2) Invoking listener io.undertow.server.protocol.http2.Http2ReceiveListener$4@34b15987 on channel io.undertow.protocols.http2.Http2HeadersStreamSinkChannel@71044200
2019-04-23 15:51:39,648+0000 TRACE [org.xnio.nio] (default I/O-2) Added task io.undertow.server.protocol.framed.AbstractFramedChannel$1@39c44788
2019-04-23 15:51:39,648+0000 TRACE [org.xnio.nio] (default I/O-2) Not polling, no wakeup
2019-04-23 15:51:39,648+0000 TRACE [org.xnio.nio] (default I/O-2) Running task io.undertow.server.protocol.framed.AbstractFramedChannel$1@39c44788
2019-04-23 15:51:39,648+0000 TRACE [org.xnio.listener] (default I/O-2) Invoking listener org.xnio.conduits.StreamSinkChannelWrappingConduit$1@10ab5ef5 on channel io.undertow.protocols.http2.Http2HeadersStreamSinkChannel@71044200
2019-04-23 15:51:39,648+0000 TRACE [org.xnio.nio.selector] (default I/O-2) Beginning select on sun.nio.ch.WindowsSelectorImpl@6b192ae (with timeout)
2019-04-23 15:51:39,648+0000 TRACE [org.xnio.nio] (default I/O-2) Select, queue is empty
2019-04-23 15:51:39,649+0000 TRACE [org.xnio.nio] (default task-2) Added task io.undertow.server.protocol.framed.AbstractFramedChannel$1@39c44788
2019-04-23 15:51:39,649+0000 TRACE [org.xnio.nio.selector] (default I/O-2) Selected on sun.nio.ch.WindowsSelectorImpl@6b192ae
2019-04-23 15:51:39,649+0000 TRACE [org.xnio.nio] (default I/O-2) Running task io.undertow.server.protocol.framed.AbstractFramedChannel$1@39c44788
2019-04-23 15:51:39,650+0000 TRACE [org.xnio.nio.selector] (default I/O-2) Beginning select on sun.nio.ch.WindowsSelectorImpl@6b192ae (with timeout)
2019-04-23 15:51:39,650+0000 TRACE [org.xnio.listener] (default task-2) Setting channel listener to Flushing channel listener -> null
2019-04-23 15:51:39,650+0000 TRACE [org.xnio.nio] (default I/O-2) Select, queue is empty
2019-04-23 15:51:39,650+0000 TRACE [org.xnio.nio] (default task-2) Added task io.undertow.server.protocol.framed.AbstractFramedChannel$1@39c44788
2019-04-23 15:51:39,650+0000 TRACE [org.xnio.nio.selector] (default I/O-2) Selected on sun.nio.ch.WindowsSelectorImpl@6b192ae
2019-04-23 15:51:39,650+0000 TRACE [org.xnio.nio] (default I/O-2) Running task io.undertow.server.protocol.framed.AbstractFramedChannel$1@39c44788
2019-04-23 15:51:39,650+0000 TRACE [org.xnio.listener] (default I/O-2) Invoking listener Flushing channel listener -> null on channel io.undertow.protocols.http2.Http2RstStreamSinkChannel@7820581d
2019-04-23 15:51:39,650+0000 TRACE [org.xnio.listener] (default I/O-2) Setting channel listener to null
2019-04-23 15:51:39,650+0000 TRACE [org.xnio.nio.selector] (default I/O-2) Beginning select on sun.nio.ch.WindowsSelectorImpl@6b192ae (with timeout)
2019-04-23 15:51:39,650+0000 TRACE [org.xnio.nio] (default I/O-2) Select, queue is empty
2019-04-23 15:51:39,650+0000 ERROR [io.undertow] (default task-2) Sending exceptionjavax.ejb.EJBException: java.nio.channels.ClosedChannelException: javax.ejb.EJBException: java.nio.channels.ClosedChannelException
at org.jboss.as.ejb3.remote.AssociationImpl.receiveInvocationRequest(AssociationImpl.java:132)
at org.wildfly.httpclient.ejb.HttpInvocationHandler.lambda$0(HttpInvocationHandler.java:132)
at org.jboss.threads.ContextClassLoaderSavingRunnable.run(ContextClassLoaderSavingRunnable.java:35)
at org.jboss.threads.EnhancedQueueExecutor.safeRun(EnhancedQueueExecutor.java:1982)
at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.doRunTask(EnhancedQueueExecutor.java:1486)
at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.run(EnhancedQueueExecutor.java:1377)
at java.lang.Thread.run(Thread.java:748)
Caused by: java.nio.channels.ClosedChannelException
at io.undertow.server.protocol.framed.AbstractFramedStreamSourceChannel.beforeRead(AbstractFramedStreamSourceChannel.java:539)
at io.undertow.server.protocol.framed.AbstractFramedStreamSourceChannel.read(AbstractFramedStreamSourceChannel.java:502)
at io.undertow.protocols.http2.Http2StreamSourceChannel.read(Http2StreamSourceChannel.java:147)
at org.xnio.conduits.StreamSourceChannelWrappingConduit.read(StreamSourceChannelWrappingConduit.java:43)
at org.xnio.conduits.ConduitStreamSourceChannel.read(ConduitStreamSourceChannel.java:127)
at io.undertow.channels.DetachableStreamSourceChannel.read(DetachableStreamSourceChannel.java:209)
at io.undertow.server.HttpServerExchange$ReadDispatchChannel.read(HttpServerExchange.java:2341)
at org.xnio.channels.Channels.readBlocking(Channels.java:294)
at io.undertow.io.UndertowInputStream.readIntoBuffer(UndertowInputStream.java:109)
at io.undertow.io.UndertowInputStream.read(UndertowInputStream.java:89)
at java.io.FilterInputStream.read(FilterInputStream.java:133)
at java.io.FilterInputStream.read(FilterInputStream.java:107)
at org.jboss.marshalling.SimpleDataInput.readUnsignedByteDirect(SimpleDataInput.java:293)
at org.jboss.marshalling.SimpleDataInput.readUnsignedByte(SimpleDataInput.java:249)
at org.jboss.marshalling.river.RiverUnmarshaller.start(RiverUnmarshaller.java:1347)
at org.wildfly.httpclient.ejb.HttpInvocationHandler$1.getRequestContent(HttpInvocationHandler.java:156)
at org.jboss.as.ejb3.remote.AssociationImpl.receiveInvocationRequest(AssociationImpl.java:130)
... 6 more
Suppressed: java.nio.channels.ClosedChannelException
at io.undertow.server.protocol.framed.AbstractFramedStreamSourceChannel.beforeRead(AbstractFramedStreamSourceChannel.java:539)
at io.undertow.server.protocol.framed.AbstractFramedStreamSourceChannel.read(AbstractFramedStreamSourceChannel.java:502)
at io.undertow.protocols.http2.Http2StreamSourceChannel.read(Http2StreamSourceChannel.java:147)
at org.xnio.conduits.StreamSourceChannelWrappingConduit.read(StreamSourceChannelWrappingConduit.java:43)
at org.xnio.conduits.ConduitStreamSourceChannel.read(ConduitStreamSourceChannel.java:127)
at io.undertow.channels.DetachableStreamSourceChannel.read(DetachableStreamSourceChannel.java:209)
at io.undertow.server.HttpServerExchange$ReadDispatchChannel.read(HttpServerExchange.java:2341)
at org.xnio.channels.Channels.readBlocking(Channels.java:294)
at io.undertow.io.UndertowInputStream.readIntoBuffer(UndertowInputStream.java:109)
at io.undertow.io.UndertowInputStream.close(UndertowInputStream.java:160)
at org.wildfly.httpclient.ejb.HttpInvocationHandler$1.getRequestContent(HttpInvocationHandler.java:201)
... 7 more
2019-04-23 15:51:39,665+0000 TRACE [org.xnio.nio.selector] (default I/O-2) Selected on sun.nio.ch.WindowsSelectorImpl@6b192ae
2019-04-23 15:51:39,665+0000 TRACE [org.xnio.nio.selector] (default I/O-2) Selected key sun.nio.ch.SelectionKeyImpl@494b6633 for java.nio.channels.SocketChannel[connected local=/10.39.146.56:8080 remote=/10.39.146.56:64153]
2019-04-23 15:51:39,665+0000 TRACE [org.xnio.nio.selector] (default I/O-2) Calling handleReady key 1 for java.nio.channels.SocketChannel[connected local=/10.39.146.56:8080 remote=/10.39.146.56:64153]
2019-04-23 15:51:39,665+0000 TRACE [org.xnio.listener] (default I/O-2) Invoking listener io.undertow.server.protocol.framed.AbstractFramedChannel$FrameReadListener@1719b766 on channel org.xnio.conduits.ConduitStreamSourceChannel@7a4899b1
2019-04-23 15:51:39,665+0000 TRACE [org.xnio.listener] (default I/O-2) Invoking listener io.undertow.server.protocol.http2.Http2ReceiveListener@62e56c17 on channel Http2Channel peer /10.39.146.56:64153 local /10.39.146.56:8080[ No Receiver [] -- [] -- []]
2019-04-23 15:51:39,666+0000 TRACE [org.xnio.safe-close] (default I/O-2) Closing resource io.undertow.protocols.http2.Http2GoAwayStreamSourceChannel@46e35bff
2019-04-23 15:51:39,667+0000 TRACE [org.xnio.listener] (default I/O-2) Invoking listener io.undertow.server.protocol.framed.AbstractFramedChannel$FrameCloseListener@53f7fc02 on channel org.xnio.conduits.ConduitStreamSinkChannel@1ec4fde9
2019-04-23 15:51:39,667+0000 TRACE [org.xnio.safe-close] (default I/O-2) Closing resource Http2Channel peer /10.39.146.56:64153 local /10.39.146.56:8080[ No Receiver [] -- [] -- []]
2019-04-23 15:51:39,667+0000 TRACE [org.xnio.safe-close] (default I/O-2) Closing resource org.xnio.nio.NioSocketStreamConnection@1acb3981
2019-04-23 15:51:39,667+0000 TRACE [org.xnio.nio] (default I/O-2) Cancelling key sun.nio.ch.SelectionKeyImpl@494b6633 of java.nio.channels.SocketChannel[connected oshut local=/10.39.146.56:8080 remote=/10.39.146.56:64153] (same thread)
2019-04-23 15:51:39,668+0000 TRACE [org.xnio.nio] (default I/O-2) Added task org.xnio.nio.QueuedNioTcpServer$2@7b865c23
2019-04-23 15:51:39,668+0000 TRACE [org.xnio.listener] (default I/O-2) Invoking listener io.undertow.server.protocol.framed.AbstractFramedChannel$FrameCloseListener@53f7fc02 on channel org.xnio.conduits.ConduitStreamSourceChannel@7a4899b1
2019-04-23 15:51:39,668+0000 TRACE [org.xnio.nio.selector] (default Accept) Selected on sun.nio.ch.WindowsSelectorImpl@3a06577
2019-04-23 15:51:39,668+0000 TRACE [org.xnio.nio] (default Accept) Running task org.xnio.nio.QueuedNioTcpServer$2@7b865c23
2019-04-23 15:51:39,668+0000 TRACE [org.xnio.nio.selector] (default Accept) Beginning select on sun.nio.ch.WindowsSelectorImpl@3a06577
2019-04-23 15:51:39,668+0000 TRACE [org.xnio.nio] (default Accept) Select, queue is empty
2019-04-23 15:51:39,668+0000 TRACE [org.xnio.nio] (default I/O-2) Added task io.undertow.server.protocol.framed.AbstractFramedChannel$1@39c44788
2019-04-23 15:51:39,668+0000 TRACE [org.xnio.nio] (default I/O-2) Not polling, no wakeup
2019-04-23 15:51:39,668+0000 TRACE [org.xnio.listener] (default I/O-2) Invoking listener io.undertow.server.AbstractServerConnection$CloseSetter@13d39f19 on channel org.xnio.nio.NioSocketStreamConnection@1acb3981
2019-04-23 15:51:39,669+0000 TRACE [org.xnio.nio] (default I/O-2) Running task io.undertow.server.protocol.framed.AbstractFramedChannel$1@39c44788
2019-04-23 15:51:39,669+0000 TRACE [org.xnio.listener] (default I/O-2) Invoking listener io.undertow.protocols.http2.Http2Channel$2@5afdae04 on channel Http2Channel peer /10.39.146.56:64153 local 0.0.0.0/0.0.0.0:8080[ No Receiver [] -- [] -- []]
2019-04-23 15:51:39,669+0000 TRACE [org.xnio.nio.selector] (default I/O-2) Beginning select on sun.nio.ch.WindowsSelectorImpl@6b192ae
2019-04-23 15:51:39,669+0000 TRACE [org.xnio.nio] (default I/O-2) Select, queue is empty
2019-04-23 15:51:39,680+0000 TRACE [org.xnio.nio.selector] (default I/O-2) Selected on sun.nio.ch.WindowsSelectorImpl@6b192ae
2019-04-23 15:51:39,680+0000 TRACE [org.xnio.nio.selector] (default I/O-2) Beginning select on sun.nio.ch.WindowsSelectorImpl@6b192ae (with timeout)
2019-04-23 15:51:39,681+0000 TRACE [org.xnio.nio] (default I/O-2) Select, queue is empty
2019-04-23 15:51:39,683+0000 ERROR [org.wildfly.httpclient.common] (default task-2) WFHTTP000006: Failed to write exception: java.nio.channels.ClosedChannelException
at io.undertow.channels.DetachableStreamSinkChannel.write(DetachableStreamSinkChannel.java:238)
at io.undertow.server.HttpServerExchange$WriteDispatchChannel.write(HttpServerExchange.java:2103)
at io.undertow.io.UndertowOutputStream.writeBufferBlocking(UndertowOutputStream.java:295)
at io.undertow.io.UndertowOutputStream.flush(UndertowOutputStream.java:277)
at java.io.FilterOutputStream.flush(FilterOutputStream.java:140)
at org.jboss.marshalling.SimpleDataOutput.flush(SimpleDataOutput.java:339)
at org.jboss.marshalling.SimpleDataOutput.finish(SimpleDataOutput.java:378)
at org.jboss.marshalling.AbstractMarshaller.finish(AbstractMarshaller.java:126)
at org.wildfly.httpclient.common.HttpServerHelper.sendException(HttpServerHelper.java:56)
at org.wildfly.httpclient.ejb.HttpInvocationHandler$1.writeException(HttpInvocationHandler.java:268)
at org.jboss.as.ejb3.remote.AssociationImpl.receiveInvocationRequest(AssociationImpl.java:132)
at org.wildfly.httpclient.ejb.HttpInvocationHandler.lambda$0(HttpInvocationHandler.java:132)
at org.jboss.threads.ContextClassLoaderSavingRunnable.run(ContextClassLoaderSavingRunnable.java:35)
at org.jboss.threads.EnhancedQueueExecutor.safeRun(EnhancedQueueExecutor.java:1982)
at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.doRunTask(EnhancedQueueExecutor.java:1486)
at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.run(EnhancedQueueExecutor.java:1377)
at java.lang.Thread.run(Thread.java:748)
2019-04-23 15:51:39,687+0000 TRACE [org.xnio.safe-close] (default task-2) Closing resource io.undertow.server.HttpServerExchange$DefaultBlockingHttpExchange@2ac6fe03
2019-04-23 15:51:40,001+0000 TRACE [org.xnio.nio.selector] (default I/O-2) Selected on sun.nio.ch.WindowsSelectorImpl@6b192ae