Undertow java.nio.BufferOverflowException - buffer-pool configuration?

[darrenjones]

I'm getting a BufferOverflowException for a certain request (full stack trace below) in WildFly 8.0.0.Final. It looks like I can change the buffer-cache configuration in the io subsytem, but I'm having trouble finding any advice on what to set it to. Can anyone help?

Here's what I have (the defaults, I think).

<buffer-pool name="default" buffer-size="16384" buffers-per-slice="128"/>

Do I need to increase the buffer-size? Is this buffer allocated per request, or is this single buffer used by multiple concurrent requests? On Linux I've found a suggestion that it should be 16384, but what about Windows?

Or is it more appropriate to increase buffers-per-slice? Is buffers-per-slice at all related to the number of concurrent requests I expect to receive?

Thanks.

The exception:

17:29:35,460 ERROR [io.undertow.request] (default task-108) Blocking request failed HttpServerExchange{ GET /MyApp/exception}: java.nio.BufferOverflowException

  at java.nio.DirectByteBuffer.put(DirectByteBuffer.java:357) [rt.jar:1.7.0_25]

  at java.nio.ByteBuffer.put(ByteBuffer.java:832) [rt.jar:1.7.0_25]

  at io.undertow.util.HttpString.appendTo(HttpString.java:204)

  at io.undertow.server.protocol.http.HttpResponseConduit.processWrite(HttpResponseConduit.java:149)

  at io.undertow.server.protocol.http.HttpResponseConduit.flush(HttpResponseConduit.java:627)

  at io.undertow.conduits.AbstractFixedLengthStreamSinkConduit.flush(AbstractFixedLengthStreamSinkConduit.java:181)

  at org.xnio.conduits.ConduitStreamSinkChannel.flush(ConduitStreamSinkChannel.java:162) [xnio-api-3.2.1.Final.jar:3.2.1.Final]

  at io.undertow.channels.DetachableStreamSinkChannel.flush(DetachableStreamSinkChannel.java:100)

  at io.undertow.server.HttpServerExchange.closeAndFlushResponse(HttpServerExchange.java:1477)

  at io.undertow.server.HttpServerExchange.endExchange(HttpServerExchange.java:1465)

  at io.undertow.server.Connectors.executeRootHandler(Connectors.java:183)

  at io.undertow.server.HttpServerExchange$1.run(HttpServerExchange.java:727)

  at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_25]

  at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_25]

  at java.lang.Thread.run(Thread.java:724) [rt.jar:1.7.0_25]

[ctomc]

I would recommend you to upgrade to 8.1.0.CR1

We changed  defaults for IO subsystem and they are now dynamically calculated based on hardware configuration you are running on.

If you still see problems after upgrade let us know.

[atchijov-vgw]

I am getting something similar with 8.1.0.Final. And it kills the worker thread ("default task-12" does not appear in the log after this exception). Not sure how often this happening. So far I have noticed only once.

00:18:44,461 ERROR U(1519264) [io.undertow.request] (default task-12) Blocking request failed HttpServerExchange{ POST /.../init}: java.nio.BufferOverflowException
     at java.nio.DirectByteBuffer.put(DirectByteBuffer.java:357) [rt.jar:1.7.0_65]
     at java.nio.ByteBuffer.put(ByteBuffer.java:832) [rt.jar:1.7.0_65]
     at io.undertow.util.HttpString.appendTo(HttpString.java:204)
     at io.undertow.server.protocol.http.HttpResponseConduit.processWrite(HttpResponseConduit.java:150)
     at io.undertow.server.protocol.http.HttpResponseConduit.flush(HttpResponseConduit.java:629)
     at io.undertow.conduits.FinishableStreamSinkConduit.flush(FinishableStreamSinkConduit.java:83)
     at org.xnio.conduits.ConduitStreamSinkChannel.flush(ConduitStreamSinkChannel.java:162) [xnio-api-3.2.2.Final.jar:3.2.2.Final]
     at io.undertow.channels.DetachableStreamSinkChannel.flush(DetachableStreamSinkChannel.java:100)
     at org.xnio.channels.Channels.flushBlocking(Channels.java:63) [xnio-api-3.2.2.Final.jar:3.2.2.Final]
     at io.undertow.servlet.spec.ServletOutputStreamImpl.close(ServletOutputStreamImpl.java:625)
     at io.undertow.servlet.spec.HttpServletResponseImpl.closeStreamAndWriter(HttpServletResponseImpl.java:451)
     at io.undertow.servlet.spec.HttpServletResponseImpl.responseDone(HttpServletResponseImpl.java:525)
     at io.undertow.servlet.handlers.ServletInitialHandler.handleFirstRequest(ServletInitialHandler.java:287)
     at io.undertow.servlet.handlers.ServletInitialHandler.dispatchRequest(ServletInitialHandler.java:227)
     at io.undertow.servlet.handlers.ServletInitialHandler.access$000(ServletInitialHandler.java:73)
     at io.undertow.servlet.handlers.ServletInitialHandler$1.handleRequest(ServletInitialHandler.java:146)
     at io.undertow.server.Connectors.executeRootHandler(Connectors.java:177)
     at io.undertow.server.HttpServerExchange$1.run(HttpServerExchange.java:727)
     at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_65]
     at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_65]
     at java.lang.Thread.run(Thread.java:745) [rt.jar:1.7.0_65]

00:18:44,471 ERROR U(1519264) [stderr] (default task-12) Exception in thread "default task-12" java.nio.BufferOverflowException
00:18:44,471 ERROR U(1519264) [stderr] (default task-12)      at java.nio.DirectByteBuffer.put(DirectByteBuffer.java:357)
00:18:44,471 ERROR U(1519264) [stderr] (default task-12)      at java.nio.ByteBuffer.put(ByteBuffer.java:832)
00:18:44,471 ERROR U(1519264) [stderr] (default task-12)      at io.undertow.util.HttpString.appendTo(HttpString.java:204)
00:18:44,471 ERROR U(1519264) [stderr] (default task-12)      at io.undertow.server.protocol.http.HttpResponseConduit.processWrite(HttpResponseConduit.java:150)
00:18:44,471 ERROR U(1519264) [stderr] (default task-12)      at io.undertow.server.protocol.http.HttpResponseConduit.flush(HttpResponseConduit.java:629)
00:18:44,472 ERROR U(1519264) [stderr] (default task-12)      at io.undertow.conduits.FinishableStreamSinkConduit.flush(FinishableStreamSinkConduit.java:83)
00:18:44,472 ERROR U(1519264) [stderr] (default task-12)      at org.xnio.conduits.ConduitStreamSinkChannel.flush(ConduitStreamSinkChannel.java:162)
00:18:44,472 ERROR U(1519264) [stderr] (default task-12)      at io.undertow.channels.DetachableStreamSinkChannel.flush(DetachableStreamSinkChannel.java:100)
00:18:44,472 ERROR U(1519264) [stderr] (default task-12)      at io.undertow.server.HttpServerExchange.closeAndFlushResponse(HttpServerExchange.java:1489)
00:18:44,472 ERROR U(1519264) [stderr] (default task-12)      at io.undertow.server.HttpServerExchange.endExchange(HttpServerExchange.java:1470)
00:18:44,472 ERROR U(1519264) [stderr] (default task-12)      at io.undertow.server.Connectors.executeRootHandler(Connectors.java:201)
00:18:44,472 ERROR U(1519264) [stderr] (default task-12)      at io.undertow.server.HttpServerExchange$1.run(HttpServerExchange.java:727)
00:18:44,472 ERROR U(1519264) [stderr] (default task-12)      at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
00:18:44,473 ERROR U(1519264) [stderr] (default task-12)      at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
00:18:44,473 ERROR U(1519264) [stderr] (default task-12)      at java.lang.Thread.run(Thread.java:745)

[ctomc]

I think something like this was fixed not too long ago.

Any chance you can give WildFly 9.0.0.Alpha1 a try? as it has latest fixes there already.

[atchijov-vgw]

I can try... but only on our test server... and so far based on logs, this problem happen twice over last two month on this server . not a high chance to get conclusive evidence that it was fixed.

[ctomc]

Any testing would help, as undertow fixes currently present in master (9 work) will get into 8.2.

It would be great to know if there are any more problems we should address.

[atchijov-vgw]

ctomc do u know if [UNDERTOW-282] randomly org.xnio.conduits.AbstractStreamSourceConduit.read() goes into infinite loop - JBoss Issue Track… was addressed in 8.2/9 ? 

[ctomc]

Andrei Tchijov wrote:

Tomaz Cerar do u know if [UNDERTOW-282] randomly org.xnio.conduits.AbstractStreamSourceConduit.read() goes into infinite loop - JBoss Issue Track… was addressed in 8.2/9 ?

Should be, given that there ware lots of fixes in this area. But impossible to tell unless it is tested and confirmed.

[rajam]

We are also getting the same "java.nio.BufferOverflowException" in WildFly 8.2 running Suse Linux OS. As per previous comment in this forum,this problem is fixed in above 8.1 x versions but we are getting in 8.2 also. Do we need to change the default buffer size? What is recommended size or any settings. Please help us.

Stack trace:

java.nio.BufferOverflowException

  at java.nio.DirectByteBuffer.put(DirectByteBuffer.java:357)

  at java.nio.ByteBuffer.put(ByteBuffer.java:832)

  at io.undertow.util.HttpString.appendTo(HttpString.java:204)

  at io.undertow.server.protocol.http.HttpResponseConduit.processWrite(HttpResponseConduit.java:166)

  at io.undertow.server.protocol.http.HttpResponseConduit.write(HttpResponseConduit.java:564)

  at io.undertow.conduits.ChunkedStreamSinkConduit.doWrite(ChunkedStreamSinkConduit.java:186)

  at io.undertow.conduits.ChunkedStreamSinkConduit.write(ChunkedStreamSinkConduit.java:126)

  at org.xnio.conduits.ConduitStreamSinkChannel.write(ConduitStreamSinkChannel.java:150)

  at io.undertow.channels.DetachableStreamSinkChannel.write(DetachableStreamSinkChannel.java:239)

  at io.undertow.server.HttpServerExchange$WriteDispatchChannel.write(HttpServerExchange.java:1850)

  at io.undertow.servlet.spec.ServletOutputStreamImpl.writeBufferBlocking(ServletOutputStreamImpl.java:567)

  at io.undertow.servlet.spec.ServletOutputStreamImpl.flushInternal(ServletOutputStreamImpl.java:482)

  at io.undertow.servlet.spec.ServletPrintWriter.write(ServletPrintWriter.java:271)

  at io.undertow.servlet.spec.ServletPrintWriter.write(ServletPrintWriter.java:300)

  at io.undertow.servlet.spec.ServletPrintWriter.print(ServletPrintWriter.java:308)

  at io.undertow.servlet.spec.ServletPrintWriter.println(ServletPrintWriter.java:380)

  at io.undertow.servlet.spec.ServletPrintWriterDelegate.println(ServletPrintWriterDelegate.java:219)

[nitin.shukla]

Hi,

I am facing this issue too with Wildfly 8.2.0 final. Is there any recommendation or documentation on how to set the buffer size? Any pointers are appreciated.

In case members have found any resolution for Wildfly 8.2.0, please share your knowledge.

Thanks.

Nitin

[nitin.shukla]

Hi,

I found this documentation Wildfly Model Reference  on the buffer pool. Regarding buffer-size it mentions default value to be 1024 but not sure if this is B/KB/MB. Does any one have information on what this value mean or have pointer(s) to any more detail documentation on this configuration.

Thanks.

Regards,

Nitin