Broken pipe exceptions from CXF - JBoss AS 5.1
Hello forum!
Some context : a (clustered) project on JBoss AS 5.1 serving jax-ws (soap) web services.
jbossws-cxf-server 3.1.2.GA
Apache CXF API 2.2.2, I believe.
We had been running the WS client application with a 60 s timeout, and were keen to reduce this timeout. We also reduced the blocking-timeout (in the datasources config) correspondingly.
Load testing our application, while monitoring memory usage (visualvm), an unfortunate, memory consuming feedback effect became apparent.
When a web service operation finishes and attempts to send, and the client has already terminated the connection, exceptions are thrown:
[org.apache.cxf.phase.PhaseInterceptorChain] (http-0.0.0.0-8080-240) Interceptor has thrown exception, unwinding now
org.apache.cxf.interceptor.Fault: Marshalling Error: null
...
Caused by: ClientAbortException: java.net.SocketException: Broken pipe
at org.apache.catalina.connector.OutputBuffer.realWriteBytes(OutputBuffer.java:399)
...
Caused by: java.net.SocketException: Broken pipe
at java.net.SocketOutputStream.socketWrite0(Native Method)
When we increased the load to increase response times to the region where the soap client times out, the creation of large amounts of these exceptions lead to very high memory usage.
We have an 8G heap limit, and normally the app only uses 1-2G. The exceptions began to eat up most of the heap. The exceptions can of course be GC'd, but first they are created. The combination of clockcycles lost to exception creation and lost to GC, caused the response times to increase and thus caused more timeouts, which in turn caused more Broken pipe exceptions etc. etc.
Eventually nearly all memory was used up by exceptions and GCs which took multiple seconds occurred.
This positive feedback between client timeouts having occurred (which in themselves aren't a problem for us) and the exceptions, which increase the timeouts, is very undesirable.
I'm wondering if it's possible to configure CXF to fail silently instead of throwing these exceptions. And if not, if there might be an alternative (drop in replacement)?
We are migrating this application to JBoss AS 7.1.
I'm also curious whether jax-ws in AS 7.1 will behave in the same way? If the issue is fixable more easily in AS 7.1 we might skip trying to address it before the migration.