Wildfly undertow stop responding..

[peppem]

Recently widlfy undertow in production stop to response.

the ejb timers continue to works correctly and the widfly service is up, but each web module stop to responding.

It happened different times (one or two each day) In the server.log there are not same indications about the reason:

One time server.log reports too many file descriptor opened on socket exception, but next time when happend again other reason:

..completed with multiple threads - thread default task-135 was in progress with java.net.SocketInputStream.socketRead0(Native Method).

when problem occurs

lsof -f | wc -l

count is 6000

netstat -anp | grep -w 80 | grep ESTABLISHED | wc -l

count 15 on port 80

count 191 on port 443

Wildfly version 10.1.0

the undertow version in

/opt/wildfly-10.1.0.Final/modules/system/layers/base/io/undertow/core/main/

is 1.4.0

How can I debug the problem? there are some setup on wildfly to got details about server status (http connection, jndi, ram used etc)?

Can I collect other details at linux os level?

[ctomc]

Giuseppe Morreale wrote:

One time server.log reports too many file descriptor opened on socket exception, but next time when happend again other

Well, that is it! Increase number of file descriptors that can be open, by default it is really small (1024) for whole server.

see https://easyengine.io/tutorials/linux/increase-open-files-limit/  how to do that.

[peppem]

Hi Tomaz,
Thank you for your feedback.

Unfortunately the limit are already setup high:

cat /proc/sys/fs/file-max -> 2403066


ulimit -Hn -> 65536


ulimit -Sn -> 65536

and after undertow stop responding the file descriptor opened (lsof -f | wc -l) are less than 6000