-
1. Re: java.net.SocketException: Too many open files on Red Hat
try to see what file handles are open using following command
lsof -p PID
I did face same problem while using Jboss 4.3 on SunOS.
We were using jboss webservices module and while load testing it was throwing same exception for WSDLjava.net.SocketException: Too many open files
I believe there is a patch available for this. We used that patch and that resolved the problem. -
2. Re: java.net.SocketException: Too many open files on Red Hat
Yes. we are also hitting a Webservices system and we post xml over http instead of WSDL.
There are 2 things that i have done to overcome this problem right now.
1) turned off the deployment scanner in the jboss-service.xml
2) added a executor tag for the server.xml so that it will close the TCP connections after there is idle time.
jboss-4.2.2.GA/server/ottserver/deploy/jboss-web.deployer/server.xml
<Executor name="appThreadpool" namePrefix="activeThread-" maxThreads="1000" minSpareThreads="30" maxIdleTime="30
for now we seem to have jumped over this max open files problem.
But there is a new problem which has cropped up now.
After running load tests for 3/4 hours using JMeter at the end of the 4th hour we saw a sudden spike in the jboss process where its hitting 100% CPU and this didn’t come back down after we stopped the requests from jmeter to the server. we had to kill the server to revive it.
any ideas as to what seems to be the problem.
also can you pls give me more info abt the patch.
thanks in advance.
Cheers
Kiran -
3. Re: java.net.SocketException: Too many open files on Red Hat
Do you really need maxThreads="1000" in your tomcat connector??
You should not set it that high, that might be reason your CPU is spiking when you do the load test, it will keep creating new threads for your incoming request.
Even if you are expecting 500 concurrent request, i think 50 threads should be able to handle that.
Also you can use acceptCount which is a queue for all your incoming connection requests when all possible request processing threads are in use.
Refer to following URL for the bug which i was talking about
https://jira.jboss.org/jira/browse/JBWS-2325 -
-
5. Re: java.net.SocketException: Too many open files on Red Hat
Yes.. I was able to resolve the problem. the executor was not mentioned which will close the connections/files that are open
Navigate to "jboss-5.1.0.GA/server/ottserver/deploy/jbossweb.sar" and edit server.xml as mentioned below.
Look for following xml snippet.
<Connector port="9090" address="${jboss.bind.address}" maxThreads="250" maxHttpHeaderSize="8192" emptySessionPath="true" protocol="HTTP/1.1" enableLookups="false" redirectPort="8443" acceptCount="100" connectionTimeout="20000" disableUploadTimeout="true" />
Replace this snippet with below one.
<Executor name="appThreadpool" namePrefix="activeThread-" maxThreads="150" minSpareThreads="3" maxIdleTime="30000" />
<Connector executor="appThreadpool" port="9090" address="${jboss.bind.address}" maxThreads="150" maxHttpHeaderSize="8192"
minSpareThreads="3" maxSpareThreads="20" emptySessionPath="true" protocol="HTTP/1.1"
enableLookups="false" redirectPort="8443" acceptCount="100" connectionTimeout="120000" disableUploadTimeout="true" />
Look for following xml snippet.
<Connector port="8009" address="${jboss.bind.address}" protocol="AJP/1.3" emptySessionPath="true" enableLookups="false" redirectPort="8443" />
Replace this snippet with below one.
<Connector executor="appThreadpool" port="8009" address="${jboss.bind.address}" protocol="AJP/1.3" emptySessionPath="true" enableLookups="false" redirectPort="8443" />
hope that helps.
thanks
Kiran
-
6. Re: java.net.SocketException: Too many open files on Red Hat
Yes. On jboss 4.2.x I resolve this problem too.
I just compile new jbossweb.jar from svn (2.0.x). It have bugfix.
At now I have CLOSE_WAIT = 0 and happy))
-
7. Re: java.net.SocketException: Too many open files on Red Hat
Hi Kiran,
i have tried the same way which you explained. but seems that i am getting again same exception.
is there any new patch is available in Jboss for this issue?
Please let me know how to resolve this issue...
Thanks in Advance.
Regards,
Silambarasan
-
8. Re: java.net.SocketException: Too many open files on Red Hat
compile this: http://anonsvn.jboss.org/repos/jbossweb/branches/2.0.x/
or use this patch https://jira.jboss.org/secure/attachment/12316577/patch.txt with JBOSSWEB_2_0_0.
IF patch work you seen close_wait arount zero:
netstat -a | grep CLOSE_WAIT .
-
9. Re: java.net.SocketException: Too many open files on Red Hat
Thanks for your reply....
Sorry for asking this...do we have any binary available for the latest with fix for jbossweb.2.0.x?
it would be great if i get jar instead of source code.
here is the exception we are getting in our jboss. but finally all the transcations getting broken and the server got hang-up...
04:55:34,751 ERROR [org.jboss.naming.Naming] Naming accept handler stopping
java.net.SocketException: Too many open files
at java.net.PlainSocketImpl.socketAccept(Native Method)
at java.net.PlainSocketImpl.accept(PlainSocketImpl.java:384)java.net.SocketException: Broken pipe
at java.net.SocketOutputStream.socketWrite0(Native Method)I believe the main issue is with the "too many open files", probably this could be solved with the latest jbossweb...
Cheers,
Silambarasan
-
10. Re: java.net.SocketException: Too many open files on Red Hat
I make myself binary and test it for jboss 4.2.1 and jboss 4.2.3.
just replace jbossweb.jar in deploy/jboss-web.deployer/ folder
-
11. Re: java.net.SocketException: Too many open files on Red Hat
We're having the same problem on JBoss 4.2.1 GA - we compiled jbossweb 2.0.0.x and replaced the jbossweb.jar file under jboss-web.deployer as suggested, but getting the following error on startup. Are you sure no other change is needed?
org.jboss.deployment.DeploymentException: - nested throwable: (java.lang.reflect.UndeclaredThrowableException)
Caused by: java.lang.NoClassDefFoundError: org/apache/catalina/Lifecycle
It's odd, because Lifecycle is definitely found within the jbossweb.jar file.
-
12. Re: java.net.SocketException: Too many open files on Red Hat
No, I do not have any problems like this