-
1. java.io.IOException: Too many open files
rareddy Mar 9, 2011 9:04 AM (in response to tawalisa)Allen,
How many client connection do you have. What platform. Teiid certainly does not throw too many files for moderate level of usage. However if you are opening lot of connections and holding on to them and doing the large queries (millions of records) there slight possibility. But you would have describe more to about your use case.
Ramesh..
-
2. java.io.IOException: Too many open files
shawkins Mar 9, 2011 10:51 AM (in response to rareddy)It may help to roughly account for things that may use a file handle:
-Each incoming socket connection
-Each outgoing socket connection to a source
-The buffermanager file cache, which is limited to 64 open files by default
-Each read from a Teiid filestore over the previous limit. There will never be more concurrent reads than maxActivePlans
-Any file usage by your sources
We have also seen a similar issue in the past with large jvm sizes (32gigs or more) and some vendor drivers.
Avoiding the issue may include increasing your open file limit, limiting source connections, etc.
Steve
-
3. java.io.IOException: Too many open files
tawalisa Mar 10, 2011 5:05 AM (in response to rareddy)Thanks your help.
platform: jboss soa 5.0.1 linux
limit parameter
core file size (blocks, -c) unlimited
data seg size (kbytes, -d) unlimited
scheduling priority (-e) 0
file size (blocks, -f) unlimited
pending signals (-i) 135167
max locked memory (kbytes, -l) 32
max memory size (kbytes, -m) unlimited
open files (-n) 1024
pipe size (512 bytes, -p) 8
POSIX message queues (bytes, -q) 819200
real-time priority (-r) 0
stack size (kbytes, -s) 10240
cpu time (seconds, -t) unlimited
max user processes (-u) 135167
virtual memory (kbytes, -v) unlimited
file locks (-x) unlimited
JVM arguments:
-Xms1303m -Xmx3303m -XX:PermSize=256m -XX:MaxPermSize=512m -XX:+UseParallelGC -XX:+UseParallelOldGC -XX:+HeapDumpOnOutOfMemoryError
This case is used 20 concurrent thread to deploy and undeploy VDB
Duration: 2 hours
Thanks
Best Regards
-
4. java.io.IOException: Too many open files
rareddy Mar 11, 2011 12:16 AM (in response to tawalisa)Allen,
Are you saying that you have 20 threads that are constantly deploying and undeploying the vdbs?
Ramesh..
-
5. java.io.IOException: Too many open files
tawalisa Mar 11, 2011 2:45 AM (in response to rareddy)Yes
-
6. java.io.IOException: Too many open files
tawalisa Mar 11, 2011 3:33 AM (in response to rareddy)Hi Ramesh,
Our logic need deploy and undeploy VDB, So we have to pass this case. Do you have any advice ?
Thanks
Best Regards
Allen
-
7. java.io.IOException: Too many open files
rareddy Mar 11, 2011 7:34 AM (in response to tawalisa)Allen,
This is a interesting test. Usually any typical applications just deploy a VDB and use it. The queries could be done using multiple threads. Anyway couple questions,
1) Are you using 20 different vdbs? or a single vdb.
2) Can provide the testcase you are using to test this.
Thanks
Ramesh..
-
8. java.io.IOException: Too many open files
tawalisa Mar 13, 2011 11:02 PM (in response to rareddy)Ramesh,
In our architecture, we will used many VDB. This VDB will be deployed and undeployed anytime.
Thanks
Best Regards
Allen
-
9. Re: java.io.IOException: Too many open files
tawalisa Mar 15, 2011 7:30 AM (in response to tawalisa)Hello Ramesh,
I have tested with deploying and undelpoying of VDB.
I found :
When deploy 2 VDB and undeploy 2 VDB, then I execute "lsof" to check. I found teiid server will increase 6 "open file"
They are:
java 6780 root 395r IPv4 117035708 TCP 192.168.3.3:59916 (LISTEN)
java 6780 root 416u IPv4 117035704 TCP 192.168.3.3:59915 (LISTEN)
java 6780 root 424u REG 253,0 749935 68519379 /opt/jboss-teiid-5.0.1/jboss-as/server/var/mwm/jboss/server/teiid-server/tmp/teiid/buffer/b_temp-stream_0_4310886018569254268.tmp
java 6780 root 426u REG 253,0 749933 68519380 /opt/jboss-teiid-5.0.1/jboss-as/server/var/mwm/jboss/server/teiid-server/tmp/teiid/buffer/b_temp-stream_0_5857253331971952634.tmp
java 6780 root 427u IPv4 117035721 TCP 192.168.3.3:59917 (LISTEN)
java 6780 root 428u IPv4 117035722 TCP 192.168.3.3:59918 (LISTEN)
I found teiid add 2 open file and 4 listen. Maybe it is caused this issue. Do you think it is leak for teiid?
Thanks
-
10. java.io.IOException: Too many open files
tawalisa Mar 15, 2011 7:24 AM (in response to rareddy)Hello Ramesh,
I met many question about teiid in our endurance test. I want to know whether it is reasonable. In our architecture, we will used many VDB.Maybe the number will over 100. And under the each VDB, it will have another vdb and an oracle. I think this architecture is so large for teiid.
Do you have any advice? Or do you think how many should teiid in the jboss?
Thanks your help.
Allen
-
11. java.io.IOException: Too many open files
rareddy Mar 15, 2011 10:21 AM (in response to tawalisa)Allen,
This is more of a tuning question, please open another discussion thread for this and post the link here. Simple answer, possible depending upon your VM size and capabilities of the server.
Thanks.
Ramesh..
-
12. java.io.IOException: Too many open files
rareddy Mar 16, 2011 12:04 AM (in response to tawalisa) -
13. java.io.IOException: Too many open files
tawalisa Mar 16, 2011 4:19 AM (in response to rareddy)Dear Ramesh,
I have tested our test case for one day.
[root@charon test]# lsof -p 6780 | grep java|wc -l
1546
We can see. This value has increased to 1546. So I think it is bug.
Thanks for your help.
Best Regards
Allen
-
14. java.io.IOException: Too many open files
rareddy Mar 16, 2011 10:32 AM (in response to tawalisa)Allen,
Yes, but I need to do further testing. Follow the above JIRA for the progress of the issue.
Thanks
Ramesh..