We are investigating a similar issue with different setup (Apache2/mod_jk 1.2.15, JBoss 3.2.7, jdk 1.5.0_06)
During testing with jmeter our server is busy but we don't get the app throughput we expect. We have a 4 CPU box and top shows all of them 90+ % busy.
HPROF reports what we perceive to be excessive time spent in socket reading, with cpu=samples:
rank self accum count trace method
1 87.00% 87.00% 29123554 302080 java.net.SocketInputStream.socketRead0
2 7.24% 94.24% 2424866 300748 java.net.PlainSocketImpl.socketAccept
3 1.49% 95.73% 497744 301511 java.net.SocketInputStream.socketRead0
4 0.90% 96.63% 301177 302061 java.net.PlainSocketImpl.socketAccept
cpu=samples is not conclusive as far as the net CPU time spent goes, but with cpu=times we still see ~ 50 time spent on these calls.
Our server is not running at capacity, bot worker threads and connetion pool is underutilized.
Has anyone made similar observations?
a thread dump is going to be more useful in your case (idle).
If this is true (the sockets consuming time), something must be wrong in your OS configuration, as a socket accept or socket read shouldn't take any CPU time at al.
Use some OS tools to figure out what's going on in your OS.