JBossDeveloper

Hey,

do you maybe have a example application that can reproduce this on your system?

if so can you share it and i will try to reproduce it on exactly same hw/sw configuration.

I thought this was reproducible even with a simple bubblesort program from a servlet?

Daniel implemented the bubble sort as a health monitor - e.g. when the sort gets slow then our app has slowed down as well. I would not go so far as to assert that the bubble sort running alone as a servlet on EAP is creating slowdowns (though I do not think we have eliminated this as a possibility either). The interesting thing is that the bubble sort only slows when running in the web thread pool. If our app starts the bubble sort via JMS then it is still fast, even when our app and the servlet bubble sort (which is running in a different application context) have slowed down to a crawl.

Summarized: something is slowing down the web threads but not impacting the JMS and also not impacting the JVM as a whole on the server (e.g. when I start the sort as a standalone it is also still fast).

This specific slowdown of the web threads is what we are trying to narrow in upon.

One thing you could perhaps try is configuring the web subsystem to use an (thread) executor of your own instead of the pre-configured thread pool that the web subsystem uses. The web subsystem xsd should have the relevant information on how to do it.

I can't recommend anything for a production system since I haven't tried or looked at the performance aspects myself. IMO, the unbounded-queue-thread-pool should be fine, but that's just a suggestion.

Summarized: something is slowing down the web threads but not impacting the JMS and also not impacting the JVM as a whole on the server (e.g. when I start the sort as a standalone it is also still fast).

 

This specific slowdown of the web threads is what we are trying to narrow in upon.

We are using quartz scheduler that has it own thread pool. When our app slows down, we also see our quartz jobs slowing down.

Daniel,

what are exact CPUs you use in VM hosts?

I am trying to reproduce your problems but for now without any success

--

tomaz

... I experienced yet another slowdown in one of our demo servers. There was no active session (no one using de demo app) except mine.

The java process was around 10-20% cpu !

I generated a thread dump but there was no activities but

sun.nio.ch.EPollArrayWrapper.poll()

and

java.net.SocketInputStream.socketRead0() (obviously my own connection)

I googled "nio poll high cpu" and found  a lot of "known bugs"

https://forums.oracle.com/thread/1146875

https://issues.apache.org/bugzilla/show_bug.cgi?id=52858

https://java.net/jira/browse/GLASSFISH-5148

etc..

any known bug related to nio with jboss ?

We are using the jdk 7u25 ? And also testing open jdk 7(no slow down at the moment).

Daniel Wehrle a écrit:

 

Hibernate question: is it possible that in hibernate there is a bug which holds objects in the hibernate session if it is closed?

you mean : in the hibernate session even if it is closed?

I'd say somehow if you're using hibernate level 2 cache for instance. Technically, the object reference are not in the session but are hold by hibernate.

We are seeing the same things: We have a quartz batch that runs during 1-2 minutes when the server is ok. When the server is slow, the batch can take more than 30 minutes. If we generate thread dumps, the only thing visible is hibernate trying to flush objects again and again. The amount of dirty objects is the same as the server is ok or not.

I wonder if it could be a memory fragmentation issue ?

see The Specter of Fragmentation in http://blog.mgm-tp.com/2013/03/garbage-collection-tuning/

or

http://javabook.compuware.com/content/memory/impact-of-garbage-collection-on-performance.aspx

We are testing the "new" G1 garbage collector in one of our test server. Wait and see.