I am facing a strange is JBoss AS 7.1.1 Final. I have a spring + hibernate based web application. After deployment all transactions are quick. But in a few days any transaction that involves updates to DB seem to become extremely slow. What would take 4 seconds, takes 40 seconds.
Selects (read-only transactions) seem to be not affected by this problem. Their performance remains largely consistent.
This happens in our production server only. I have tried to recreate it in a lower environment, but it simply does not happen.
Server restart brings performance back to normal. But in a few days it becomes slow again. I use a MySQL database in production as well as non-production. I do not suspect MySQL because, JBoss restart is resolving the slow down. I asked the production DBA, and they do not see any deadlocks or bottlenecks on the DB side. My JBoss connection pool never goes beyond 10 used connections.
In production I have -Xms512m -Xmx2048m
In my test server I only have -Xms64m -Xmx512m - and it still does not slow down. Did not see any slow down on consistent production like use for 10 days. Production normally gets slow by 3rd or 4th day.
I have reviewed jmap heapdumps taken every night. There does not seem to be a leak, as the utilization comes back to normal very regularly. The time spent on GC is also very less. I have reviwed jstack thread dumps and they all seem normal. CPU utilization is always below 10% and Memory (RAM) utilization is always below 30%.
The only thing that stands out is - UPDATES only are becoming slow.
Any ideas how I can debug this? Is there any specific log message I can look for? In production my log level is ERROR and I do not want to change it for all classes.
Thanks in advance for any pointer that can help me crack this issue.
Best Regards,
sagar