Recently, we had a customer issue where our application was running OOM. Our application uses Jboss 4.2.3GA and JRE1.6 and is a 32-bit application. On debugging with the help of VisualVM, we noticed that several SFSB get passivated after an idle time of 5 minutes, but the memory usage does not come down. VisualVM memory dump showed that those SFFB instances in memory. We also noticed that on getting re-activated, a duplicate instance of the SFSB getting created. I went through the source code for SimpleStatefulCache, StatefulBeanContext and StatefulSessionFilePersistenceManager (all for Jboss 4.2.3GA). From analyzing the memory dumps and the source file this is what I think is happening -
- For creating an instance of SFSB, the SimpleStatefulCache invokes the Pool (LocalThreadPool). After creation, both the Cache and the Pool keep reference to the new SFSB.
- During passivation, the SFSB reference is removed from the Cache. But, I didn't see it getting removed from the Pool. Both the source code and the memory snapshot after passivation indicated this. As a result the SFSB did not get removed from memory even after passivation.
- When re-activated, I see that a new instance of the SFSB is getting created (by de-serialization) and a reference to it is added in the Cache. Again the Pool is not involved here and it still keeps the reference to the original instance of the SFSB. In the memory snapshot, I could see 2 instances of the same SFSB.
- Now, if the user session ends, the SFSB gets removed and I see that only the one in Cache (or the passivated file) gets removed from memory. From code I see that it calls remove from Pool as well - but the Pool does not have this instance (it has original one). This leaves behind the original SFSB in the Pool.
- This results in kind of memory leak and finally after many user sessions, we run into OOM.
- Also, noted that if the user closes session without idling for 5 minutes (no passivation) then the SFSB gets removed from memory. Also, if NoPassivationCache is used, then also the SFSB gets removed from memory when the user session ends.
My questions are -
- Is my analysis of the source code correct? Or am I missing something? (I some how feel I am missing something)
- If there is an issue with the SimpleStatefulCache in Jboss 4.2.3GA, is this a known issue? Is there a fix for this?
Thanks a lot in advance.