10 Replies Latest reply on Feb 20, 2012 10:19 AM by ronnys

    Pending transactions not removed, page files not deleted

    ronnys

      Hi Clebert,

       

      we recently ran into some new paging issues with v2.2.5, that might be related to issues already posted here. I did some tests with a v2.2.11 AS7 version build from the SVN tag with the same name: same issues there.

       

      We've a Jboss 4.0.4 (it's old, I know, but that's the current situation) server with some MDBs connected to HornetQ. For whatever reason - most likely caused by restarts of the server - HornetQ stopped removing page files for a queue. It stays in paging mode, even if there are no messages to retrieve, and adds new files, but never cleans them up.

       

      In order to reproduce the issue I did some tests with a producer and consumer based on the javax.jms API and found out, that the issue we saw might have been caused by open transactions. If a) a transaction isn't committed before the session is being closed or b) if the session isn't closed at all (but the connection only) or c) if the session is rolled back or d) if the sending client crashed or gets killed, the page file containing the pending transaction stays and all subsequent page files aren't cleaned up either during *runtime*; they are cleaned up only after HornetQ gets restarted and in some cases it needs the connect of the first client as well. Setting connection-ttl-override or transaction-timeout* doesn't help. You may use the attached HQMessageGenerator.java and HQRetrievalTest.java to reproduce it; run the generator first, check the page files, run the retrieval tool and check the page files again. HornetQ at least cleans up the page files when it gets restarted in this scenario. This helps a bit, but I'd expect a better housekeeping here. If there is a need to keep certain page files longer than others I'd assume, that the pages are more or less independent. But right now, a single pages can block all other subsequent - potentially unused - pages from being deleted which eats a lot of spaces and slows down HornetQ restarts dramatically, not speaking of the vast amount of memory required when it scans all its pages.

       

      Could you please have a look?

       

      On Jboss 4.0.4 (using the HornetQ-JMS-RA), the issue looks a bit more worse with HornetQ 2.2.5: In addition to the problems above, HornetQ doesn't even clean up the pages after it got restarted. It might be related to Jboss or the configured XA transactions (local transactions don't work for whatever reason - message are received properly but never committed - different issue ...). However, I cannot test it with hornetq-ra.rar v2.2.11 to check if it makes any difference, as it fails to load the class org.jboss.tm.XAResourceRecovery (HornetQResourceRecovery is derived from it), which isn't part of the (or this) Jboss 4.0.4 installation. Just FYI, that this version is not 100% compatible with Jboss 4 anymore.

       

      Best regards,

      Ronny