8 Replies Latest reply on Jun 18, 2011 10:00 AM by Leos Bitto

    Memory Leak in ServerLocator.factories?

    Geert Pante Newbie

      Hi all,

       

      We use HornetQ using JMS API in a web application.

      When a message needs to be sent, we use:

       

      con = connectionFactory.createConnection();

      con.start():

      ses = con.createSession( false, Session.CLIENT_ACKNOWLEDGE );

      producer=ses.createProducer(new HornetQQueue(name))

      msg = ses.createObjectMessage(...);

      producer.send(msg);

      producer.close();

      ses.close();

      con.close();

       

      After a few days running and sending lots of messages like this, we get an OutOfMemoryError.

      Inspecting the heap dump, we see 200MB retained heap in 100.000 instances of ClientSessionFactoryImpl in the ServerLocator factories map.

       

      If I see it correctly, everytime a HornetQConnection is created, a linked sessionFactory is created. This sessionFactory is created by the ServerLocator in the ConnectionFactory, and retained in the factories map.

       

      When the connection is closed, the attached sessionFactory is closed as well, but it is not removed from the serverLocator's factories map.

      The sessionFactory does have a link to the serverLocator, and ServerLocator has a method 'factoryClosed', but this method is not used.

       

      Did you forget this?

        • 1. Re: Memory Leak in ServerLocator.factories?
          Andy Taylor Master

          which version of HornetQ are you using, does this still occur with trunk.

           

          Also, looks like your not using the correct factory, in JEE you should use the pooled factory, i.e. java:JmsXA

          • 2. Re: Memory Leak in ServerLocator.factories?
            Geert Pante Newbie

            Sorry, forgot to mention the version. We have this in HornetQ 2.2.2.Final and also with yesterday's version of Branch_2_2.

            We didn't try any other versions, but it seems that ServerLocator.factoryClosed() isn't called anywhere.

             

            We're connecting to a remote server, using a remote JNDI lookup of '/ConnectionFactory'. Can you use a pooled factory remote?

             

            It's used in a spring DefaultMessageListenerContainer managed with Spring managed transactions.

             

            http://static.springsource.org/spring/docs/2.0.x/api/org/springframework/jms/listener/DefaultMessageListenerContainer.html

            Note that this listener container will automatically reobtain all JMS handles for each transaction in case of an external transaction manager specified, for compatibility with all J2EE servers (in particular JBoss).

             

            So with a default receiveTimeout of 1 second, this creates and closes a new connection every second.

            If you close a connection, the attached clientSessionFactory should be removed from the ServerLocator automatically, I'd guess.

             

            Greets, Geert.

             

            Message was edited by: Geert Pante

            • 3. Re: Memory Leak in ServerLocator.factories?
              Andy Taylor Master

              We're connecting to a remote server, using a remote JNDI lookup of '/ConnectionFactory'. Can you use a pooled factory remote?

              Yes, just configure the resource adapter, take a look at some of the jca examples.

               

              So with a default receiveTimeout of 1 second, this creates and closes a new connection every second.

              If you close a connection, the attached clientSessionFactory should be removed from the ServerLocator automatically, I'd guess.

              creating a new connection for every message sent/received is an anti pattern, connections are designed to be long lived objects, possible memory leak aside your going to get poor peformance and possible memory problems. This is why JCA is used, it pools connections to save you creating lots of new ones.

              • 4. Re: Memory Leak in ServerLocator.factories?
                Clebert Suconic Master

                I'm fixing the factoryClosed call.

                 

                 

                If you open a JIRA.. I will close it as soon as I commit it.

                 

                 

                However, you shouldn't be creating that many sessionFactories anyway. you probably need a pool as Andy mentioned.

                • 5. Re: Memory Leak in ServerLocator.factories?
                  Geert Pante Newbie

                  OK, see HORNETQ-724

                   

                  And indeed in DefaultMessageListenerContainer, you can make it cache the Connection, so that should solve it, too.

                  • 6. Re: Memory Leak in ServerLocator.factories?
                    Leos Bitto Novice

                    Geert Pante wrote:

                     

                    And indeed in DefaultMessageListenerContainer, you can make it cache the Connection, so that should solve it, too.

                     

                    The problem is that if you configure DefaultMessageListenerContainer to cache the Connection and you use the JCA ConnectionFactory, the resulting Session might not be enlisted in the XA transaction properly. I have solved this by writing my own ConnectionFactory, which wraps the real XAConnectionFactory from the JMS provider, and creates Connections which wrap the real XAConnections and handle the caching and enlisting of the Sessions to the XA transactions.

                    • 7. Re: Memory Leak in ServerLocator.factories?
                      Andy Taylor Master

                      If you use the jca connection factory you dont have to cache it, its pooled anyway.

                       

                      Im not sure what spring does bit the jca layer will automatically add the created session to the current transaction.

                      • 8. Re: Memory Leak in ServerLocator.factories?
                        Leos Bitto Novice

                        Andy Taylor wrote:

                         

                        If you use the jca connection factory you dont have to cache it, its pooled anyway.

                         

                        Im not sure what spring does bit the jca layer will automatically add the created session to the current transaction.

                        Spring's DefaultMessageListenerContainer only starts and ends the XA transaction (when configured to use the class org.springframework.transaction.jta.JtaTransactionManager), it does not enlist the XAResource (from XASession#getXAResource()) to the running transaction - it relies on the JCA ConnectionFactory to do this. However, at least with JBoss 4.3 EAP (that's what we have used - not tried EAP 5.x yet), when the client does not call ConnectionFactory#createConnection (and caches the Connection instead), the user JMS Session does not get enlisted to the running XA transaction. That's why I have added my comment to Geert Pante's comment "in DefaultMessageListenerContainer, you can make it cache the Connection".

                         

                        Another reason why I wrote the caching myself is that I wanted to have full control over assigning of the cached Sessions with MessageConsumers to my threads, to avoid the situation when some MessageConsumers (with messages pushed from the JMS server to their client-side cache) were idling in the cache of the JCA layer and therefore adding unwanted latency to the processing of the messages.