13 Replies Latest reply on Oct 3, 2011 1:07 AM by bfp

    JBossWS 3.0.3 memory leak

    marlboro

      Hi,

      We are running an EJB3/JBossWS aplication on JBoss 4.2.2 with JBossWS - native 3.0.3. Our application causes JBoss to crash with "out of memory error" under load, and we have memory dump from Java 1.6. After analyze of this dump we suspect JBossWS to cause the memory leak. There are 22942513 instances of class java.util.HashMap$Entry with 367080208 bytes and 58967 instances of class [Ljava.util.HashMap$Entry; with 231301484 bytes. Both HashMap entries contain some data from data type's definitions of our endpoints.

      For better understating: our application has two parts. At this moment, both parts are deployed on one server but in the future it will be possible to deploy them on two distinct servers. One part is web service client, second one is web service endpoint. We tested our application with Java 1.5 and also 1.6 but we are able to obtain memory dump only with Java 1.6 . Nevertheless both JVMs have the same behavior - they crash approximately after 48 hours.

      Any help will be greatly appreciated. If you will need more data/infos please feel free to contact me.

      Thanks,
      Vratislav Kuzela

        • 1. Re: JBossWS 3.0.3 memory leak
          asoldano

          For sure more info would help, perhaps further details on the webservice endpoint(s), the contract and the full memory dump.

          • 2. Re: JBossWS 3.0.3 memory leak
            marlboro

            Hi Alessio,

            I'm sorry for the delay. The endpoints are part of web layer of our application. I will send you both wsdl files via email, and also I will send you link to the web page with JHat result because the memory dump is 1.17GB large. Do you need more data or these are sufficient?

            Thanks,
            Vratislav Kuzela

            • 3. Re: JBossWS 3.0.3 memory leak
              manoj.kumar.jain

              I am also facing the similar problem.
              HashMapEntries are growing and those have reference to resourceblacklist of org.jboss.mx.loading.InifiedClassLoaderRepository3.

              Further investigation shows it is trying to holding the Schema (.xsd) for WebService defination of type-
              ....../JbossWS_jaxb.dev.java.net_array10100 .xsd

              Any help will be highly appriciated

              • 4. Re: JBossWS 3.0.3 memory leak
                ropalka

                Switch to JBoss AS 5.0.0.GA. This problem shouldn't be there.

                • 5. Re: JBossWS 3.0.3 memory leak
                  marlboro

                  Now I'm trying to use 5.0.0 and I expect to have results next week.
                  I want to ask if there is any record in Jboss Jira which is refers to our problem?

                  Thank you!

                  • 6. Re: JBossWS 3.0.3 memory leak
                    ecbdc

                    Hi Marlboro, were you able to resolve the issue by upgrading to JBoss 5.0?

                    I'm having a similar issue w/ the web services memory leak.

                    Thanks

                    • 7. Re: JBossWS 3.0.3 memory leak

                      I am having the same issue with 4.2.2 with WS 2.0.1 SP1.. I tried with JBoss 5.0.0 GA and I see many warning messages (EJBTHREE-1337, EJBTHREE-1246). Also looks threads are getting blocked when I run under the load. Is anyone able to find the solution or work arounds?

                      Thanks,
                      Venkat

                      • 8. Re: JBossWS 3.0.3 memory leak
                        ropalka

                        And AS 5.0.1 GA + JBossWS 3.1.0?

                        • 9. Re: JBossWS 3.0.3 memory leak
                          sintetik7

                           

                          "richard.opalka@jboss.com" wrote:
                          And AS 5.0.1 GA + JBossWS 3.1.0?


                          I had similar problems wtih AS 5.0.1 GA (see http://www.jboss.org/index.html?module=bb&op=viewtopic&p=4233605). Found some kinda solution but not sure if good one.



                          • 10. Re: JBossWS 3.0.3 memory leak

                            Hello!

                            We are running into the same problem. The JVM runs out of memory and heap analysis shows huge HashSets containing entries like "JBossWS_..._20342.xsd". Following the path to GC root shows that resourceBlackList in the unified classloader holds references to these hashmaps.

                            We are running JBoss AS 4.2.2.GA with JBossWS 3.0.1-native-2.0.4.GA and cannot upgrade to JBoss AS 5, so more information on this problem would be greatly appreciated.

                            • 11. Re: JBossWS 3.0.3 memory leak

                              I found a poller method in our code that created new instances of WS client objects in the poller method. Even though these were not retained they apparently added data to the classloader's (unifiedclassloader3) resourceBlackList.

                              • 12. Re: JBossWS 3.0.3 memory leak
                                hedenberg

                                Im running Jboss 4.2.3 in a production environment and have this very same issue due to creating vast amounts of new JAXWS clients. I have a quite lock process to go through to upgrade the production environment, but I must do something to get rid of the restarts every 48hrs.

                                 

                                Is there any way to patch JBossWS in Jboss 4.2.3 or make a workarround to avoid this issue? The only one I can think of is a Pool of WS clients for my system, but that's no tiny fix that you just throw into a prod environment.

                                 

                                Anyone got any ideas or further info about this error?

                                • 13. Re: JBossWS 3.0.3 memory leak
                                  bfp

                                  Hello,

                                  I am experiencing a similar issue with JBoss 4.2.3 and JBossWS 3.1.0. By upgrading the JBossWS to 3.1.1 was fixing the leak but introduced a side effect with null pointer under load.

                                  Does anyone has a proper recipe for those issues without the change of JBoss AS 4.2.3 ?

                                   

                                  For more details please refer the below thread: http://community.jboss.org/thread/171831