2 Replies Latest reply on Nov 29, 2011 4:18 AM by grahamp

    Memory consumption

    grahamp
    grahamp Nov 28, 2011 7:05 AM

      We are using ESB to integrate systems that need intermittent file transfers.  File is usually small (0 - 5Kb) but occasionally large (2 - 4 Mb).  In the route this could transform into a 6Mb message which we then send to another system via a web API.

       

      Looking at the ubuntu system monitor, we see a single such file causing the ESB to go from 140Mb idle to 850Mb busy.  Or worse .. sometimes out-of-memory exception on a 1Gb JVM.  Why is camel routing consuming so much memory?  Only simple nodes in the route, no queues:

      - Processor to convert csv to xml

      - validate against an xsd

      - transform with an xslt

      - Processor to write to the OutputStream of a web-api post operation

      - Processor to check the response from the post operation

       

      We could understand a few 6Mb copies being generated during the exchange, but 700Mb overhead is a shock.  Would we gain anything by splitting the route up with file endpoints?

      • 104 Views
      • Tags:
        • 1. Re: Memory consumption
          njiang
          njiang Nov 28, 2011 8:47 PM (in response to grahamp)

          Which version of Fuse ESB are you using?

          What kind of Web API are you using?

           

          We need more information to dig the issue.

            • Actions
          • 2. Re: Memory consumption
            grahamp
            grahamp Nov 29, 2011 4:18 AM (in response to njiang)

            We are using servicemix 4.3.1 and observe the behaviour in both versions fuse-03-01 (on ubuntu) and fuse-02-05 (windows server 2008).  Currently there is no web api in the bundle at all.  The routes simply pick up from network file areas.  They all start with something like

              from("file:////vmtest/esb_in")

             

            The increasing memory load happens with one file on one route - everything else idle.

              • Actions