2 Replies Latest reply on Nov 29, 2011 4:18 AM by Graham Perkins

    Memory consumption

    Graham Perkins Newbie

      We are using ESB to integrate systems that need intermittent file transfers.  File is usually small (0 - 5Kb) but occasionally large (2 - 4 Mb).  In the route this could transform into a 6Mb message which we then send to another system via a web API.

       

      Looking at the ubuntu system monitor, we see a single such file causing the ESB to go from 140Mb idle to 850Mb busy.  Or worse .. sometimes out-of-memory exception on a 1Gb JVM.  Why is camel routing consuming so much memory?  Only simple nodes in the route, no queues:

      - Processor to convert csv to xml

      - validate against an xsd

      - transform with an xslt

      - Processor to write to the OutputStream of a web-api post operation

      - Processor to check the response from the post operation

       

      We could understand a few 6Mb copies being generated during the exchange, but 700Mb overhead is a shock.  Would we gain anything by splitting the route up with file endpoints?