4 Replies Latest reply on Apr 26, 2013 6:51 AM by sekaijin

    FTP Consumer Template thread problem.

    sekaijin

      I'm running Karaf 2.2.2 Camel 2.8.1 on Windows, Linux and HPUX

       

      I've a route that recieve HL7 messages via MLLP.

      each message content a file name.

      My route content a processor that intenciate a ftp consumer template for retriving only this file.

      like

      *ConsumerTemplate consumer = context.createConsumerTemplate();

      consumer.start();

      Exchange out = consumer.receive(myFtpUrlForMyFile);

      consumer.stop();*

       

      I'va added

      context.removeEndpoints(myFtpUrlForMyFile);

      to be sure that the endpoint are destroyed.

       

      For each message received by MLLP a FTP end point and a thead are created.

      the endpoint are destroyed, and i can agreagte the hl7 message and the file.

      but the tread are not destroyed

       

      I'm receive a bilions of messages and the thread number crach the JVM.

      I'm not sure but I think the max thread number is too large on camel defalut conf. on HPUX the max number of thread by process (JVM) is ~3000

       

      I've read the camel doc on thread pool architecture. I've read that is possible to configure it.

      I've seen a lite xml fragment that configure it but not ware I can put this frangment (doese not run in blueprint.xml)

       

      I'v seen that i can get a threadpoolservice on contexte but in confiure route on my routebuilder when I call getContext() the object returned doese not support any methods for this.

       

      My bundle is définied using Blueprint and the route is défined on java DSL on a routeBuilder.

       

      thank for your help.

      Bye JYT

      Ps: sorry for my approximative english

        • 1. Re: FTP Consumer Template thread problem.
          davsclaus
          • 2. Re: FTP Consumer Template thread problem.
            sekaijin

            thank for this response but

             

            My problem comes with a consumer template and not a producer template.

            My problem is not to create a "multi thread" consumer template .

             

            my processor code is

            public void process(Exchange exchange) throws Exception {

               ...

               ConsumerTemplate con = exchange.getContext().createConsumerTemplate();

            //retrive the file associated with message

            //one thread is created, and one EndPoint is created

               exTdd = con.receive(urlFinal"fileName="name, TIME_OUT);

               if (null != exTdd) {

            //enrich the current exchange with the downloaded file

                  *... *

            //I'm not sure con.doneUoW(exchange); or con.doneUoW(exTdd);

                  con.doneUoW(exchange);

               }                  

            //stop the consummer template

               con.stop();

            //remove the endpoint

                

            • exchange.getContext().removeEndpoints((urlFinal"fileName="name)*

            // the endpoint is deleted. but the thread remains

            }

             

            Bye

            • 3. Re: FTP Consumer Template thread problem.
              davsclaus

              The same idea applies for consumer template. Create it once, and keep it around. And stop it when you stop your app.

              • 4. Re: FTP Consumer Template thread problem.
                sekaijin

                thank but

                 

                I can't stop and restart my application every time I get a message. of thousand times a day.

                 

                I created A single template in my context. forevery message I get, I must create a different endpoint, my url depend on the content of the message.

                 

                 

                I create a different endpoint, so a thread every time. I stop the endpoint and removes it after I received the content. but the thread is still not released.

                 

                 

                public void process(Exchange exchange) throws Exception {

                ...

                ConsumerTemplate con = getStaticConsumerTemplate();

                //retrive the file associated with message

                //one thread is created, and one EndPoint is created

                exTdd = con.receive(urlFinal"fileName="name, TIME_OUT);

                if (null != exTdd) {

                //enrich the current exchange with the downloaded file

                ...

                //I'm not sure con.doneUoW(exchange); or con.doneUoW(exTdd);

                con.doneUoW(exchange);

                }

                //stop the consummer template

                con.stop();

                //remove the endpoint

                exchange.getContext().removeEndpoints(urlFinal"fileName="name)

                // the endpoint is deleted. but the thread remains

                }

                 

                 

                I've always creates a thread per message received and the JVM crashes because it exceed the cota of thread per process, supported by the OS.