3 Replies Latest reply on Apr 23, 2013 3:57 AM by Gareth Healy

    FTP route fails to stop

    Gareth Healy Newbie

      I have a 4 routes which have the following uri - just the "somefolder" is different per route:

       

      ftp://someone@somewhere/somefolder?password=##&initialDelay=5000&delay=60000&useFixedDelay=true&readLock=changed&sortBy=file:modified&delete=true&disconnect=true&passiveMode=true&binary=true&filter=#downloadFileFilter&recursive=true

       

      When i need to stop the bundle, it fails to stop because there are inflight exchanges. i.e.:

       

      2013-04-19 16:28:51,189 | INFO  | 0 - ShutdownTask | DefaultShutdownStrategy      | ultShutdownStrategy$ShutdownTask  523 | 119 - org.apache.camel.camel-core - 2 .10.0.fuse-71-049 | Waiting as there are still 2 inflight and pending exchanges to complete, timeout in 300 seconds.

       

      It will then reach 0 and then go into minus figures. The only way to stop this is to kill the ESB container process. The inflight messages seem to be connections to the FTP which have not timedout or been released as every now and again i see exceptions such as:

       

      org.apache.camel.component.file.GenericFileOperationFailedException: File operation failed: 421 No transfer timeout (900 seconds): closing control connection

      FTP response 421 received.  Server closed connection.. Code: 421

              at org.apache.camel.component.file.remote.FtpOperations.listFiles(FtpOperations.java:725)[243:org.apache.camel.camel-ftp:2.10.0.fuse-71-051]

              at org.apache.camel.component.file.remote.FtpConsumer.doPollDirectory(FtpConsumer.java:88)[243:org.apache.camel.camel-ftp:2.10.0.fuse-71-051]

              at org.apache.camel.component.file.remote.FtpConsumer.pollDirectory(FtpConsumer.java:51)[243:org.apache.camel.camel-ftp:2.10.0.fuse-71-051]

              at org.apache.camel.component.file.GenericFileConsumer.poll(GenericFileConsumer.java:107)[119:org.apache.camel.camel-core:2.10.0.fuse-71-049]

              at org.apache.camel.impl.ScheduledPollConsumer.doRun(ScheduledPollConsumer.java:142)[119:org.apache.camel.camel-core:2.10.0.fuse-71-049]

              at org.apache.camel.impl.ScheduledPollConsumer.run(ScheduledPollConsumer.java:92)[119:org.apache.camel.camel-core:2.10.0.fuse-71-049]

              at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)[:1.6.0_24]

              at java.util.concurrent.FutureTask$Sync.innerRunAndReset(FutureTask.java:351)[:1.6.0_24]

              at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:178)[:1.6.0_24]

              at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:165)[:1.6.0_24]

              at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:267)[:1.6.0_24]

              at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1146)[:1.6.0_24]

              at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)[:1.6.0_24]

              at java.lang.Thread.run(Thread.java:679)[:1.6.0_24]

      Caused by: org.apache.commons.net.ftp.FTPConnectionClosedException: FTP response 421 received.  Server closed connection.

              at org.apache.commons.net.ftp.FTP.__getReply(FTP.java:363)[244:org.apache.commons.net:3.1.0]

              at org.apache.commons.net.ftp.FTP.__getReply(FTP.java:290)[244:org.apache.commons.net:3.1.0]

              at org.apache.commons.net.ftp.FTP.sendCommand(FTP.java:479)[244:org.apache.commons.net:3.1.0]

              at org.apache.commons.net.ftp.FTPClient._openDataConnection_(FTPClient.java:769)[244:org.apache.commons.net:3.1.0]

              at org.apache.commons.net.ftp.FTPClient._openDataConnection_(FTPClient.java:657)[244:org.apache.commons.net:3.1.0]

              at org.apache.commons.net.ftp.FTPClient.initiateListParsing(FTPClient.java:3097)[244:org.apache.commons.net:3.1.0]

              at org.apache.commons.net.ftp.FTPClient.initiateListParsing(FTPClient.java:3072)[244:org.apache.commons.net:3.1.0]

              at org.apache.commons.net.ftp.FTPClient.listFiles(FTPClient.java:2731)[244:org.apache.commons.net:3.1.0]

              at org.apache.commons.net.ftp.FTPClient.listFiles(FTPClient.java:2778)[244:org.apache.commons.net:3.1.0]

              at org.apache.camel.component.file.remote.FtpOperations.listFiles(FtpOperations.java:718)[243:org.apache.camel.camel-ftp:2.10.0.fuse-71-051]

              ... 13 more

       

      Any idea why a connection would be held for such a long time and not released? Is there a way i can lower the shutdown strategy for these routes?

       

      Cheers.

        • 1. Re: FTP route fails to stop
          Willem Jiang Newbie

          Hi,

           

          Gracefull shutdown just want to make sure you don't lose any inflight messages.

          It looks like the FTP client still try to pull the FTP server before the timeout error is thrown. Can you show us your camel route ?

          • 2. Re: FTP route fails to stop
            Jonathan Anstey Novice

            You can't specify a shutdown timeout for each route; there is a global timeout used in the DefaultShutdownStrategy of 300 seconds. You could lower the global value or you could override the DefaultShutdownStrategy with your own. I would suggest trying to lower the timeouts as well (see http://camel.apache.org/ftp.html).

             

            Cheers,

            Jon

            • 3. Re: FTP route fails to stop
              Gareth Healy Newbie

              My route is very simple. It downloads a file, logs a message using wiretap and then writes that file to disc. The files which it attempts to download are all a few kb - i.e.: very small.

               

              <route id="downloadFileFor_uk">

                <from uri="ftp ://user @ somewhere.com/uk?password=password&initialDelay={{downloadFileFor.from.initialDelay}}

                delay={{downloadFileFor.from.delay}}&useFixedDelay=true&readLock=changed&sortBy=file:modified&delete=true&

                disconnect=true&passiveMode=true&binary=true&filter=#downloadFileFilter&recursive=true"/>

                <setHeader headerName="System" id="setHeader14">

                  <constant>uk</constant>

                </setHeader>

                <wireTap uri="log:BoohooLogESB?level=INFO&amp;showBodyType=false&amp;showExchangePattern=false" id="loggingFactory-getDownloadingMessage">

                  <body/>

                </wireTap>

                <to uri="file:///home/kinspeed/../VendaSystems/uk" id="file:///home/kinspeed/../VendaSystems/uk"/>

              </route>

               

              The filter is as follows:

               

                  private static final String ASC_EXTENSION = "asc";

                  private static final String ZIP_EXTENSION = "zip";

                  private static final String ORDERS_DOWNLOAD_FOLDER = "Order_Send_Export";

                  private static final String STOCK_RECONCILIATION_DOWNLOAD_FOLDER = "StockFeed";

                 

                  @Override

                  public boolean accept(GenericFile<T> file) {

                      String fileName = file.getFileName();

                     

                      Boolean isAccepted = false;

                      if (file.isDirectory()) {

                          isAccepted = fileName.equalsIgnoreCase(DownloadFileFilter.ORDERS_DOWNLOAD_FOLDER) || fileName.equalsIgnoreCase(DownloadFileFilter.STOCK_RECONCILIATION_DOWNLOAD_FOLDER);

                      } else {

                          String extension = FileHelper.getFileNameExtension(fileName);

                          isAccepted = extension.equalsIgnoreCase(DownloadFileFilter.ASC_EXTENSION) || extension.equalsIgnoreCase(DownloadFileFilter.ZIP_EXTENSION);

                      }

               

                      return isAccepted;

                  }