5 Replies Latest reply on Apr 27, 2007 11:28 AM by marklittle

    How do you avoid a listener loop processing?

      Hi,

      Suppose I have a FTP listener, configured to listen to ftp://abcprovider.net. The provider puts there a file, abc.xml. I have an username/password for an user that has only read only access to that file.

      How do you avoid a loop in the listener processing in this scenario?
      As I understand, a listener will scan the "folder" every x seconds, and if it finds any files with the given pattern (ex: *.xml) will try to process them. The way to avoid a multiple processing is to rename or delete the file after it was processed.

      More important, suppose that the file abc.xml is needed by 2 or more clients, and I have to send this file to all my clients, each with it's own "notifier". How do you implement such a feature in JBoss ESB?


      The same thing with a WS listener, how do you identify that you have already called the WS?

        • 1. Re: How do you avoid a listener loop processing?
          kconner

          At present the gateway assumes that write access is available for the user as the transitions in the processing are reflected in renames/deletes of the file on the ftp server. In this way it is treated in a similar fashion to a JMS queue rather than a JMS topic.

          It should be relatively straight forward to write a new gateway to meet your requirements, certainly where processing the message is concerned. You could key this on the name or modification time of the file.

          There is one drawback however. If the file is never deleted/renamed then each query to the FTP server will return information that the client will have to ignore (and this could become large). Some of this processing can be mitigated, for example querying the modification time on the directory, but in the end you will always return all matching filenames, whether processed or not.

          "anescu wrote:
          More important, suppose that the file abc.xml is needed by 2 or more clients, and I have to send this file to all my clients, each with it's own "notifier". How do you implement such a feature in JBoss ESB?

          If you are restricted to using FTP then you would need a copy of the file for each client (using the current gateway). This could be achieved by creating a separate directory for each.

          To give a better answer we would need to understand more about your requirements and restrictions. Is it possible to use JMS topics?

          "anescu" wrote:
          The same thing with a WS listener, how do you identify that you have already called the WS?

          Not sure what you mean by this. We normally call a webservice as a consequence of processing an ESB message. Individual messages would trigger an associated calls.


          • 2. Re: How do you avoid a listener loop processing?

            Thanks for the reply.

            We are trying to analyze if JBoss ESB is fit for the requirments of the project.

            We will have various providers (email, FTP, WS, etc).
            And clients. A client will define a chain of actions to do between the inbound gateway and the outbound gateway (notifier).
            So 2 or more clients may very well define the same input as the starting point (for example you get 5 clients that all work with company X)
            Now suppose company X is a large distributor, that uses FTP/WS to allow partners to retrieve the price list for the next month. As I stated in the first post, X will probably put the file somewhere in one folder, and give each partner the same username/account (or different), but with read only access (and if he has some few hundreds partners, it's more likely to create one file for each, not one for every partner). Or he could provide a WS that would return the file, the same for each query.
            For other types of gateway, the "message" is usually consumed by the process (JMS queue, email for example), but here the "message" would be there after each process, so how do you stop? To me this situation seems like an infinite loop.
            Hope to make sense to you...

            • 3. Re: How do you avoid a listener loop processing?
              marklittle

              If you don't delete the message then it will be continually re-processed each time with the current out-of-the-box implementations. Your best bet would be to implement your own custom code that keeps a record of which files have been processed and builds up an "ignore" list as it processes and scans. Presumably you have some out-of-band garbage collection in mind for removing such files once all clients have done with them?

              • 4. Re: How do you avoid a listener loop processing?

                Hi,

                Thanks, I reached the same conclusion, you would need custom code to check already processed files. Today we talked even about checking if a file was sent twice, so it would fall under the same category. So it seems to be a "custom gateway" situation. Although it would be probably easier to just stop the message flow for such a message?

                • 5. Re: How do you avoid a listener loop processing?
                  marklittle

                  How are you thinking about garbage collecting these files?