1 2 Previous Next 16 Replies Latest reply on Jan 29, 2007 11:57 AM by Kurt Stam

    File Listener Gateway Configuration

    Asa Colson Newbie

      I am trying to set up a FileListenerGateway that creates an ESB message when a new file is received. I only want to propogate a file event and not send the contents of the file in the message as it may be huge. In debugging my issues I found that the FileGatewayListener looks to the actions defined in the configuration to use as the file composer. I have defined the actions but continue to get the following at start up:

      09:28:23,427 WARN [AbstractFileGateway] No element found in cofiguration - Using default composer class : org.jboss.soa.esb.listeners.gateway.FileGatewayListener$PackageFileContents

      My configuration is as follows:

      <?xml version = "1.0" encoding = "UTF-8"?>
      <jbossesb parameterReloadSecs="50" xmlns="http://anonsvn.labs.jboss.com/labs/jbossesb/trunk/product/etc/schemas/xml/jbossesb-1.0.xsd" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">


      <fs-provider name="TestFile">
      <fs-bus busid="test-file"/>
      </fs-provider>


      <service is-gateway="true" name="FileListeningService" description="Service that listens for files coming in" category="FileEvent" target-service-category="FileEventActions" target-service-name="FileActionEvent">

      <fs-listener busidref="test-file" maxThreads="1" name="testFileListener" poll-frequency-seconds="10" >
      <fs-message-filter input-suffix="txt" directory="C:\\FileTest" post-delete="false" post-directory="C:\\FileTest\\inProcess" error-directory="C:\\FileTest\\inError"/>
      </fs-listener>





      <service is-gateway="false" name="FileActionEvent" description="Service that processes receipt of files" category="FileEventActions" >

      <fs-listener busidref="test-file" maxThreads="1" name="testFileListener" poll-frequency-seconds="10" >
      <fs-message-filter input-suffix="txt" directory="C:\\FileTest" post-delete="false" post-directory="C:\\FileTest\\inProcess" error-directory="C:\\FileTest\\inError"/>
      </fs-listener>








      Thanks,
      Asa

        • 1. Re: File Listener Gateway Configuration
          Asa Colson Newbie

          Sorry the configuration didn't go through completely:

          <?xml version = "1.0" encoding = "UTF-8"?>
          <jbossesb parameterReloadSecs="50" xmlns="http://anonsvn.labs.jboss.com/labs/jbossesb/trunk/product/etc/schemas/xml/jbossesb-1.0.xsd" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
          
           <providers>
           <fs-provider name="TestFile">
           <fs-bus busid="test-file"/>
           </fs-provider>
           </providers>
           <services>
           <service is-gateway="true" name="FileListeningService" description="Service that listens for files coming in" category="FileEvent" target-service-category="FileEventActions" target-service-name="FileActionEvent">
           <listeners>
           <fs-listener busidref="test-file" maxThreads="1" name="testFileListener" poll-frequency-seconds="10" >
           <fs-message-filter input-suffix="txt" directory="C:\\FileTest" post-delete="false" post-directory="C:\\FileTest\\inProcess" error-directory="C:\\FileTest\\inError"/>
           </fs-listener>
           </listeners>
           <actions>
           <action name="CreateFileMessage" class="com.spss.batch.esb.CreateFileMessage" process="receivedFile"/>
           </actions>
           </service>
           <service is-gateway="false" name="FileActionEvent" description="Service that processes receipt of files" category="FileEventActions" >
           <listeners>
           <fs-listener busidref="test-file" maxThreads="1" name="testFileListener" poll-frequency-seconds="10" >
           <fs-message-filter input-suffix="txt" directory="C:\\FileTest" post-delete="false" post-directory="C:\\FileTest\\inProcess" error-directory="C:\\FileTest\\inError"/>
           </fs-listener>
           </listeners>
           <actions>
           <action name="CreateFileMessage" class="com.spss.batch.esb.CreateFileMessage" process="receivedFile"/>
           </actions>
           </service>
           </services>
          </jbossesb>
          


          • 2. Re: File Listener Gateway Configuration
            Asa Colson Newbie

            Sorry in trying to figure out what I was doing wrong I had a bad configuration. Here is the proper one that causes my problem:

            <?xml version = "1.0" encoding = "UTF-8"?>
            <jbossesb parameterReloadSecs="50" xmlns="http://anonsvn.labs.jboss.com/labs/jbossesb/trunk/product/etc/schemas/xml/jbossesb-1.0.xsd" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
            
             <providers>
             <fs-provider name="TestFile">
             <fs-bus busid="test-file"/>
             </fs-provider>
             </providers>
             <services>
             <service is-gateway="true" name="FileListeningService" description="Service that listens for files coming in" category="FileEvent" target-service-category="FileEventActions" target-service-name="FileActionEvent">
             <listeners>
             <fs-listener busidref="test-file" maxThreads="1" name="testFileListener" poll-frequency-seconds="10" >
             <fs-message-filter input-suffix="txt" directory="C:\\FileTest" post-delete="false" post-directory="C:\\FileTest\\inProcess" error-directory="C:\\FileTest\\inError"/>
             </fs-listener>
             </listeners>
             <actions>
             <action name="CreateFileMessage" class="com.spss.batch.esb.CreateFileMessage" process="receivedFile"/>
             </actions>
             </service>
             <service is-gateway="false" name="FileActionEvent" description="Service that processes receipt of files" category="FileEventActions" >
             <listeners>
             <fs-listener busidref="test-file" maxThreads="1" name="anotherFileListener" poll-frequency-seconds="10" >
             <fs-message-filter input-suffix="txt" directory="C:\\FileTest\\inProcess" post-delete="false" post-directory="C:\\FileTest\\inProcess\\inProcess" error-directory="C:\\FileTest\\inProcess\\inError"/>
             </fs-listener>
             </listeners>
             <actions>
             <action name="CreateFileMessage" class="com.spss.batch.esb.CreateFileMessage" process="receivedFile"/>
             </actions>
             </service>
             </services>
            </jbossesb>
            


            • 3. Re: File Listener Gateway Configuration
              Bernard Tison Newbie

              Asa,

              You're right. The GatewayGenerator class, responsible for generating the jbossesb-gateway.xml configuration file doesn't bind the definitions to the listener definition. You can check this ion the generated jbossesb-gateway.xml file. So when configuring itself the FilegatewayListener doesn't find your custom file composer action and instead falls back on its default composer.
              I guess this is a bug. I will submit a Jira request for it.

              Bernard

              • 4. Re: File Listener Gateway Configuration
                Mark Little Master

                Bernard, I've updated your JIRA issue and marked this to be fixed in the GA release. Thanks.

                • 5. Re: File Listener Gateway Configuration
                  Kurt Stam Master

                  You are right. We don't support this right now. And I don't really understand why we need it. Why can't you wait to until after the message comes into the listener? I mean you can just drop the entire file on the gateway and then turn it into a nice message once it hits the listener?

                  Maybe I'm missing something.

                  --Kurt

                  • 6. Re: File Listener Gateway Configuration
                    Esteban Schifman Newbie

                    Kurt, if the files are huge, it could make sense to send a message just saying 'file XXX is ready for processing' without packaging huge contents inside the message.
                    The message aware action can then process the file directly.

                    Having said this, what is stopping Asa from specifying a different composer class that just packages the file name into the message.

                    • 7. Re: File Listener Gateway Configuration
                      Mark Little Master

                      Pass by reference makes sense.

                      • 8. Re: File Listener Gateway Configuration
                        Mark Little Master

                        Kurt, are you suggesting that the JIRA task Bernard registered isn't valid? This appears to relate to a configuration issue.

                        • 9. Re: File Listener Gateway Configuration
                          Tom Fennelly Master

                          Does this force the general issue on "how do we handle huge messages"??

                          Do we need a lazy load type message impl which can also page in the message.

                          I can't see how doing it by reference can work consistently - what if the gateway is on one machine and the ESB aware listener is on another?

                          • 10. Re: File Listener Gateway Configuration
                            Esteban Schifman Newbie

                            It doesn't force anything. There are mechanisms for ftp transfers that will handle the situation Tom's talking about.

                            • 11. Re: File Listener Gateway Configuration
                              Mark Little Master

                               

                              "tfennelly" wrote:
                              Does this force the general issue on "how do we handle huge messages"??

                              Do we need a lazy load type message impl which can also page in the message.

                              I can't see how doing it by reference can work consistently - what if the gateway is on one machine and the ESB aware listener is on another?


                              Chunking or streaming of large messages across a network is fairly well understood (a lot of grid applications do this extremely well). It often requires an additional hand-shake protocol between the endpoints in order to negotiate the flow and throttle it back/forward dynamically over time. The current architecture doesn't support it because it's predominately by-value. The only way of doing this at the moment would be to pass by reference and then punt to the higher level to determine how the streaming of the actual message happens. That certainly puts more of an effort on the higher level developer, but at the moment there is little choice. We should look at any architecture changes for after the 4.0 GA.

                              • 12. Re: File Listener Gateway Configuration
                                Mark Little Master

                                 

                                "estebanschifman" wrote:
                                It doesn't force anything. There are mechanisms for ftp transfers that will handle the situation Tom's talking about.


                                Yes, FTP is a good way of doing this currently. However, if you can't use FTP then other changes will be necessary.

                                • 13. Re: File Listener Gateway Configuration
                                  Tom Fennelly Master

                                   

                                  "estebanschifman" wrote:
                                  It doesn't force anything. There are mechanisms for ftp transfers that will handle the situation Tom's talking about.


                                  Sure, "force" was too strong - there are prob ways of getting around the issue in the short term. It's a problem we're going to have to face some time in the future and I don't suspect it's going to be a piece of cake.

                                  • 14. Re: File Listener Gateway Configuration
                                    Kurt Stam Master

                                    Colsona,

                                    Are you doing this because your files are big?

                                    --Kurt

                                    1 2 Previous Next