5 Replies Latest reply on May 13, 2009 10:38 AM by desbonns62

    Fs-provider line by line

      Hello,

      Is there a way to, instead of sending the whole data contained in a file, sending it throw the ESB line by line ?

      I have a fs-provider and my action is to route data to a JMS consumer. I am using big files so my JMS messages are very big.
      I think my solution can start with the smooks_file_splitter_router quickstart but it doesn't work without the smooks transformation (I suppose I don't need this transformation).

      Thanks in advance for your help.

        • 1. Re: Fs-provider line by line

          No ideas ?

          Sorry for my English, thank you to read "through the ESB" instead of "throw the ESB"...

          In my opinion, this seems to be a basic transformation for an ESB but I didn't found it in Jboss yet. Or maybe there is a best practices for doing this in another way.

          Thanks again.

          • 2. Re: Fs-provider line by line

            Salut !

            Je n'ai pas vu qqch permettant de lire avec le FS-Provider ligne par ligne un fichier.
            (I didn't saw soemthing to read a file on per line basis using the fs-provider)

            Soit tu passes la reference a Smooks comme dans l'exemple huge message, soit tu lis le fichier d'un bloc et tu le split ensuite (ce que fais l'exemple que tu cites si ma memoire est bonne), ce qui en soit reviens a transformer ton message d'origine.
            (You may pass the file's reference to Smooks, like in the huge message example, or you can read the file in one pass and then split it (what does the example you router-splitter if my memory isn't too bad), so in this case it's a transformation of your original message.)

            • 3. Re: Fs-provider line by line

              Thanks for your answer Cedric,

              I already tried both methods that you recommend.
              I didn't managed to use smooks transformation for splitting at the end of random line (not xml)...
              And when I tried the "huge quickstarts" I got the full contents in one message instead of one message by line.

              I think I will try again the second method...

              • 4. Re: Fs-provider line by line

                Some news...

                I tried to manage it with smooks but I a have a problem.

                With the following configuration I can route single message (single line) to another esb service. This works with smooks 1.2 (download and replace jars in smooks.esb)
                I am using the csv reader and do like the csv file contains only one field (=> I process the entire row).

                But I still have one problem: if my file contains 20 lines, only 10 lines are processed by smooks (one line is skipped and one line is processed...)
                I don't know if the problem comes from my jboss config or my smooks config.

                Here is my jboss config (2 services):

                <service category="esb_services" name="SplitterService"
                 description="Splits out the csv lines and routes them.">
                 <listeners>
                 <fs-listener name="FileGateway" busidref="JMSTransformChannel_fs"
                 is-gateway="true" poll-frequency-seconds="2">
                 <property name="composer-class"
                 value="org.jboss.soa.esb.smooks.splitting.FileStreamSplitter" />
                 <property name="splitterConfig" value="/smooks-config2.xml" />
                 </fs-listener>
                 <jms-listener busidref="JMSTransformChannel_esb1" name="JMSTransformListener_esb1" />
                 </listeners>
                 <actions>
                
                 <!--Is it possible to have an esb service without actions ?-->
                 <action name="action1" class="org.jboss.soa.esb.actions.SystemPrintln">
                 <property name="printfull" value="false"/>
                 </action>
                 </actions>
                 </service>
                
                 <service category="esb_services" name="RouteMessage" description="Routes the message">
                 <listeners>
                 <jms-listener busidref="JMSTransformChannel_esb1" name="JMSTransformListener_esb1" />
                 </listeners>
                 <actions mep="OneWay">
                 <action class="org.jboss.soa.esb.actions.routing.JMSRouter"
                 name="RouteMessage">
                 <property name="connection-factory" value="ConnectionFactory" />
                 <property name="jndiName" value="queue/jms_transform_consumer" />
                 <property name="unwrap" value="true" />
                 <property name="persistent" value="false" />
                 </action>
                 </actions>
                 </service>



                and my smooks config (the one I uses with smooks 1.2):

                <smooks-resource-list xmlns="http://www.milyn.org/xsd/smooks-1.1.xsd"
                 xmlns:csv="http://www.milyn.org/xsd/smooks/csv-1.2.xsd"
                 xmlns:file="http://www.milyn.org/xsd/smooks/file-routing-1.1.xsd"
                 xmlns:ftl="http://www.milyn.org/xsd/smooks/freemarker-1.1.xsd">
                
                 <!-- Use the SAX filter - more memory efficient and handle huge streams. -->
                 <params>
                 <param name="stream.filter.type">SAX</param>
                 </params>
                
                 <csv:reader fields="value" rootElementName="rootElmt" recordElementName="csvRecord">
                 <csv:singleBinding beanId="csvRecord" class="java.util.HashMap"/>
                 </csv:reader>
                
                 <ftl:freemarker applyOnElement="csvRecord">
                 <ftl:template>
                 <!--${csvRecord.value}--></ftl:template>
                 <ftl:use>
                 <ftl:bindTo id="theLine" />
                 </ftl:use>
                 </ftl:freemarker>
                
                 <resource-config selector="csvRecord">
                 <resource>org.jboss.soa.esb.smooks.FragmentRouter</resource>
                 <param name="beanId">theLine</param>
                 <param name="serviceCategory">esb_services</param>
                
                 <param name="serviceName">RouteMessage</param>
                 </resource-config>
                 </smooks-resource-list>
                


                Briefly the scenario:
                SplitterService(ESB service) ----> FileStreamSplitter (smooks) ---> route one line of the csv file (smooks) ----> RouteMessage(ESB service) (send jms message)

                Is it a known problem? Do you have any idea?
                Do not hesitate to ask me for more info if needed.

                Regards
                Simon


                • 5. Re: Fs-provider line by line

                  Sorry for the last message, there is no problem (it was my fault, I just didn't catch all messages).
                  I think it is possible to do better code and config but it works fine for me...

                  Here are my config files for those who want to do such things.
                  I tried it with smooks 1.1 embedded in Jboss ESB 4.3.

                  smooks-config.xml:

                  <?xml version="1.0"?>
                   <smooks-resource-list xmlns="http://www.milyn.org/xsd/smooks-1.1.xsd"
                   xmlns:csv="http://www.milyn.org/xsd/smooks/csv-1.1.xsd"
                   xmlns:file="http://www.milyn.org/xsd/smooks/file-routing-1.1.xsd"
                   xmlns:ftl="http://www.milyn.org/xsd/smooks/freemarker-1.1.xsd"
                   xmlns:jb="http://www.milyn.org/xsd/smooks/javabean-1.1.xsd">
                  
                  
                   <!--
                   Use the SAX filter - more memory efficient and handle huge streams.
                   -->
                   <params>
                   <param name="stream.filter.type">SAX</param>
                   </params>
                  
                   <!-- Read the file and creates an xml such as
                   <csv-set>
                   <csv-record>
                   <value> (your file line) </value>
                   ...
                   </csv-record>
                   </csv-set> -->
                   <csv:reader fields="value" />
                  
                   <!-- Bind the element csv-record to the id 'record' and the value element to the id 'line' -->
                   <jb:bindings beanId="record" class="java.util.HashMap" createOnElement="csv-record">
                   <jb:value property="line" data="csv-set/csv-record/value" />
                   </jb:bindings>
                  
                   <!-- Will be processes for each csv-record element (the output template contains one line)-->
                  
                   <ftl:freemarker applyOnElement="csv-record">
                   <!-- <ftl:template>template.ftl</ftl:template> -->
                   <ftl:template> <!--${record.line}-->
                  
                   </ftl:template>
                   <ftl:use>
                   <ftl:bindTo id="theLine" />
                   </ftl:use>
                   </ftl:freemarker>
                  
                   <!-- Route the line to another service -->
                  
                   <resource-config selector="csv-record">
                   <resource>org.jboss.soa.esb.smooks.FragmentRouter
                   </resource>
                   <param name="beanId">theLine</param>
                   <param name="serviceCategory">esb_services</param>
                   <param name="serviceName">DestinationService</param>
                  
                   </resource-config>
                   </smooks-resource-list>
                  
                  

                  and my jboss services:

                  <services>
                   <service category="esb_services" name="SplitterService"
                   description="Splits out the order items and routes them.">
                  
                   <listeners>
                   <fs-listener name="FileGateway" busidref="JMSTransformChannel_fs"
                   is-gateway="true" poll-frequency-seconds="2">
                   <property name="composer-class"
                   value="org.jboss.soa.esb.smooks.splitting.FileStreamSplitter" />
                   <property name="splitterConfig" value="/smooks-config.xml" />
                  
                   </fs-listener>
                   <jms-listener busidref="JMSTransformChannel_esb1" name="JMSTransformListener_esb1" />
                   </listeners>
                   <actions>
                   <action name="print" class="org.jboss.soa.esb.actions.SystemPrintln">
                  
                   <property name="printfull" value="false" />
                   <property name="message" value="[- SplitterService -] " />
                   </action>
                   </actions>
                   </service>
                  
                   <service category="esb_services" name="DestinationService"
                  
                   description="Routes the message">
                   <listeners>
                   <jms-listener busidref="JMSTransformChannel_esb2" name="JMSTransformListener_esb2" />
                  
                   </listeners>
                   <actions mep="OneWay">
                   <action class="org.jboss.soa.esb.actions.routing.JMSRouter"
                   name="RouteMessage">
                   <property name="connection-factory" value="ConnectionFactory" />
                   <property name="jndiName" value="queue/jms_transform_consumer" />
                   <property name="unwrap" value="true" />
                   <property name="persistent" value="false" />
                   </action>
                   </actions>
                   </service>
                   </services>


                  And if you need more info about smooks you could find a lot of info here:
                  http://www.nabble.com/milyn---user-f11972.html

                  Simon