1 2 Previous Next 16 Replies Latest reply on Jul 31, 2007 9:20 AM by asachde

    Transform and persist from queue to filesystem

      How can you read a message from a queue, apply smooks transformation to the message and persist to a file using filesystem provider? I see the reverse is shown in sample but short of navigating the config tree and finding properties of the filesystem provider and writing to the filesystem from within a listener is there a jBoss ESB elegant way of doing so?

      Thanks in advance.

        • 1. Re: Transform and persist from queue to filesystem
          tfennelly

          So I think your question is mostly re persisting a message to file, right?

          If so, there's a "org.jboss.soa.esb.notification.NotifyFiles" action class. Believe it or not, I'm not sure if it works or not coz there are no quickstarts on it :-( Hard to believe we haven't been pushed on this one. If you could give it a whirl and let us know how you get on :-)

          I assume you can work the transformation end from the "transform_*" quickstarts, yeah?

          • 2. Re: Transform and persist from queue to filesystem

            Ypu... the example I am trying to work on is a message or series of messages that get sent from an application that eventually generates a file (different format than the message). The file and a checksum file (at the end of processing) gets ftp'ed to a remote site or partner. I'll give it a whirl and post my logic.

            Thanks.

            • 3. Re: Transform and persist from queue to filesystem
              burrsutter

              Adding a quickstart that shows all the notify options is on my "todo" list but it is a low priority concern at the moment.

              Worst case scenario is write a little bit of java code to write a file to the disk.

              • 4. Re: Transform and persist from queue to filesystem
                tcunning

                I've added a QS for NotifyFiles into the trunk. It uses a StaticRouter and it splits a message, and displays it both in the console and persists it to a file :

                Steps to installing :
                Either check out the trunk, or in the "samples/quickstarts" folder of your jbossesb-4.2.MR3, do the following :

                svn co http://anonsvn.labs.jboss.com/labs/jbossesb/trunk/product/samples/quickstarts/helloworld_file_notifier

                Hope it helps.

                • 5. Re: Transform and persist from queue to filesystem
                  burrsutter

                  Sweet!

                  And apparently the feature works! :-)

                  Thanks Tom.

                  • 6. Re: Transform and persist from queue to filesystem

                    Couple of follow-up questions::

                    1) When I run the example I get binary content in the output file if the listener is not a gateway. How do you deserialize the content from the Message object to write to the output file?

                    2) I am trying to do a on the fly transformation and then write the transformed result to the file but the untransformed content keeps getting written to the file. Maybe I am doing something wrong with the smooks configuration. See config below:

                     <service category="ESB_INIT" name="InitInputListener"
                     description="Init and Input Listener">
                     <listeners>
                     <fs-listener name="FileGateway"
                     busidref="FileChannel" maxThreads="1" is-gateway="true"
                     poll-frequency-seconds="10" />
                     <jms-listener name="semsInitListener"
                     busidref="semsInitChannel" maxThreads="1" />
                     </listeners>
                     <actions>
                     <action name="initScheduler"
                     class="scheduler.InterfaceInitializer"
                     process="initInterface" />
                     </actions>
                     </service>
                     <service category="ESB_PROCESS"
                     name="SEMS_PROCESS_LISTENER" description="Sems out listener">
                     <listeners>
                     <jms-listener name="semsInListener"
                     busidref="semsInChannel" maxThreads="1" is-gateway="true" />
                     <jms-listener name="semsOutListener"
                     busidref="semsOutChannel" maxThreads="1" />
                     </listeners>
                     <actions>
                     <action name="transform"
                     class="org.jboss.soa.esb.actions.converters.SmooksTransformer">
                     <property name="resource-config"
                     value="/smooks-res.xml" />
                     </action>
                     <action name="print-after-tranform"
                     class="org.jboss.soa.esb.actions.SystemPrintln">
                     <property name="message"
                     value="[transform_Smooks_Intermediate_format] Message after XML transformation" />
                     </action>
                     <action name="notificationAction"
                     class="org.jboss.soa.esb.actions.Notifier">
                     <property name="okMethod" value="notifyOK" />
                     <property name="notification-details">
                     <NotificationList type="ok">
                     <target class="NotifyConsole" />
                     <target class="NotifyFiles">
                     <file append="true"
                     URI="c:/tmp/output/results.log" />
                     </target>
                     </NotificationList>
                     <NotificationList type="err">
                     <target class="NotifyConsole" />
                     <target class="NotifyFiles">
                     <file append="false"
                     URI="c:/tmp/err/error.log" />
                     </target>
                     </NotificationList>
                     </property>
                     </action>
                    
                     </actions>
                     </service>
                    


                    Thanks,
                    Amit

                    • 7. Re: Transform and persist from queue to filesystem
                      burrsutter

                      I believe the challenge is that the transformer places the transformed version of the message in a different location than the original input.

                      Try this option to see if you can spot where the transformed content is:

                       <action name="dump" class="org.jboss.soa.esb.actions.SystemPrintln">
                       <property name="printfull" value="true"/>
                       </action>
                      


                      • 8. Re: Transform and persist from queue to filesystem

                        This is what I see in the output... Looks like the defaultEntry is the correct (transformed) xml ... but not the one picked up by the next action in the pipeline.

                        [ message: [ JBOSS_XML ]
                        [ header: [ To: EPR: PortReference < <wsa:Address jms://localhost/queue/sems_out/>, <wsa:ReferenceProperties jbossesb:java.naming.factory.initial : org.jnp.interfaces.NamingContextFactory/>, <wsa:ReferenceProperties jbossesb:java.naming.provider.url : jnp://127.0.0.1:1099/>, <wsa:ReferenceProperties jbossesb:java.naming.factory.url.pkgs : org.jboss.naming:org.jnp.interfaces/>, <wsa:ReferenceProperties jbossesb:specification-version : 1.1/>, <wsa:ReferenceProperties jbossesb:connection-factory : ConnectionFactory/> >, From: null, ReplyTo: null, FaultTo: null, Action: null, MessageID: 49cdd436-2f26-4126-aaa5-b013e9bf828b, RelatesTo: null ]
                        context: [ null ]
                        body: [ byte[]: <Service>
                         <Location org="fam" ou="dhs">
                         <action inittype="interface">
                         <interface name="sems"/>
                         </action>
                         </Location>
                        </Service>, objects: {defaultEntry=<Service><org>fam</org><ou>dhs</ou><action><inittype>interface</inittype><interface>sems</interface></action></Service>, org.jboss.soa.esb.message.transport.jms.messageID=ID:3-11857578528021, EXTRACTED_BEANS_HASH={}, org.jboss.soa.esb.actions.current.after=<Service><org>fam</org><ou>dhs</ou><action><inittype>interface</inittype><interface>sems</interface></action></Service>} ]
                        fault: [ null, null ]
                        attachments: [ Named:{}, Unnamed:[] ]
                        properties: [ {org.jboss.soa.esb.message.time.dod=Sun Jul 29 21:10:55 EDT 2007, org.jboss.soa.esb.message.time.dob=Sun Jul 29 21:10:55 EDT 2007, org.jboss.soa.esb.message.source=PortReference < jms://localhost/queue/sems_out >, org.jboss.soa.esb.message.transport.type=JMS, JMSXDeliveryCount=1, org.jboss.soa.esb.gateway.original.queue.name=queue/sems_input} ] ] ]
                        


                        • 9. Re: Transform and persist from queue to filesystem

                          FYI I added the NotifyFiles in the quickstarts CSV2XML and had the same effect. The CSV contents got written to the file not the XML contents. Maybe I am missing something...

                          • 10. Re: Transform and persist from queue to filesystem
                            tfennelly

                            Sorry, some of the docs on this are a little out of date. I've updated the class Javadoc as well as the MessageActionGuide. The following is the relevant extract:

                            This action gets the transformation input, and sets the transformation output based on the "input-location" and "output-location" configuration properties.
                            These properties name the {@link Body.Message.Body} location where the transformation input and output are attached.

                            If either these properties are not set, the action class defaults that value to being "defaultEntry". In other words,
                            if "input-location" is not configured on the action, the action will attempt to load the transformation input from the {@link Body Message.Body} location named "defaultEntry".
                            If the "output-location" is not configured on the action, the action will set the transformation result/output in the {@link Body Message.Body} location named "defaultEntry".

                            From a Java code perspective, setting and getting on the "defaultEntry" is done simply by calling the no-name-arg
                            {@link Body#add(Object)} and {@link Body#get()} methods.


                            • 11. Re: Transform and persist from queue to filesystem

                              That makes sense... I could get the contents by using the Body#get method from the action after the transformation in the action pipeline.

                              Sorry for the nag... maybe there is a quickstart where this is done. I am still having issues getting this to work.

                              I did configure the output-location (to) to a service that was newly defined. I moved the filenotification to that new service. However when I do a dump of printfull the body still has the old message and the new message is in the defaultEntry.

                              <action name="transform"
                               class="org.jboss.soa.esb.actions.converters.SmooksTransformer">
                               <property name="resource-config"
                               value="/smooks-res.xml" />
                               <property name="from" value="ESB_PROCESS:SEMS_PROCESS_SERVICE"/>
                               <property name="to" value="ESB_OUT:SEMS_OUT_SERVICE"/>
                               </action>
                               <action name="routeAction"
                               class="org.jboss.soa.esb.actions.StaticRouter">
                               <property name="destinations">
                               <route-to service-category="ESB_OUT"
                               service-name="SEMS_OUT_SERVICE" />
                               </property>
                               </action>
                              


                              New service definition:
                              <service category="ESB_OUT" name="SEMS_OUT_SERVICE" description="Service to write to log file and audit contents">
                               <listeners>
                               <jms-listener name="semsAuditListener"
                               busidref="semsAuditChannel" maxThreads="1" />
                               </listeners>
                               <actions>
                               <action name="dump" class="org.jboss.soa.esb.actions.SystemPrintln">
                               <property name="printfull" value="true"/>
                               </action>
                               <action name="notificationAction"
                               class="org.jboss.soa.esb.actions.Notifier">
                               <property name="okMethod" value="notifyOK" />
                               <property name="notification-details">
                               <NotificationList type="ok">
                               <target class="NotifyConsole" />
                               <target class="NotifyFiles">
                               <file append="true"
                               URI="c:/tmp/output/results.log" />
                               </target>
                               </NotificationList>
                               <NotificationList type="err">
                               <target class="NotifyConsole" />
                               <target class="NotifyFiles">
                               <file append="false"
                               URI="c:/tmp/err/error.log" />
                               </target>
                               </NotificationList>
                               </property>
                               </action>
                               </actions>
                               </service>


                              • 12. Re: Transform and persist from queue to filesystem

                                For the time being I extended the NotifyFiles class and in my implementation added the following code:

                                 public void sendNotification (Message message) throws NotificationException {
                                 message.getBody().setContents(message.getBody().get().toString().getBytes());
                                 super.sendNotification(message);
                                 }
                                


                                Probably not the elegant solution...

                                • 13. Re: Transform and persist from queue to filesystem
                                  tfennelly

                                   

                                  "asachde" wrote:
                                  I did configure the output-location (to) to a service that was newly defined. I moved the filenotification to that new service. However when I do a dump of printfull the body still has the old message and the new message is in the defaultEntry.


                                  Not sure what you mean here. Can you clarify please? Specifically, what do you mean by "I did configure the output-location (to) to a service that was newly defined"? The "output-location" and "to" properties have nothing to do with each other. Perhaps I'm misunderstanding you.

                                  • 14. Re: Transform and persist from queue to filesystem

                                    Well I think I misunderstood the output-location to the to property. So how do you get the result of the transformation set back to the message body?

                                    1 2 Previous Next