1 2 3 4 Previous Next 56 Replies Latest reply on May 9, 2006 6:08 AM by marklittle

    Transformation service interface

    marklittle

      I'm really interested in leveraging Smooks as one of the out-of-the-box transformation engine implementations. As the architecture defines, this will be a service (as with all services, it may be seen as a logical single entity but in reality may be backended my multiple services). As with the registry, I don't have any specific interface in mind, but will post some suggestions in the next day or so if no one else does.

      Mark.

        • 1. Re: Transformation service interface
          marklittle

          To kick start this discussion (I'm hoping any resident transformation engine gurus will join in), from an architectural perspective it seems to me that we need two levels of API.

          The first is the service interface that users see: as with all services within JBossESB, it may appear to a user as a single instance, but may be implemented by distributed (federated) instances. So logically this interface needs to allow the user to specify the data to be transformed and the type of transformation that is required. I suppose it may be possible to determine from the data what the transformation is (e.g., some attribute within the data schema), but that's an implementation detail at this stage. Essentially this client-service interface takes the same information: data to be transformed, and the type of transformation required; ultimately the result is a transformed data stream (or some error indication).

          The implementation of this client-service transformation interface then needs to determine where appropriate transformation engines (service instances) reside. In which case, we're back into the usual service-lookup situation and we'd look within the registry. That may mean we want a more powerful (QoS enabled) client-service transformation API with more than just the ability to specify the type of transformation that is needed.

          Once we get to the actual transformation service invocation, the interface to the transformation service will presumably be just doTransformation (or somesuch): input the data to be transformed and get the resultant transformation as the output.

          Now of course any given engine may provide support for multiple types of transformation, but rather than clutter the service interface, I think it makes sense to simply say that such engines appear as physically distinct service instances (of course they don't have to be implemented as such, but as far as clients are concerned, they are different services).

          Any comments?

          • 2. Re: Transformation service interface
            tfennelly

            Are we trying to standardise on an interface for all Service implementations, irrespective of the role the Service is performing? At present, I can't see why a transformation service needs a *special* interface!

            What about JSR208 - NormalizedMessage and MessageExchange (http://java.sun.com/integration/1.0/docs/sdk/api/index.html)?

            • 3. Re: Transformation service interface
              marklittle

              So firstly we're talking about the ESB core, which is not JBI compliant: we'll layer JBI on top of that. Doesn't mean we can't have a service interface that is similar to JBI, just that we don't want to have to bring in JBI at this level. Furthermore, transformation is just another service in the grand scheme of things (at least within JBossESB): it doesn't have some priority behaviour/status over other services and isn't mandated for all message interactions.

              Secondly, we're looking to provide a standard interface for transformation services - not for all services irrespective of their role. (At the lowest level, the interface for all services is the same: doWork). The reason we need an interface is that it's a service. If you check out the architecture document, you'll see that everything is a service and communicated with via messages. The interface is the contract between the client and the service: for a start it defines the message exchanges that can legally be performed.

              Now I assumed that a single interface for all transformation services would be possible. What I didn't want (unless it's really necessary) was to have a set of interfaces for each type of transformation.

              • 4. Re: Transformation service interface
                tfennelly

                Sure, I need to have a closer read of the architecture doc.

                To be honest, I think that until a few transformation service have been implemented, it's going to be difficult to generalise a "Transformation" specific interface. That's why I was sugesting something more general along the lines of (but perhaps not tied directly to) the JBI interfaces.

                I also don't see the need to define an interface specificly for transformation services. Perhaps I will after having a proper read of the arch docs.

                • 5. Re: Transformation service interface
                  marklittle

                  As I said, ultimately all services have the same interface (at least conceptually), which is doWork on the received message. If you check the architecture doc (and the code in svn), you'll see that this is reflected in the APIs too, with clients essentially using send and services using receive. However, the actual message exchange pattern dictates the contract definition between clients and services and this is specific to the type of service being used. Whether we define this in WSDL or something else (something else is my preference) is immaterial: it needs to be defined. Ultimately this definition may appear as some concrete represetation within a specific implementation language, to help abstract/isolate the client from the lower level details of the communication mechanism, i.e., an interface (or a client/service stubs in reality). This will be specific to the service.

                  • 6. Re: Transformation service interface
                    timfox

                    Does the transformation API support doing work on a stream rather than a whole message?

                    I'm really thinking about large messages e.g 10s or 100s of MB where you'd want to avoid loading the entire message into memory just to (say) change a small part, which could give some serious performance issues.

                    Many transformations can be applied to a stream and don't require the entire message, e.g. for an xml message you could stream the message through until you see the element you want to transform then just transform that part.

                    (Many messaging systems provide messages with stream bodies so they can handle large messages performantly which I guess is something that could be tapped into by the ESB)

                    Apologies if this is already covered :)


                    • 7. Re: Transformation service interface
                      marklittle

                      There isn't an API yet: that's the reason for this topic ;-) However, you're right in that we need to support streaming.

                      • 8. Re: Transformation service interface
                        tfennelly

                        So how about the following:

                        interface TransformationService {
                        public transform(Message message, Properties transformationParams) throws TransformationException;
                        }

                        Message message: Could be a stream, DOM or whatever.
                        Properties transformationParams: Perhaps this is too general i.e. we need to add more strongly typed method params.

                        Defining the interface in Java, but obviously it may need to be defined some other way.

                        Where do I look to see the existing ESB codebase?

                        • 9. Re: Transformation service interface
                          tfennelly

                          Forgot to add thoughts on what the actual transformationParams may be:

                          1. messageType
                          2. messageSource
                          3. messageDestination

                          These should be all the transformation service impl needs to make its decision as to which transformation to perform. Note I'm not suggesting telling the transformation Service "perform transformation X". This seems better to me! Thoughs?

                          4+. What about other misc parameters that can be fed directly into the transformation process executed within the service?

                          • 10. Re: Transformation service interface
                            marklittle

                            Tom, I'll send you something about the code location soon - am away from my computer at the moment.

                            Mark.

                            • 11. Re: Transformation service interface
                              marklittle

                              The idea is that there'll potentially be multiple transformation services out there. These may perform different types of transformation, or may have different qualities of service (QoS). So that's why in my original text I suggested the need for a first-level interface: essentially locating and binding to the right service. The TransformationService interface seems to be more appropriate for the second-level interface - the actual service interface. True?

                              • 12. Re: Transformation service interface
                                marklittle

                                Would we necessarily define things like messageType and messageSource as parameters? Wouldn't they be within the actual message itself? Maybe it's a different meaning of Message, but here's where I'm coming from:

                                a message consists of:







                                Within the header element, there'll be some addressing information (which includes stuff like wsa:From, wsa:ReplyTo, wsa:From etc.) and can be used for routing. So, for instance, what's the difference between messageSource and wsa:From, or messageDestination and wsa:ReplyTo?

                                Mark.

                                • 13. Re: Transformation service interface
                                  arvinder

                                  Assuming the message has already been parsed and validated
                                  you could do either
                                  - strongly type it or
                                  - pass a generic set of parameters


                                  Would we necessarily define things like messageType and messageSource as parameters? Wouldn't they be within the actual message itself?


                                  Then there would be no need as they are already present inside the
                                  Message. But that implies tieing the Message type to the Transformation
                                  service.

                                  Either way, for transformation, you need at the very least

                                  1. messageType
                                  2. messageSource

                                  But the service would additionally need to which direction to perform
                                  the transformation. This could be part of the Endpoint/Dispatcher
                                  configuration so you end up having something like

                                  <dispatcher>
                                  <services>
                                  ...
                                  <transformation-service>
                                  <direction>entry</direction>
                                  <messageType>CUSTOMER_UPDATE</messageType>
                                  <messageSource>SAP</messageSource>
                                  ..
                                  <xslt-sheet>sap2siebel.xslt</xslt-sheet>
                                  OR <processor>org.jboss.SAP2SiebelMessageProcessor</processor>
                                  ..
                                  </transformation-service>
                                  ...
                                  </dispatcher>


                                  The above is just an example, what i am trying to convey is that there
                                  needs to be a configuration paramter somewhere which tells the transformation service which xslt/sheet or processor (as in smooks impl) to use depending on the the type of message, where it came form and which direction it is going.


                                  So, for instance, what's the difference between messageSource and wsa:From, or messageDestination and wsa:ReplyTo?
                                  [/quot]

                                  I think messageSource == wsa:From, but I don't think the message necessarily needs to know where it is going, the esb routes it based on the header informstion and the esb configuration.

                                  comments?


                                  • 14. Re: Transformation service interface
                                    arvinder

                                    Ack! Bad formatting, here we go again

                                    Assuming the message has already been parsed and validated
                                    you could do either
                                    - strongly type it or
                                    - pass a generic set of parameters


                                    Would we necessarily define things like messageType and messageSource as
                                    parameters? Wouldn't they be within the actual message itself?


                                    Then there would be no need as they are already present inside the
                                    Message. But that implies tieing the Message type to the Transformation
                                    service.

                                    Either way, for transformation, you need at the very least

                                    1. messageType
                                    2. messageSource

                                    But the service would additionally need to which direction to perform
                                    the transformation. This could be part of the Endpoint/Dispatcher
                                    configuration so you end up having something like

                                    <dispatcher>
                                    <services>
                                    ...
                                    <transformation-service>
                                    <direction>entry</direction>
                                    <messageType>CUSTOMER_UPDATE</messageType>
                                    <messageSource>SAP</messageSource>
                                    ..
                                    <xslt-sheet>sap2siebel.xslt</xslt-sheet>
                                    OR
                                    <processor>org.jboss.SAP2SiebelMessageProcessor</processor>
                                    ..
                                    </transformation-service>
                                    ...
                                    </dispatcher>


                                    The above is just an example, what i am trying to convey is that there
                                    needs to be a configuration paramter somewhere which tells the transformation
                                    service which xslt/sheet or processor (as in smooks impl) to use depending
                                    on the the type of message, where it came form and which direction it is going.


                                    So, for instance, what's the difference between messageSource and wsa:From, or
                                    messageDestination and wsa:ReplyTo?


                                    I think messageSource == wsa:From, but I don't think the message necessarily
                                    needs to know where it is going, the esb routes it based on the header informstion
                                    and the esb configuration.

                                    comments?

                                    1 2 3 4 Previous Next