1 2 3 4 Previous Next 56 Replies Latest reply on May 9, 2006 6:08 AM by marklittle Go to original post
      • 45. Re: Transformation service interface
        tfennelly

        As a matter of interest... theoretically, the Transformation Dispatcher may need to break up the Message[] and dispatch to more than one transformation service i.e. break the incoming Message[] into multiple arrays depending on the transformation service to be used on each message.

        • 46. Re: Transformation service interface
          marklittle

           

          "tfennelly" wrote:
          Lovely Mark, that all makes sense to me and was pretty much what my understanding was.

          So, the little green boxes for the interface to the transformation service. To go back to your earlier post:
          There's a need because every service has to have an interface of one form or another. That interface may be extremely simple, and that's fine. But it has to exist.


          Questions:
          1. Are you saying there needs to be a "unique" interface for each service (or service family/type e.g. Transformation, Registry etc)?


          Each type of service. In the pretty much the same way as, say, WSDL or CORBA IDL. Each type of service will define its own message exchange pattern which will influence the way in which users interact with it. Backend implementation differences should be dealt with at the message level (or we lose SOA).


          2. Could 2 or more types of service not share a common interface signature?


          They could, but as things grow and extend, it then becomes more and more difficult to ensure that one interface matches both sets of requirements.


          3. Do you expect Client applications external to the ESB to be calling Transformation Service impls?


          Not sure at the moment, but since the services are published in a registry, there's nothing to prevent it (apart from security, for instance). As long as a client knows what to do with the information received from the registry (and it has the right credentials to do so) it should be able to interact.

          Did you have some concerns? If so, maybe we should discuss in a separate thread.


          So, where are am I coming from here? Well, I'm not sure how to confidently dissect a "service(Message[] msgs)" type signature and specialise it for a transformation service. I don't think one can rule out being able to dream up a usecase that might require access to any of the composite parts of the Message type as defined in the requirements doc.


          Let's return to the interface you proposed earlier. Furthermore, for the sake of this discussion, let's assume that is the service API, i.e., the interface that the transformation service implementer made sure the implementation conformed to and that the client uses. So at this point we're at the top of the client picture (before any dispatchers have fired) and at the top of the service picture (before any messages have been dispatched up from the network).

          Can you take me through the problem as you see it at this point?


          Also, because I was considered a Transformation Service as being internal to the ESB, I was think the service interface could be something like "service(Message[] msgs)" i.e. a generic service interface. We can specialise later, once we've built some of this stuff and have a better feel for what a more specialised service interface might look like - no external dependents.


          At the very highest level, clients won't use service(Message[] msgs) to talk to all of their services. They could, but that would be the way in which distributed systems were used prior to stub technologies. We're looking at the very highest point in the stack, where a transformation service looks different to a transaction service looks different to a security service, as far as the client is concerned. The client (and server) stubs. Within these stubs, everything does look like send(Message[] msgs) to the client, but the stubs (and the dispatchers) do a lot of hard work: creating the message for a start. This is work we don't want to force the client to do.

          Does that make sense?


          Of course, you can turn that last argument on its head and say: lets start with what we know we want and start adding to the interface. I'd ordinarily go with that approach only for the fact that I think there are so many transformation usecases out there that you can never rule out requiring access to all parts of the Message.

          Am I making sense at all, or am I still missing the point completely??


          OK, so if you are talking specifically about transformation services, then I understand and I'm happy to try that route. Then my only question is: how is service(Message[] msgs) different from the interface you originally proposed?

          BTW: good discussion :-)

          • 47. Re: Transformation service interface
            marklittle

             

            "tfennelly" wrote:
            sorry about the grammatical booboos


            wich gramitical boobooks?

            • 48. Re: Transformation service interface
              marklittle

               

              "tfennelly" wrote:
              As a matter of interest... theoretically, the Transformation Dispatcher may need to break up the Message[] and dispatch to more than one transformation service i.e. break the incoming Message[] into multiple arrays depending on the transformation service to be used on each message.


              OK, so we definitely should support that. I'd only considered the pipelining approach so far (where the message is transformed by calling multiple services in sequence before calling the actual service, or it transformed enroute by passing through multiple transformation services).

              • 49. Re: Transformation service interface
                tfennelly

                 

                3. Do you expect Client applications external to the ESB to be calling Transformation Service impls?


                Not sure at the moment, but since the services are published in a registry, there's nothing to prevent it (apart from security, for instance). As long as a client knows what to do with the information received from the registry (and it has the right credentials to do so) it should be able to interact.

                Did you have some concerns? If so, maybe we should discuss in a separate thread.


                Not sure yet. I guess, considering that we (well, I perhaps) can't see a transformation service with an interface more specialised than "service(Message[] msgs)", I think it's probably best to keep the transformation services internal i.e. not visible to external Client apps. Not sure!!

                At the very highest level, clients won't use service(Message[] msgs) to talk to all of their services. They could, but that would be the way in which distributed systems were used prior to stub technologies. We're looking at the very highest point in the stack, where a transformation service looks different to a transaction service looks different to a security service, as far as the client is concerned. The client (and server) stubs. Within these stubs, everything does look like send(Message[] msgs) to the client, but the stubs (and the dispatchers) do a lot of hard work: creating the message for a start. This is work we don't want to force the client to do.

                Does that make sense?


                Absolutely! This is why I think I'd keep Transformation Services internal until such time as we can do this meaningfully i.e. until we can create a transformation service interface that works Vs infuriates. So, for not I'm saying lets create a model where we don't have to worry about the Client interface until we have better info/understanding. I hope this isn't dangerous - I can't see how it would be. I think you've bought in on this too :-)

                Then my only question is: how is service(Message[] msgs) different from the interface you originally proposed?

                I don't think what I'm proposing now ("service(Message[] msgs)") is all that different from what I was proposing originally, which was based on the JBI message interfaces. Since then, I've read the architecture docs and some of the code and tried to match it all back. I think my reasons/emphasis may have changed a bit as my understanding of what we're trying to do (and how) has improved. I must find time to read the relevant WS specs!!!

                Do you think I'm being inconsistent?

                • 50. Re: Transformation service interface
                  marklittle

                   

                  "tfennelly" wrote:
                  3. Do you expect Client applications external to the ESB to be calling Transformation Service impls?


                  Not sure at the moment, but since the services are published in a registry, there's nothing to prevent it (apart from security, for instance). As long as a client knows what to do with the information received from the registry (and it has the right credentials to do so) it should be able to interact.

                  Did you have some concerns? If so, maybe we should discuss in a separate thread.


                  Not sure yet. I guess, considering that we (well, I perhaps) can't see a transformation service with an interface more specialised than "service(Message[] msgs)", I think it's probably best to keep the transformation services internal i.e. not visible to external Client apps. Not sure!!


                  By external you mean not ESB clients? Because obviously clients in the ESB on different nodes will be invoking transformation services throughout the ESB deployment.


                  At the very highest level, clients won't use service(Message[] msgs) to talk to all of their services. They could, but that would be the way in which distributed systems were used prior to stub technologies. We're looking at the very highest point in the stack, where a transformation service looks different to a transaction service looks different to a security service, as far as the client is concerned. The client (and server) stubs. Within these stubs, everything does look like send(Message[] msgs) to the client, but the stubs (and the dispatchers) do a lot of hard work: creating the message for a start. This is work we don't want to force the client to do.

                  Does that make sense?


                  Absolutely! This is why I think I'd keep Transformation Services internal until such time as we can do this meaningfully i.e. until we can create a transformation service interface that works Vs infuriates.


                  I don't have a problem with a minimal interface. I didn't come to this discussion with any interface in mind, except that we'd need one and it needs to sit within the SOA-everywhere paradigm for JBossESB. You're the transformation "guru" :-)


                  So, for not I'm saying lets create a model where we don't have to worry about the Client interface until we have better info/understanding. I hope this isn't dangerous - I can't see how it would be. I think you've bought in on this too :-)


                  Yes and no ;-) We need some interface for ourselves (don't want developers having to hand-craft messages that often - it leads to errors). That interface can be just as simple as you mention, though.



                  Then my only question is: how is service(Message[] msgs) different from the interface you originally proposed?

                  I don't think what I'm proposing now ("service(Message[] msgs)") is all that different from what I was proposing originally, which was based on the JBI message interfaces. Since then, I've read the architecture docs and some of the code and tried to match it all back. I think my reasons/emphasis may have changed a bit as my understanding of what we're trying to do (and how) has improved. I must find time to read the relevant WS specs!!!

                  Do you think I'm being inconsistent?


                  Not at all. I was just wondering if there were any significant reasons behind the change.

                  • 51. Re: Transformation service interface
                    tfennelly

                     

                    By external you mean not ESB clients? Because obviously clients in the ESB on different nodes will be invoking transformation services throughout the ESB deployment.


                    Sure, I meant non-ESB clients. In fact, I?m trying to figure out why any client, internal or external to the ESB, would need to invoke transformation services directly. I?d have thought that invoking a transformation service was an operation that the ESB would perform on behalf of the client during the process of routing the client message to the target service.

                    Can you elaborate on the usecase where a client (internal or external) needs to invoke transformation services directly i.e. the Transformation Service is the ?primary target? (if I can use that term)? I?m not talking about something like a Transformation Dispatcher here, because to me that?s really part of the core framework of the ESB, yeah? I mean ?business? type client apps. Sorry if this is a stupid question :-(

                    You're the transformation "guru" :-)

                    I am? My ass!! :-)

                    We need some interface for ourselves (don't want developers having to hand-craft messages that often - it leads to errors). That interface can be just as simple as you mention, though.


                    Do you mean handcrafting the messages in raw XML? When I refer to a ?Message? in terms of a TransformationService service method signature, I?m talking about the Message type as described in the architecture docs i.e. containing the Header, Context etc.

                    I suppose this all depends on the above i.e. whether or not clients really do need to be able to invoke transformation services directly, or if the ESB infrastructure does it on its behalf.

                    So, my understanding is as follows ? sorry for rehashing this again :-). Client apps invoke services via a contract specific stub (as you described earlier). The stub magically builds up a normalised ?Message? (containing a Header, Context etc) that it then pushes down the dispatcher chain (via a ClientPlugin??). Along the way, the message reaches a Transformation Dispatcher, which routes the ?Message? to a TransformationService implementation. The transformed Message comes back, and the Transformation Dispatcher send it on to the next Dispatcher in the chain and on towards the Service etc. So, the client developers code to a service contract (stub) in the way you?d want them to ? the normalised Message is internal to the ESB and the Transformation Service is not invoked directly.

                    I?m sure there are usecases other than that described above, but I?d have thought this would be the most common by some distance, no???

                    I wonder are there really 2 types of transformation service here, and they need to be considered separately???
                    1. The type of transformation service described above ? ESB Message translation in support of integration. This type of transformation is managed completely by the ESB and is invisible to the client or service. In this case, the transformation service is never the primary target.
                    2. Transformation services that are invoked directly by client apps. Still dunno about this!! Why would one create or use such a service?? Anyway, in this case the transformation service is always the primary target.

                    Well??

                    I must try sketch out something re what might be involved in doing the above with Smooks i.e. what might be involved from a developer?s perspective etc? What had other people got in mind re how and when a transformation service would be built? I suppose what I mean is, do we want to build a SmooksTransformationService that gets configured with plugins of some sort, or do we want to build a SAPToSiebelTransformationService?


                    • 52. Re: Transformation service interface
                      marklittle

                       

                      "tfennelly" wrote:
                      By external you mean not ESB clients? Because obviously clients in the ESB on different nodes will be invoking transformation services throughout the ESB deployment.


                      Sure, I meant non-ESB clients. In fact, I?m trying to figure out why any client, internal or external to the ESB, would need to invoke transformation services directly. I?d have thought that invoking a transformation service was an operation that the ESB would perform on behalf of the client during the process of routing the client message to the target service.


                      Probably, but because it's a service and available via the registry, anyone can use it assuming they have the right credentials/authority. I can't see any good reason to restrict this any further: just because we can't think of a use for direct interactions, doesn't mean they don't exist. Plus, since the ESB will be doing most of the transformation interactions, it's acting as a client anyway, so we know the scenario should work.


                      Can you elaborate on the usecase where a client (internal or external) needs to invoke transformation services directly i.e. the Transformation Service is the ?primary target? (if I can use that term)? I?m not talking about something like a Transformation Dispatcher here, because to me that?s really part of the core framework of the ESB, yeah? I mean ?business? type client apps. Sorry if this is a stupid question :-(


                      So one of the things you may want to do within a transformation service is utilise XPath expressions. The result of such an evaluation (which can be handled entirely by the transformation service) may lead you (as a client) to perform some application specific work (requiring semantic information that it not available at the transformation service) before you ultimately do the invocation on some other service.



                      You're the transformation "guru" :-)

                      I am? My ass!! :-)

                      We need some interface for ourselves (don't want developers having to hand-craft messages that often - it leads to errors). That interface can be just as simple as you mention, though.


                      Do you mean handcrafting the messages in raw XML? When I refer to a ?Message? in terms of a TransformationService service method signature, I?m talking about the Message type as described in the architecture docs i.e. containing the Header, Context etc.


                      Yes and that's all XML :-)


                      I suppose this all depends on the above i.e. whether or not clients really do need to be able to invoke transformation services directly, or if the ESB infrastructure does it on its behalf.


                      Whether or not clients invoke the service directly doesn't help us much at the moment. We (developers) are the client, even at the dispatcher level because we don't have any nice tools to auto-generate these XML messages just yet.


                      So, my understanding is as follows ? sorry for rehashing this again :-). Client apps invoke services via a contract specific stub (as you described earlier). The stub magically builds up a normalised ?Message? (containing a Header, Context etc) that it then pushes down the dispatcher chain (via a ClientPlugin??). Along the way, the message reaches a Transformation Dispatcher, which routes the ?Message? to a TransformationService implementation. The transformed Message comes back, and the Transformation Dispatcher send it on to the next Dispatcher in the chain and on towards the Service etc. So, the client developers code to a service contract (stub) in the way you?d want them to ? the normalised Message is internal to the ESB and the Transformation Service is not invoked directly.


                      Yes, that sounds right. But because we are working with a recursive architecture, the invocation on the transformation service looks the same as the client invocation on the "real" service: interface, dispatchers etc.


                      I?m sure there are usecases other than that described above, but I?d have thought this would be the most common by some distance, no???


                      I think so too, but I don't want us to encode any artificial restrictions at this stage (same argument as for your proposed transformation service interface).


                      I wonder are there really 2 types of transformation service here, and they need to be considered separately???
                      1. The type of transformation service described above ? ESB Message translation in support of integration. This type of transformation is managed completely by the ESB and is invisible to the client or service. In this case, the transformation service is never the primary target.
                      2. Transformation services that are invoked directly by client apps. Still dunno about this!! Why would one create or use such a service?? Anyway, in this case the transformation service is always the primary target.

                      Well??


                      I don't think we need two. One should be sufficient.

                      I must try sketch out something re what might be involved in doing the above with Smooks i.e. what might be involved from a developer?s perspective etc? What had other people got in mind re how and when a transformation service would be built? I suppose what I mean is, do we want to build a SmooksTransformationService that gets configured with plugins of some sort, or do we want to build a SAPToSiebelTransformationService?


                      • 53. Re: Transformation service interface
                        tfennelly

                        OK :-)

                        So, where should we go from here? Are we saying we have something to work with at this stage?

                        • 54. Re: Transformation service interface
                          marklittle

                          I think we are. Now we just need to create this basic interface and tie in Smooks as a service (I'm in favour of using Smooks). At the moment we have a basic tcp invocation test (plugin and dispatcher). It should be possible to leverage that until we get something else.

                          • 55. Re: Transformation service interface
                            tfennelly

                            Perhaps we should just close out this thread by stating the proposed TransformationService interface.

                            Pick one of the follow (defined as a Java interface):

                            interface TransformationService {
                            public Message[] transform(Message[] msgs) throws TransformationException;
                            }

                            OR

                            interface TransformationService {
                            public Message transform(Message msg) throws TransformationException;
                            }

                            Do we require a specialised exception for this service? This raises another question for me - which type of exception "philosophy" will JBossESB be following i.e. checked Vs unchecked exceptions (ala Spring, Hibernate etc)?

                            • 56. Re: Transformation service interface
                              marklittle

                              I'd got for the former. It gives us more scope for flexibility.

                              I think we should have specific exceptions to indicate as much about failures as possible. Whether that means one exception per failure type, or a failure code within a smaller set of exceptions, can be discussed (my preference is one per failure).

                              1 2 3 4 Previous Next