12 Replies Latest reply on Mar 23, 2011 3:13 AM by beve

    Camel Component

    kcbabo

      Since we plan on adding a Camel component for the 0.1 release, I figured I would kick off a discussion thread on how that component should take shape.  We have had some hallway-type discussions about this in the past and Daniel has done some great prototyping work, so it's time to pull that together and git 'er done.

       

      There are two ways that Camel can be integrated in Switchyard:

       

      1) Use Camel endpoints as gateways within SwitchYard.  This would come in the form of a camel binding, where the Camel endpoint configuration appears in the binding declaration.

       

      <sca:service name="camelTest" promote="SimpleCamelService">

         <camel:binding.xyz>

                ...

         </camel:binding.xyz>

      </sca:service>

       

      In the above example, "xyz" represents a binding type and the configuration would be contained in a child element (or attributes) and defined by a schema.  Personally, I prefer this over just pointing to a Camel configuration as it tightens up the configuration in terms of validation and provides more flexibility from a tooling standpoint.

       

      We could always allow for a degenerate case of specifying an opaque <camel:binding> which simply pointed to a camel context with configuration in it or maybe a URI string with the config there.  Personally, I don't see this as an acceptable long term solution, but it might be something that we leave as an option for bindings where we don't have an explicit binding configuration defined.

       

      2) Use Camel routes to handle pipeline orchestration.  In this case, Camel is used in a similar way to the ESB 4.x action processing pipeline.

       

      <component name="SimpleCamelService">

         <service name="SimpleCamelService">

            <interface.???>

          </service>

          <reference name="SomeOtherService">

               <interface.???>

          </reference>

          <implementation.camel>   

             <route>

                <split>

                   <xpath>/status/person</xpath>

                   <to uri="switchyard:SomeOtherService"/>

                </split>

             </route>

          </implementation.camel>

      </component>

       

      Some notes on the above:

      • interface.??? needs to be replaced with the appropriate service interface type.  This could be Java, WSDL, or another basic type that we add which allows for a simple specification of the mep and expected message/fault types.
      • The route definition is embedded in the camel implementation config.  We may need more than route there, but for now I'm just focusing on the routing support.
      • We could move to a model where a Java DSL could be specified in the project and picked up by a configuration scanner.  This would be super awesome if it dovetailed with our CDI support.
      • Notice that we are routing to a SwitchYard service and not a concrete endpoint in the Camel route.  IMO, this is the best way to use Camel within SwitchYard because it provides a nice dependency view (note the <reference>) and makes it straightforward to externalize the binding configuration from the routing logic.

       

      I'll pause there and let others shoot holes in the above. :-)

        • 1. Re: Camel Component
          beve

          I like the sound of this! I have some comments below that are mainly me trying to verify that I have understood this correctly.

           

          For the first option, where we enable SwitchYard access to camel endpoints, do you mean something like this:

           

          <sca:service name="camelTest">
             <camel:binding.camel>
                 <beans>
                    <camelContext id="camelContext" xmlns="http://camel.apache.org/schema/spring">
                        <route>
                           <from uri="file://inputDir?fileName=input-message.txt"/>
                           <log message="Before orderProcessService:  ${body}"/>
                           <to uri="switchyard://orderProcessService"/>  
                        </route>
                    </camelContext>
                 </beans>
             </camel:binding.camel>
           </sca:service>
          
          

          In the above example we have a complete Spring DSL. Users that have existing Camel Spring DSL configurations can stick them into the binding.camel element.

           

          In the above example we are routing to a switchyard service but this could just as well have been a route that does not have any services implemented in SwitchYard. By using Camel to write the integration logic the decision to deploy to SwitchYard or a different runtime could be made later and avoids any vendor lock-in. I also think that this is a valid use-case when migrating from a different runtime and you might want to gather all your integrations into one runtime container to avoid maintenance overhead. 

           

          I also believe that Camel users use Camel's Java DSL at least as often as they use Camel's Spring XML DSL, which I think we should also cater for. Perhaps we could use Camel's 'packageScan' feature which will automatically discover and initialize routes in the specified packages. For example, the camelContext configuration would look something like this:

          <camelContext xmlns="http://camel.apache.org/schema/spring">
              <packageScan>
                  <package>org.switchyard.example</package>
                    <excludes>**.*Excluded*</excludes>
                    <includes>**.*</includes>
                  </packageScan>
          </camelContext>
          
          
          

          But this is probably just as bad as pointing to a Camel configuration file...Need to look into what options we have and try them out I think.

           

          <component name="SimpleCamelService">
             <service name="SimpleCamelService">
                <interface.???>
              </service>
              <reference name="SomeOtherService">
                   <interface.???>
              </reference>
              <implementation.camel>    
                 <route>
                    <split>
                       <xpath>/status/person</xpath>
                       <to uri="switchyard:SomeOtherService"/>
                    </split>
                 </route>
              </implementation.camel>
          </component>
          
          

          So, in this we don't have any 'from' element in the route definition since this Camel route is triggered by calling the SwitchYard service 'SimpleCamelService'.

           

          I'll try out these concepts and post back here for feedback (and probably more questions ).

           

          Thanks,

           

          /Daniel

          • 2. Re: Camel Component
            kcbabo

            Sorry for the late reply.  Spent all day yesterday working on another issue.  Comments below ....

            Daniel Bevenius wrote:

             

            For the first option, where we enable SwitchYard access to camel endpoints, do you mean something like this:

             

            <sca:service name="camelTest">
               <camel:binding.camel>
                   <beans>
                      <camelContext id="camelContext" xmlns="http://camel.apache.org/schema/spring">
                          <route>
                             <from uri="file://inputDir?fileName=input-message.txt"/>
                             <log message="Before orderProcessService:  ${body}"/>
                             <to uri="switchyard://orderProcessService"/> 
                          </route>
                      </camelContext>
                   </beans>
               </camel:binding.camel>
            </sca:service>



            In the above example we have a complete Spring DSL. Users that have existing Camel Spring DSL configurations can stick them into the binding.camel element.


             

            I don't really see a camel route as a service binding.  The route itself is an orchestration of services, much like BPEL, BPMN2, etc., although in the case of Camel is more of a "pipeline" orchestration.  A service binding should describe how a service is bound to a wire protocol.  The routing logic belongs in an implementation component (implementation.camel in my example).

             

            For the binding, I would expect something like this:

            <sca:service name="camelTest" promote="SimpleCamelService">
               <camel:binding.file>
                  <inputDir>/work</inputDir>
                  <fileName>input-message.txt</fileName>
               </camel:binding.file>
             </sca:service>
            

             

            This does require the user to pull apart the existing Camel context definition, but the mapping should be straightforward.

            I also believe that Camel users use Camel's Java DSL at least as often as they use Camel's Spring XML DSL, which I think we should also cater for. Perhaps we could use Camel's 'packageScan' feature which will automatically discover and initialize routes in the specified packages. For example, the camelContext configuration would look something like this:

             

            I was thinking about how this might integrate with our CDI support.  A CDI bean already provides the service contract information, so if we can find an elegant way of wiring the Java DSL into our CDI @Service beans, then that would kill two birds with one stone:

            1) Defines the contract for the service

            2) Makes the route discoverable

             

            So, in this we don't have any 'from' element in the route definition since this Camel route is triggered by calling the SwitchYard service 'SimpleCamelService'.

             

            Yep

            • 3. Re: Camel Component
              beve
              I don't really see a camel route as a service binding.  The route itself is an orchestration of services, much like BPEL, BPMN2, etc., although in the case of Camel is more of a "pipeline" orchestration.  A service binding should describe how a service is bound to a wire protocol.  The routing logic belongs in an implementation component (implementation.camel in my example).

              Ah right, that makes more sense. Thanks for clearing that up for me. I'll have another go at this.

               

              I was thinking about how this might integrate with our CDI support.  A CDI bean already provides the service contract information, so if we can find an elegant way of wiring the Java DSL into our CDI @Service beans, then that would kill two birds with one stone:

              1) Defines the contract for the service

              2) Makes the route discoverable


              Cool, I'll take a look and see what we can do.

               

              Thanks,

               

              /Daniel

              • 4. Re: Camel Component
                kcbabo

                Still waiting on that email list (beyond my control unfortunately), so thought I would update the thread with the JIRA for this work in 0.1.

                 

                https://issues.jboss.org/browse/SWITCHYARD-172

                • 5. Re: Camel Component
                  beve

                  I've been playing around with the camel.binding and have a question about how to determine the operation name of the service. What I mean by this is that if you take a look at the soap.binding you have something like this:

                   

                   

                  <composite xmlns="http://docs.oasis-open.org/ns/opencsa/sca/200912" name="m1app">
                          <sca:service xmlns:sca="http://docs.oasis-open.org/ns/opencsa/sca/200912" name="OrderService" promote="OrderService">
                              <soap:binding.soap xmlns:soap="urn:switchyard-component-soap:config:1.0">
                                  <soap:serverPort>18001</soap:serverPort>
                                  <soap:wsdl>wsdl/OrderService.wsdl</soap:wsdl>
                              </soap:binding.soap>
                          </sca:service>
                          <component name="InventoryService">
                              <service name="InventoryService">
                                  <interface.java xmlns="urn:switchyard-component-bean:config:1.0" interface="org.switchyard.quickstarts.m1app.InventoryService"/>
                              </service>
                              <implementation.bean xmlns="urn:switchyard-component-bean:config:1.0" class="org.switchyard.quickstarts.m1app.InventoryServiceBean"/>
                          </component>
                          <component name="OrderService">
                              <service name="OrderService">
                                  <interface.java xmlns="urn:switchyard-component-bean:config:1.0" interface="org.switchyard.quickstarts.m1app.OrderService"/>
                              </service>
                              <reference name="InventoryService">
                                  <interface.java xmlns="urn:switchyard-component-bean:config:1.0" interface="org.switchyard.quickstarts.m1app.InventoryService"/>
                              </reference>
                              <implementation.bean xmlns="urn:switchyard-component-bean:config:1.0" class="org.switchyard.quickstarts.m1app.OrderServiceBean"/>
                          </component>
                      </composite>
                  
                  

                  In the above case a web service will take the contents of the SOAPMessage and send it to the service named 'OrderService'. It determines what operation to invoke by inspecting the SOAPMessage and the WSDL, as far as I can tell.

                   

                  Now for the camel.binding there would in many cases be no way to determine the operation name on the service to invoke. As a suggestion, I've added a 'operationSelector' element to the camel.binding that looks like this:

                   

                  <sca:composite>
                      <sca:service name="SimpleCamelService">
                          <camel:binding.direct uri="direct://input">
                              <camel:operationSelector operationName="print"/>
                          </camel:binding.direct>
                      </sca:service>
                  
                      <sca:component name="ComponentName">
                          <bean:implementation.bean class="org.switchyard.component.camel.deploy.support.SimpleCamelServiceImpl"/>
                          <sca:service name="SimpleCamelService">
                              <sca:interface.java interface="org.switchyard.component.camel.deploy.support.SimpleCamelService"/>
                          </sca:service>
                      </sca:component>
                  
                  </sca:composite>
                  
                  

                  You can also see that I'm using the 'uri' attribute of the binding. This seemed like a nice option but we can still extend the binding.xxx to add whatever elements required.

                   

                  For this to work I also had to add the CamelActivator to the Deployment class. The activators are stored in a map and are keyed by the 'type' of the binding. For example, for 'binding.direct' the 'direct' part is the type. So this meant that I needed to add one entry for each type even though they all use the same CamelActivator. There might be a better way to do this so please let me know if there is. I was thinking that perhaps the namespace could be used instead of the type as the key in the map of activators, but I'm not sure about what implications this would have.

                   

                  If you care to take a closer look at this you can take a look the following repositories:

                  1. Switchyard Components

                  2. Switchyard Core

                   

                  Comments/thoughts/suggestions?

                   

                  Thanks

                   

                  /Daniel

                  • 6. Re: Camel Component
                    kcbabo

                    Just want to point out that I hate this forum software with the passion of 1000 burning suns.  I will now type out my response again after having my previous session wiped clean.

                    • 7. Re: Camel Component
                      kcbabo

                      Yep, operationSelector is the way to go.  I'm fine with the static definition you have in your example.  It's likely that we will want more dynamic selectors which determine the operation based on the input message, but that can come after we get this first bit down.

                       

                      I'm not a big fan of the URI-based configuration for Camel bindings.  Validation is snarky at best and it's tough to provide a meaningful presentation in tooling.  That said, I realized that it's unlikely that we will have a typed configuration model for all Camel components, so we need some type of generic configuration mechanism for the types we don't define directly.  Perhaps we can follow this approach:

                       

                      1) Add a generic camel:binding config element which has a uri attribute.  The URI will define the type of binding via the scheme and the configuration.  This can be used for any Camel binding where we don't have a typed configuration.

                       

                      2) Add specific config elements for Camel bindings as we go, e.g. camel:binding.ftp, camel:binding.file, etc.  The configuration will be backed by an XSD and a SwitchYard config model.

                       

                      Does that sound reasonable?  I can help with #2 so that you don't have to do everything. ;-)

                       

                      Re: the change to Deployer to add the Camel activator, that's fine for now.  We really need to externalize that in configuration some time in 0.1 (https://issues.jboss.org/browse/SWITCHYARD-189).

                      • 8. Re: Camel Component
                        beve

                        1) Add a generic camel:binding config element which has a uri attribute.  The URI will define the type of binding via the scheme and the configuration.  This can be used for any Camel binding where we don't have a typed configuration.

                        Sounds good, so I've added a 'binding.camel' which is generic and is defined by CamelBindingModel and a concrete implementation in V1CamelBindingModel .

                        The other binding types, for example the 'binding.file' type, would extend V1CamelBindingModel and would also be possible to revert to just using a uri attribute. I think this makes sense as the sca:bindings element has an 'uri' attribute and this could always valid to specify here. What do you think about this?

                         

                         

                        2) Add specific config elements for Camel bindings as we go, e.g. camel:binding.ftp, camel:binding.file, etc.  The

                        configuration will be backed by an XSD and a SwitchYard config model.

                        We have a schema in place but the current bindings only extend the generic binding so we can add types that extend the generic one and add elements and attributes.

                         

                         

                        Does that sound reasonable?  I can help with #2 so that you don't have to do everything. ;-)

                         

                        Yep, this should great.

                        Re: the change to Deployer to add the Camel activator, that's fine for now.  We really need to externalize that in configuration some time in 0.1 (https://issues.jboss.org/browse/SWITCHYARD-189).

                         

                        Ah perfect, then I'll leave this as is for the time being!

                         

                         

                        Thanks

                        • 9. Camel Component
                          kcbabo
                          Sounds good, so I've added a 'binding.camel' which is generic and is defined by CamelBindingModel and a concrete implementation in V1CamelBindingModel .

                          The other binding types, for example the 'binding.file' type, would extend V1CamelBindingModel and would also be possible to revert to just using a uri attribute. I think this makes sense as the sca:bindings element has an 'uri' attribute and this could always valid to specify here. What do you think about this?

                           

                          I don't think we should permit the uri attribute in specific binding types like binding.file.  The generic camel:binding does not have a configuration schema, so the user's only option is to use the URI format.  For a specific binding, there is a configuration schema defined, so the user can (and should) use that instead of the URI.  Allowing both options in the specific binding case could create some confusing scenarios, e.g. what if both are specified, which values do we choose? 

                           

                          Good catch on the sca:binding uri attribute.  The SCA spec disallows use of the binding uri on service definitions, so we may want to change the attribute name to avoid confusion.  Perhaps we should call it "configURI"  or just "config" and the type could be xs:anyURI ?

                           

                          We have a schema in place but the current bindings only extend the generic binding so we can add types that extend the generic one and add elements and attributes.

                           

                          I view the generic camel binding as a distinct binding type and not a parent binding to other camel binding types.  If we need a base camel binding type, I think we should create a base type and have generic, file, ftp, etc. inherit from that.

                           

                          Ah perfect, then I'll leave this as is for the time being!

                           

                          Fantastic.  Thanks very much for moving this forward.  It's looking great.  I will take a spin through the code today and provide further comments.

                          • 10. Re: Camel Component
                            beve

                            Perhaps we should call it "configURI"  or just "config" and the type could be xs:anyURI ?

                            Yeah, that is a better idea. Let me rework this and also unwind the inheritence that I have at the moment for these binding models.

                            • 11. Re: Camel Component
                              kcbabo

                              Went through the Camel component in your "camel" branch and it looks great.  Looks like the current implementation supports service bindings using Camel endpoints, which provides a nice library of gateways to choose from. :-)

                               

                              In terms of next steps, we need to add:

                              o Support for reference bindings, so that we can use Camel endpoints to invoke external services.

                              o Support for an implementation.camel so you can use Camel DSL for routing between SwitchYard services.

                               

                              Would you like to commit what you have and then we can work on the above together?

                               

                              Thanks very much, btw.  This is an awesome contribution.

                              • 12. Re: Camel Component
                                beve

                                Would you like to commit what you have and then we can work on the above together?

                                 

                                Sounds great! I've created pull requests and commented on the jira:

                                https://issues.jboss.org/browse/SWITCHYARD-172

                                 

                                Thanks,

                                 

                                /Dan