Camel Transformer

[beve]

Hi,

I was thinking about transformer and if we could somehow delegate to Camel's Type Converters.

As an example we might create a SwitchYard Transformer that looks somethings like this:

public class CamelTransformer extends BaseTransformer
{
    private CamelContext _camelContext;

    ...

    @Override
    public Object transform(final Object from)
    {
        if (from == null)
            return null;

        TypeConverter typeConverter = getCamelContext().getTypeConverter();
        return typeConverter.convertTo(JavaService.toJavaMessageType(getTo()), from);
    }
}

Now this transformer could be specified as a JavaTransformer in a switchyard.xml file. For example:

  <transforms>
        <transform.java xmlns="urn:switchyard-config:transform:1.0" from="java:java.lang.String" to="java:java.lang.Integer" class="org.switchyard.component.camel.transform.CamelTransformer"/>
  </transforms>     

Camel ships with a number of type converters and users may have their own customer type converter too.

Using a CamelTransformer like the one above would give our users access to the type converters in Camel but they will have to register every 'to' and 'from' type like the above example does or we would have to register the ones that are known.

TomF and I discussed this and Tom suggested that we query Camel for all the type that Camel has registered then we register those types in SwitchYard and use the CamelTransformer. But as far as I can tell we can't query Camel for this. What we can do it ask Camel's TypeConverterRegistry if it has a TypeConverter for a 'to' and 'from' type combination:

    TypeConverter lookup(Class<?> toType, Class<?> fromType);

Hmm, what about creating a CamelTransformerRegistry that would first delegate to Camel's TypeConverters and fallback to SwitchYards default, so it would be a wrapper/adapter. This would only be used when the Camel component is in use so it has to be registered. Currently the TransformerRegistry to use is created in AbstractDeployment in the createDomain method. If we make this configurable someway this might be an option.

Any thoughts on this?

/Daniel

[moofish32]

So would this set the precedence then for each Camel 'like' component that SwitchYard adds, the implementor would also need to create a Transformer 'sub' Registry and add it to the main SY one? Then the component would always give preference to its native registry first and the SY one second.  I would also have to assume that if a registry was not added that it would automatically use the default one. 

If that is where this is going I actually like it.  Provides great configurability to easily override the default transformers.  I think a similar path is approachable with something like protocol buffers using the FieldDescriptors for the from to classes.

-Mike

[kcbabo]

Sorry for not replying earlier.  Forum notification fail.

Can we define a META-INF/transforms.xml that captures all of the converters available in camel and then include that in the Camel component archive?  When creating a domain, we already search for classes which implement our Activator contract.  The existing search in TransformerRegistryLoader should return all instances of META-INF/switchyard/transforms.xml on the classpath (which would include the transforms.xml supplied with any components).

[kcbabo]

So would this set the precedence then for each Camel 'like' component that SwitchYard adds, the implementor would also need to create a Transformer 'sub' Registry and add it to the main SY one? Then the component would always give preference to its native registry first and the SY one second.  I would also have to assume that if a registry was not added that it would automatically use the default one. 

If that is where this is going I actually like it.  Provides great configurability to easily override the default transformers.  I think a similar path is approachable with something like protocol buffers using the FieldDescriptors for the from to classes.

At present, transformers are scoped to a specific domain.  Each domain provides an API hook to the TransformRegistry, so any client of that API can programmatically register transformers at runtime.  Tom and I were punting around the idea of a hierarchical domain model, which in theory would allow you to register a transformer at a parent/primordial level which could be inherited by children.  There are probably a fair amount of details to work out with that approach, however.

However we solve this problem, we need to make sure that (a) there's a public API contract available for it, and (b) any configuration-based approaches beyond an API should still use (a).  I think my suggestion above satisfies this requirement, but I could be missing something.

[beve]

Can we define a META-INF/transforms.xml that captures all of the converters available in camel and then include that in the Camel component archive?

I've played around with this and I think this would work. What I've got at the moment is a transforms.xml that looks like this:

<transforms xmlns="urn:switchyard-config:switchyard:1.0"
            xmlns:trfm="urn:switchyard-config:transform:1.0">

    <!-- GenericFile -->
    <trfm:transform.java 
        from="java:org.apache.camel.component.file.GenericFile" 
        to="java:java.lang.String" 
        class="org.switchyard.component.camel.transformer.CamelTransformer"/>
    <trfm:transform.java 
        from="java:org.apache.camel.component.file.GenericFile" 
        to="java:java.io.InputStream" 
        class="org.switchyard.component.camel.transformer.CamelTransformer"/>
    <trfm:transform.java 
        from="java:org.apache.camel.component.file.GenericFile" 
        to="java:java.io.Serializable" 
        class="org.switchyard.component.camel.transformer.CamelTransformer"/>
    <!-- TODO: Add all known converters and their types as has been done for GenericFile above -->


</transforms>

The CamelTransformer is pretty much as described above and delegates to Camel's TypeConverter mechanism. So we could add all the known types to the transforms.xml above and if users have thier own custom type converters they can package them with their deployment and they will get picked up upon deployment.

Thoughts?

/Daniel

[kcbabo]

Looks reasonable to me.  For the long term, it would be great if we could have a solution where we can just rely on the Camel converter definitions directly instead of creating a separate list of them in our <transform> config.  One way to do that would be to use the Camel converter definitions as a source and transform/generate the transformer definitions as part of our build.  Another way would be for the Camel component to get a hook to the SwitchYard transformer registry and add transformers dynamically at runtime.  Not sure how workable that second idea is, but it would eliminate the duplicate definitions.  Not something we have to do immediately, but it would be good to think about.

[beve]

For the long term, it would be great if we could have a solution where we can just rely on the Camel converter definitions directly instead of creating a separate list of them in our <transform> config.

I agree....is going to be hell adding them

One way to do that would be to use the Camel converter definitions as a source and transform/generate the transformer definitions as part of our build. 

Never thought about that. Might take quick look at what could be done here.

Another way would be for the Camel component to get a hook to the SwitchYard transformer registry and add transformers dynamically at runtime.  Not sure how workable that second idea is, but it would eliminate the duplicate definitions.  Not something we have to do immediately, but it would be good to think about.

I agree, that would be much better.

[beve]

One way to do that would be to use the Camel converter definitions as a source and transform/generate the transformer definitions as part of our build.

I've played around with this and pushed an initial suggestion. At the moment I've just used this to generate the transforms.xml, but we should be able to uses this either at deploy/build time, or at runtime.

/Daniel

[kcbabo]

Yikes!  Good thing you didn't have to write that transform file by hand!

So for now, I guess the approach will be to include this file in the camel component archive.  I don't think we have the right hook in place today for you to register these transformers dynamically in the component.  That would require a reference to the Domain.  So for now we can go with a static definition and look to make it more dynamic in 0.2.