Example of simple reqular-expression based substitution in Camel Route

[olivern]

Hi -

I have a application where I have incoming XML messages that contain several namespaces that have embedded version numbers in the namespace. To address some backwards-compatibility scenarios I want to substitute the namespaces with ones containing earlier version numbers.  The substitutions are along the lines of:

  xmlns:ns1="http://domain/schemas/simpleService/1.4"  

  to

  xmlns:ns1="http://domain/schemas/simpleService/1.3"

and what I really am after is the ability to 'downcast' incoming schema namespaces with the same major version number ('1' in this case) to the version that the service understands.  So in this example I want to convert namespaces of the form .../1.X to .../1.3.

There may be multiple namespace substitution rules defined for each incoming message, and I'd like to do the transformation in one routing step if possible.

I'm not an XLST guru, and the XSLT syntax to replace namespaces in this fashion looks a bit tough to figure out, so I'm hoping there is a simpler regular expression-based way of doing this in Camel. I looked around for a similar example but didn't come across anything.

Anyone know of a simple way of doing this?

Thanks,

-Oliver

[janstey]

This could probably be improved a bit, but you can do pretty much whatever you want to the message body using a custom bean or Processor. Something like this should work fine.

    protected RouteBuilder createRouteBuilder() {
        return new RouteBuilder() {
            public void configure() {
                from("direct:in").beanRef("myBean", "doReplace");
            }
        };
    }

    public static class MyBean {
        public void doReplace(@Body String body, Exchange exchange) {
            Pattern pattern = Pattern.compile("1.4");
            exchange.getIn().setBody(pattern.matcher(body).replaceAll("1.3"));            
        }
    }

Cheers,

Jon

[olivern]

Ok, thanks....

I'm thinking that maybe a general component that takes a sed/perl-like regexpr syntax for string match /replace would be useful. The idea would be to allow for a single expression inline, or a set of expressions stored in a file (similar to how Perl works).  I'll give that a shot...

If anyone has any design guidance/idea for such a component, let me know!

-Oliver

[janstey]

Oliver,

Yeah, would be nicer to have built in support for this. Perhaps just a custom Processor which is linked up into the DSL would be sufficient?

By the way, there is already a bit of regex stuff in camel-core (which I just found ). So my original example can be shortened to

from("direct:in").setBody(ExpressionBuilder.regexReplaceAll(body(), "1.4", "1.3")); 

[janstey]

Made this a little nicer in CAMEL-1357. Now you can do this

from("direct:in").setBody(regexReplaceAll(body(), "1.4", "1.3"));

which essentially is equivalent to sed 's/1.4/1.3/g' on the body.

[olivern]

Rapid action - thanks!

I'm using the Spring XML configuration (and still learning it).  Do you happen to have a snippet illustrating how to do the string replace using XML?

-Oliver

[davsclaus]

Ha ha now we we got to one of the big short comings of the Spring XML. You can not invoke code easily.

All these nice supporter for the DSL we have in Java DSL, ExpressionBuilder, PredicateBuilder, ValueBuilder etc. can be imported static so you get this nice fluent syntax.

However in Spring DSL you can not do this unless we go create special XML tags for each and every operation, and that is tedious and not easy to keep in sync.

So what you usually have to do in Spring DSL is to use some sort of scripting language to invoke the operation yourself.

We have a ticket for this issue in JIRA, to think about how to improve that for Camel 2.0 so you can invoke these builder methods nice and easy using a few XML tags.

Any feedback is appreciated. How would you like the XML to be for such an operation, to eg. do a regex replace all on the body?

I recently added a TokenizerExpression to Camel 2.0 that operates on the body. But that is just fixing a little stratch, eg with the Splitter where you usually split a body using some sort of token.

Maybe we can add a generic xxxExpression tag to the XML where you can specifiy the operation as an attribute/content etc?

But yet agin that might start to get a bit messy and ugly.

[janstey]

So yeah, as Claus says here we don't have a perfect mapping between Java and Spring DSLs yet. Its quite a tedious process really...

But in the meantime, you can do something like this in the Spring DSL

<from uri="direct:in">
<setBody>
  <el>${in.body.replaceAll('1.4','1.3')}</el>
</setBody>
...

Just make sure to add the following dependency to your pom

<dependency>
  <groupId>org.apache.camel</groupId>
  <artifactId>camel-juel</artifactId>
  <version>your version of FUSE MR</version>
</dependency>

[olivern]

Ok - thanks for the info (and the overall background on the XML issues)

I'll ponder the XML issue a bit. Providing a clean Camel-plugin model in the XML world would definitely be a good thing.  I think it probably comes down to a choice between soft-typed fields (rather than new XML tags) and XML tags in a separate XML namespace for each plugin.

One interesting model is the extension mechanism used by ATOM. In ATOM it is possible to add new elements anywhere in an XML document, as long as the new elements reside in their own namespace that is separate from the documents base namespace. The impact of using this model (the XML 'open content model') is that validation based on XmlSchema 1.0 doesn't work and you have to use alternate validation methods (e.g. RelaxNG, Schematron).  In XmlSchema 1.1 they added explicit support for an open content model for this very reason, but adoption of XmlSchema 1.1 is probably still a ways off!

Thanks again for the feedback!

-Oliver

[davsclaus]

Hi Oliver

Thanks for the insight.

Hadrian, also a Camel rider, have thought about some sort of pluggable DSL in Camel, so each component can provide its own DSL language for: Java, Spring, Annotations and optionally in Groovy, Scala, Ruby and whatnot.

As its now we have to add DSL support directly in the camel core.

The people behind the open health care framework based on top of Apache Camel with Groovy have kinda proven the dynamic nature of Groovy it would be possible to add dynamic DSL to the table.

Well its a bit side tracked. Back to the XML world

Maybe with Spring 3.0 and its EL language it gets very popular, that end users adopt this and we can consider that a good choice for expressing predicate and expressions in the XML.

[olivern]

Hi -

I created a bare-bones Camel SED processor/component to filter message bodies based on SED expressions like:

's/searchString/replaceString/g'

The expressions can be specified in a configuration file that conforms to the usual UNIX SED/Perl script conventions, so it is theoretically possible to use SED/Perl to test the expressions before setting up the routes. Code attached in case anyone finds it useful.

I do have one problem though - I tried to create a SedComponent from the SedProcessor using the ProcessorEndpoint helper class. (see the code in the attachment).  It compiles ok, but at run time I get the following error if I try to use a URI with the 'sed:' scheme instead of just the plain bean reference (which works fine).  I don't quite understand why the endpoint isn't getting created/resolved properly.  Any ideas appreciated! (the error should be re-creatable by uncommenting the component-style variant of the SED route in the sample camelContext.xml, and running the usual 'mvn camel:run'

-Oliver

-

---

Failed: org.apache.camel.ResolveEndpointFailedException: Failed to resolve endpoint: sed:./src/mai n/resources/modifyXmlNamespaceVersion due to: java.lang.NullPointerException

org.apache.camel.ResolveEndpointFailedException: Failed to resolve endpoint: sed:./src/main/resourc es/modifyXmlNamespaceVersion due to: java.lang.NullPointerException

        at org.apache.camel.impl.DefaultCamelContext.getEndpoint(DefaultCamelContext.java:351)

        at org.apache.camel.util.CamelContextHelper.getMandatoryEndpoint(CamelContextHelper.java:53 )

        at org.apache.camel.model.RouteType.resolveEndpoint(RouteType.java:99)

        at org.apache.camel.impl.DefaultRouteContext.resolveEndpoint(DefaultRouteContext.java:106)

        at org.apache.camel.impl.DefaultRouteContext.resolveEndpoint(DefaultRouteContext.java:112)

        at org.apache.camel.model.SendType.resolveEndpoint(SendType.java:57)

        at org.apache.camel.model.SendType.createProcessor(SendType.java:51)

        at org.apache.camel.model.ProcessorType.createOutputsProcessor(ProcessorType.java:2011)

        at org.apache.camel.model.ProcessorType.createOutputsProcessor(ProcessorType.java:102)

        at org.apache.camel.model.InterceptorRef.createProcessor(InterceptorRef.java:66)

        at org.apache.camel.model.ProcessorType.makeProcessor(ProcessorType.java:1895)

        at org.apache.camel.model.ProcessorType.addRoutes(ProcessorType.java:106)

        at org.apache.camel.model.RouteType.addRoutes(RouteType.java:220)

        at org.apache.camel.model.RouteType.addRoutes(RouteType.java:89)

        at org.apache.camel.impl.DefaultCamelContext.startRouteDefinitions(DefaultCamelContext.java :644)

[davsclaus]

Oliver

That is fantastic with the SED component. You have come a long way with Camel since you know how to create your own components now.

I really think its useful.

Another sign the community is growing since we get more and more community granted components, data formats, languages etc. to Camel.

I suggest you take a look how the Velocity component is done. It might help you a bit to build it on top of ResourceBasedEndpoint as you then can leverage the existing flexible code to find resoures in classpath, file, etc. It is in fact Spring IO resource that does it. Then you dont need to fiddle that yourself. Then end users can write: classpath://xxx. file:// or jar:// to lookup the resource file.

And it might also have the help for the other NPE error.

If you could get this done then we can take a look at it again. And see if we can get the last pieces working.

[olivern]

Ok - thanks for the tip. I'll give it a shot!

Might be a week or two before I get a chance to look at Velocity - it's looking like a document generation week for me 

-Oliver