I think we fixed something with the file -> split and using delete or move options on the file. So try out Camel 2.5 when its released.
The tokenizer uses java.util.Scanner, so go read its javadoc how to use other tokens than \n etc.
Thank you davclaus for that information. I do hope that I can figure mine out.