We are looking to setup an ESB but I'm not sure what our best architecture option is.
We mostly have two types of jobs, one that would kicked of by watching for a file or by a timer and then the type is a just a massive download operation. It would download 500k or so small files (< 1 mb). This download job would be kicked off by messages appearing on the JMS queue.
I would like to scale the large download job out amongst several severs so that it can be handled as quick as possible.
On the other hand, if we cluster this setup - how can we guarantee that the jobs that are watching for a file or are timer based only get kicked off once?