Batch processing library - QueuJ
div Sep 24, 2010 3:58 AMI hope this is the correct place to post this. We have recently open sourced a batch processing library that integrates well into Seam.
http://sourceforge.net/projects/queuj/
Some of the features:
- Sophisticated concurrency controls by extending QueueRestriction. Default is to run all processes concurrently.
- Optional persistence with transactional sections so a process will restart from the section that was running after a JVM restart.
- Scheduling of processes for the future, recurring on a schedule or run immediately with options to catch up on missed runs, run once for any missed runs or forget missed runs.
- Separate schedule for recovery of failed processes.
- Configure multiple hierarchical queues with each inheriting properties of the parent queue.
There is no documentation on the site yet, but there is a sample seam application in the source repository: http://queuj.svn.sourceforge.net/viewvc/queuj/queuJSeamSample/trunk/
An example of creating a queue with a simple QueueRestriction in an initialisation bean:
@Name("initialiser") @Startup(depends={"DEFAULT_QUEUE"}) @Scope(ScopeType.APPLICATION) public class Initialiser { @In public Queue<SeamProcessBuilder> DEFAULT_QUEUE; @Out public Queue<SeamProcessBuilder> SAMPLE_QUEUE; @Create public void init() { QueueBuilder<SeamProcessBuilder> qb = DEFAULT_QUEUE.newQueueBuilder(); qb.setQueueRestriction(new TestQueueRestriction()); SAMPLE_QUEUE = qb.newQueue(); } public static class TestQueueRestriction extends QueueRestriction { @Override protected boolean canRun(Queue queue, Process process) { int running = process.getContainingServer().getProcessIndexes().countOfRunningProcesses(queue); int waiting = process.getContainingServer().getProcessIndexes().countOfWaitingToRunProcesses(queue); return (running + waiting) < 100; } } }
And an example of running a process with the sample queue:
@In public Queue<SeamProcessBuilder> SAMPLE_QUEUE; public void runJob1() { SeamProcessBuilder pb = SAMPLE_QUEUE.newProcessBuilder(Locale.getDefault()); pb.setProcessName("Test 1"); pb.setProcessDescription("Test 1"); pb.setProcessDetails(TestRunner.class, "run", new Class[] {}, new Object[] {}); pb.setProcessPersistence(false); pb.setParameter("random", Math.random()); pb.newProcess(); } @Name("testRunner") public class TestRunner { @In public double random; public void run() { ......
Hope people find this useful.
Cheers,
Dave.