2 Replies Latest reply on Sep 27, 2010 7:30 PM by div

    Batch processing library - QueuJ

    div

      I hope this is the correct place to post this. We have recently open sourced a batch processing library that integrates well into Seam.


      http://sourceforge.net/projects/queuj/


      Some of the features:



      • Sophisticated concurrency controls by extending QueueRestriction. Default is to run all processes concurrently.

      • Optional persistence with transactional sections so a process will restart from the section that was running after a JVM restart.

      • Scheduling of processes for the future, recurring on a schedule or run immediately with options to catch up on missed runs, run once for any missed runs or forget missed runs.

      • Separate schedule for recovery of failed processes.

      • Configure multiple hierarchical queues with each inheriting properties of the parent queue.



      There is no documentation on the site yet, but there is a sample seam application in the source repository: http://queuj.svn.sourceforge.net/viewvc/queuj/queuJSeamSample/trunk/


      An example of creating a queue with a simple QueueRestriction in an initialisation bean:


      @Name("initialiser")
      @Startup(depends={"DEFAULT_QUEUE"})
      @Scope(ScopeType.APPLICATION)
      public class Initialiser {
      
          @In
          public Queue<SeamProcessBuilder> DEFAULT_QUEUE;
      
          @Out
          public Queue<SeamProcessBuilder> SAMPLE_QUEUE;
      
          @Create
          public void init() {
              QueueBuilder<SeamProcessBuilder> qb = DEFAULT_QUEUE.newQueueBuilder();
              qb.setQueueRestriction(new TestQueueRestriction());
              SAMPLE_QUEUE = qb.newQueue();
          }
      
          public static class TestQueueRestriction extends QueueRestriction {
      
              @Override
              protected boolean canRun(Queue queue, Process process) {
                  int running = process.getContainingServer().getProcessIndexes().countOfRunningProcesses(queue);
                  int waiting = process.getContainingServer().getProcessIndexes().countOfWaitingToRunProcesses(queue);
                  return (running + waiting) < 100;
              }
          }
      }



      And an example of running a process with the sample queue:


          @In
          public Queue<SeamProcessBuilder> SAMPLE_QUEUE;
      
          public void runJob1() {
      
              SeamProcessBuilder pb = SAMPLE_QUEUE.newProcessBuilder(Locale.getDefault());
              pb.setProcessName("Test 1");
              pb.setProcessDescription("Test 1");
              pb.setProcessDetails(TestRunner.class, "run", new Class[] {}, new Object[] {});
              pb.setProcessPersistence(false);
              pb.setParameter("random", Math.random());
              pb.newProcess();
          }
      
      @Name("testRunner")
      public class TestRunner {
      
          @In
          public double random;
      
          public void run() {
              ......



      Hope people find this useful.


      Cheers,
      Dave.