1 2 Previous Next 15 Replies Latest reply on Aug 13, 2010 2:19 AM by oferiko Go to original post
      • 15. Re: Very slow server start
        oferiko

        OK, we've tested the system, and it was loaded with 2.5M scheduled jobs  in just under 3 minutes.

        That is a very big improvement from the 5 hours we started with.

        We understand our use case is not the standard one for HornetQ. Our system has  millions of jobs it needs to process a day repeatedly. The time they get  processed is not "written in stone", so i can live with short delays.  There are a few dousens machines that process them, so the  quick distribution of the jobs and the ability to acknowledge or rolback a job is the thing we were looking for.

         

        We actually looked at quartz, but working with a DB to store the persistent jobs made the work rather slow, at least once we had more than 1M jobs, and since we need to get the jobs in multiple VMs.

         

        For now HornetQ, with the scheduled messages, seems to be the best solution for us, having a quick messaging system, that is also persistent and can also schedule future jobs.

         

        In case i realize that having too many scheduled messages is too much for HornetQ (though i'm hoping that clustering can help with that), i'm thinking about combining the two together.

        Having Quartz as a scheduler, and having one process working with it, but doing all the actual jobs distribution to the worker machines and using HornetQ, but i hope we won't have to get to that.

         

        We wil update you all once we'll continue our tests.

         

        We want to also note the great support we've been getting from the team at HornetQ, especially from Clebert Suconic, who was working hard to find out what the probelm was.

         

        Levona & Ofer

        1 2 Previous Next