10 Replies Latest reply on Jul 9, 2009 10:29 PM by nathan dennis

    Quartz Cron Job Executing Twice

    nathan dennis Expert

      i have a project, unfortunately that has already gone into production, where it was recently discovered that the  all our quartz jobs were executing twice.... i can't find anything on google or in the forum where others have experienced this behavior when not in a cluster,,, which we are not.


      jboss 4.2.2
      seam 2.0.1.GA


      some code


      components.xml


      <!-- Install the QuartzDispatcher -->
          <async:quartz-dispatcher/>
      



      scheduler.... it would appear the problem is here


      import java.util.Calendar;
      import java.util.Date;
      import org.jboss.seam.annotations.In;
      import org.jboss.seam.annotations.Logger;
      import org.jboss.seam.annotations.Name;
      import org.jboss.seam.annotations.Observer;
      import org.jboss.seam.annotations.Transactional;
      import org.jboss.seam.async.QuartzTriggerHandle;
      import org.jboss.seam.log.Log;
      @Transactional
      @Name("startTimecap")
      public class StartTimecap {
      
           @Logger
           private Log log;     
           
           @In
           TimecapProcessor timecapProcessor;
      
      
           /**
            * This is called by seam after initialization
            * has finished.
            */
           @Observer("org.jboss.seam.postInitialization")
           public void observer() 
           {
                try
                {
                     log.info("Trying to Start Timecap Cron Job");
                     Calendar cal = Calendar.getInstance ();
                     cal.set (2040, Calendar.MAY, 10);
                     QuartzTriggerHandle handle = timecapProcessor.scheduleTimecapQue(new Date(), "0 4 2 * * ?", cal.getTime());
                
                }
                catch(Exception e)
                {
                     e.printStackTrace();
                     log.info("Operation Failed");
                }
           }
      }
      


      this is executed twice


      @Name("timecapProcessor")  
      @AutoCreate
      public class TimecapProcessor {
      
           
           @Logger Log log;
           
           @In
           EntityManager entityManager;
           
           @SuppressWarnings("unchecked")
           @Asynchronous
           @Transactional @TransactionAttribute(TransactionAttributeType.REQUIRES_NEW)
           public QuartzTriggerHandle scheduleTimecapQue(@Expiration Date when, @IntervalCron String cron, @FinalExpiration Date endDate){
      
                log.info("\n\n\n\n-------------------------------------" +
                          "Scheduler Started, Executing Timecap Job ----------------\n\n\n\n");
      
                //do your processing here
                
                return null;
           }     
      }
      



      im not sure if this is a bug or what.. i had to upgrade the version of seams several times over the course of the project.
      hopefully someone will have seen this before or possibly have a hint as to what is going on.
      i'm not beyond writing something that will look to see if the job is already scheduled if someone could point me in the right direction.


      any help would be greatly appreciated.

        • 1. Re: Quartz Cron Job Executing Twice
          Nikolay Elenkov Master

          Do you have multiple WAR's in an EAR? If so, the org.jboss.seam.postInitialization will be raised
          when WAR's Seam is initialized. Then you might get multiple copies of the same job scheduled, since
          StartTimecap.observer is called multiple times.


          • 2. Re: Quartz Cron Job Executing Twice
            Gregory Nikle Apprentice

            Hi,
            try this


                 public QuartzTriggerHandle scheduleTimecapQue(@Expiration Date when, @IntervalCron String cron, @FinalExpiration Date endDate){
            
            synchronized (TimecapProcessor.class) {
                      log.info("\n\n\n\n-------------------------------------" +
            }
                      return null;
                 }
            



            this should help :)


            • 3. Re: Quartz Cron Job Executing Twice
              nathan dennis Expert

              correction to my seam revision : Seam 2.0.2.SP1


              you are on to something here. unfortunately, it isnt that obvious. i started looking a little further up in the log and saw the project actually trying to start twice.. it was throwing 


              [Initialization] two components with same name, higher precedence wins: org.jboss.seam.core.locale.
              


              kind of warnings.


              im not sure if this would cause it.. but i think i might have fat fingered the seam's lib when i was upgrading the revision.


              would having two separate revisions of the seams-core cause this sort of behavior?


              thanks for the quick responses.




              • 4. Re: Quartz Cron Job Executing Twice
                nathan dennis Expert

                i tried synchronized no such luck.


                updated all the libs. stared at the exploded ear for 2 hours trying to find a war that isnt there.
                searched jira or issued related to


                org.jboss.seam.postInitialization
                


                being called twice. any other ideas?

                • 5. Re: Quartz Cron Job Executing Twice
                  Gregory Nikle Apprentice

                  Can u post your components.xml ?

                  • 6. Re: Quartz Cron Job Executing Twice
                    nathan dennis Expert

                    components.xml
                    i thought at first it might be the mulipart filter,, but i tried it without that with no luck


                    <?xml version="1.0" encoding="UTF-8"?>
                    <components xmlns="http://jboss.com/products/seam/components"
                                xmlns:core="http://jboss.com/products/seam/core"
                                xmlns:persistence="http://jboss.com/products/seam/persistence"
                                xmlns:drools="http://jboss.com/products/seam/drools"
                                xmlns:bpm="http://jboss.com/products/seam/bpm"
                                xmlns:security="http://jboss.com/products/seam/security"
                                xmlns:mail="http://jboss.com/products/seam/mail"
                                xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
                                xmlns:async="http://jboss.com/products/seam/async"
                                xmlns:transaction="http://jboss.com/products/seam/transaction"
                                xsi:schemaLocation=
                                    "http://jboss.com/products/seam/core http://jboss.com/products/seam/core-2.0.xsd 
                                     http://jboss.com/products/seam/persistence http://jboss.com/products/seam/persistence-2.0.xsd 
                                     http://jboss.com/products/seam/drools http://jboss.com/products/seam/drools-2.0.xsd
                                     http://jboss.com/products/seam/bpm http://jboss.com/products/seam/bpm-2.0.xsd
                                     http://jboss.com/products/seam/security http://jboss.com/products/seam/security-2.0.xsd
                                     http://jboss.com/products/seam/mail http://jboss.com/products/seam/mail-2.0.xsd
                                     http://jboss.com/products/seam/components http://jboss.com/products/seam/components-2.0.xsd
                                     http://jboss.com/products/seam/async http://jboss.com/products/seam/async-2.0.xsd">
                    
                       <core:init debug="false" jndi-pattern="@jndiPattern@"/>
                         
                       <core:manager concurrent-request-timeout="60000" 
                                     conversation-timeout="900000" 
                                     conversation-id-parameter="cid"/>
                        
                       <persistence:managed-persistence-context name="entityManager"
                                                         auto-create="true"
                                          persistence-unit-jndi-name="java:/memoriesEntityManagerFactory"/>                          
                         
                    <!-- drools security rules config -->
                       <drools:rule-base name="securityRules">
                           <drools:rule-files>
                               <value>/META-INF/security.drl</value>
                           </drools:rule-files>
                       </drools:rule-base>
                    
                    <!-- login authenticator to use -->
                       <security:identity security-rules="#{securityRules}" 
                                                  authenticate-method="#{authenticator.authenticate}" 
                                                  remember-me="true"/>
                       
                       <event type="org.jboss.seam.security.notLoggedIn">
                           <action execute="#{redirect.captureCurrentView}"/>
                       </event>
                       <event type="org.jboss.seam.security.loginSuccessful">
                           <action execute="#{redirect.returnToCapturedView}"/>
                       </event>
                    <!-- jboss mail config -->
                       <mail:mailSession host="foo.com" port="25" username="do-not-replyfoo.com" password="foo" tls="true" ssl="true"/>
                            
                    
                    
                    
                    <!-- Install the QuartzDispatcher -->
                        <async:quartz-dispatcher/>
                       
                    
                         <component class="org.jboss.seam.web.MultipartFilter">
                           <property name="createTempFiles">true</property>
                           <property name="maxRequestSize">10000000</property>
                           <property name="disabled">true</property>
                         </component>
                    <!-- factory that contains the sessionTimeoutSeconds for timeout.js -->
                        <factory name="sessionTimeoutSeconds" 
                                     scope="SESSION" 
                                     value="#{facesContext.externalContext.getSession(true).getMaxInactiveInterval()}"/>
                         <factory name="remoteAddr" value="#{facesContext.externalContext.request.remoteAddr}" auto-create="true"/>                 
                    </components>
                    

                    • 7. Re: Quartz Cron Job Executing Twice
                      Nikolay Elenkov Master

                      A couple of questions:



                      • did you confirm that StartTimecap.observer is called twice?

                      • are you saving your Quartz jobs in the database?



                      You can use this code to list the jobs currently scheduled:


                      QuartzDispatcher dispatcher = QuartzDispatcher.instance();
                      Scheduler scheduler = dispatcher.getScheduler();
                      
                      String[] jobGroups = scheduler.getJobGroupNames();
                      
                      for (int i = 0; i < jobGroups.length; i++) {
                        log.debug("Group: " + jobGroups[i]);
                        String[] jobsInGroup = scheduler.getJobNames(jobGroups[i]);
                      
                        for (int j = 0; j < jobsInGroup.length; j++) {
                          log.debug("\t" + jobsInGroup[j]);
                        }
                      }
                      



                      As a workaround for you problem, you can list the jobs and not call timecapProcessor.scheduleTimecapQue
                      if your job is already scheduled.


                      Not sure about the root cause of the problem though. Does it happen when you deploy your app
                      as an ear? (not exploded)

                      • 8. Re: Quartz Cron Job Executing Twice
                        nathan dennis Expert

                        im at a loss... it happens when i deploy as a ear as well.


                        i took a cheap shot at it by counting cron jobs since im not sure how to assign names to them based on the seams examples. one issue i ran into doing this was unless QuartzTriggerHandle is called the DEFAULT group is never initialized so i cant count jobs. i had to create a dummy job that gets scheduled twice.


                        this is not a good fix. i would really like to find the root of the issue and correct it because if one job fails to start a critical job might get scheduled twice.


                        thanks for the responses... are there any other ideas where i might look?

                        • 9. Re: Quartz Cron Job Executing Twice
                          Nikolay Elenkov Master

                          Nathan Dennis wrote on Jul 09, 2009 11:45:


                          i took a cheap shot at it by counting cron jobs since im not sure how to assign names to them based on the seams examples. one issue i ran into doing this was unless QuartzTriggerHandle is called the DEFAULT group is never initialized so i cant count jobs. i had to create a dummy job that gets scheduled twice.



                          Seam assigns automatically generated names to your jobs (UUID's). But you can get the job name from QuartzTriggerHandle like so:


                          String jobName = handle.getTrigger().getJobName();
                          



                          Then it should be easy to check if the job is already scheduled.



                          thanks for the responses... are there any other ideas where i might look?


                          Attach a debugger and see who/what is scheduling your jobs.


                          Btw, you didn't answer my questions:



                          • did you confirm that StartTimecap.observer is called twice?

                          • are you saving your Quartz jobs in the database?



                          If you are saving jobs in the database, the situation might
                          be a little more complicated.



                          • 10. Re: Quartz Cron Job Executing Twice
                            nathan dennis Expert

                            i apologize.
                            i did confirm that StartTimecap.observer is called twice.
                            and i wasnt saving my quartz jobs to a db, but it looks like it will be on my todo list if i have to retrieve those handles.


                            i started to post the debug... and i was searching through it for the relevant information and even got much of it copied into this field.
                            but i wanted to make the problem move before i made everyone loose interest with a 2 page debug output... so i tried a couple things,,, deploy on a fresh copy of jboss 4.2.2-- no luck... create an new project and try some of the same code... which worked. so then i set my mind to moving the src and xhtml over into the new project... so far so good... moved the configuration one component at the time and watched for it to break.... and what do you know... i found it... it was bad stupid. i have our test platform set up just like a production deployment,, apache front end, same OS version and everything.


                            jboss-web.xml


                            <jboss-web>
                                <context-root>foo</context-root>
                                 <virtual-host>www.foo.com</virtual-host>
                                <virtual-host>test.foo.com</virtual-host>
                            </jboss-web>
                            



                            WHAT THE HECK IS THAT DOING THERE???? two war's will do it. and believe it or not... restarting the same war will do it too. im about to go through the cvs logs and the remainder of this patch will be my foot work to someone's soon to be newly dented sitting device. 


                            thanks for the help guys... unfortunately this has been more of a learning experience than i wanted this week.