8 Replies Latest reply on May 31, 2013 11:21 AM by tomjenkinson

    Periodic Recovery runs at every 2 minutes in jboss as 7.1 (com.arjuna.ats.arjuna)

    rameshchokkapu

      Hi Gud mrng..

       

      please answer  for below  question anyone.

       

      Periodic Recovery is running in my jboss as 7 server at every 2 minutes. In that time my server working as 'stop the world'. after recovery completed then again its working normal... recovery taking too long time means ( between 30 secs and  120 secs).


      And i want to increase this 2 minutes to 10 minutes.. and where should i configure this time in my standalone xml...

      (or)

      how can i disable this Periodic Recovery?

       

      In standalone.xml :

       

      <subsystem xmlns="urn:jboss:domain:transactions:1.1">

                  <core-environment>

                      <process-id>

                          <uuid/>

                      </process-id>

                  </core-environment>

                  <recovery-environment socket-binding="txn-recovery-environment" status-socket-binding="txn-status-manager" recovery-listener="false"/>

                  <coordinator-environment default-timeout="300"/>

         </subsystem>

       

       

      and

      In Hibernate :

       

      cfg.setProperty("hibernate.transaction.factory_class","org.hibernate.transaction.JTATransactionFactory");

      cfg.setProperty("hibernate.transaction.jta.platform", "org.hibernate.service.jta.platform.internal.JBossAppServerJtaPlatform");

       

      In Ejb :

       

      Bean managed trasations. but i am not using any transations.

       

       

      how can i disable transation management in jboss as 7 or at  Ear Application level ?

       

       

      04:56:11,392 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) PeriodicRecovery: background thread Status <== SCANNING

      04:56:11,393 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) PeriodicRecovery: background thread scanning

      04:56:11,393 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) Periodic recovery first pass at Thu, 30 May 2013 04:56:11

      04:56:11,394 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) AtomicActionRecoveryModule first pass

      04:56:11,394 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) processing /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction transactions

      04:56:11,395 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) 

      04:56:11,396 DEBUG [com.arjuna.ats.txoj] (Periodic Recovery) TORecoveryModule - first pass

      04:56:11,396 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) 

      04:56:11,397 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) Local XARecoveryModule - first pass

      04:56:11,397 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) 

      04:56:21,398 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) Periodic recovery second pass at Thu, 30 May 2013 04:56:21

      04:56:21,398 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) AtomicActionRecoveryModule second pass

      04:56:21,399 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) 

      04:56:21,399 DEBUG [com.arjuna.ats.txoj] (Periodic Recovery) TORecoveryModule - second pass

      04:56:21,399 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) 

      04:56:21,400 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) Local XARecoveryModule - second pass

      04:56:21,400 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) Local XARecoveryModule.transactionInitiatedRecovery completed

      04:56:21,401 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) Local XARecoveryModule.resourceInitiatedRecovery completed

      04:56:21,401 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) 

      04:56:21,402 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) PeriodicRecovery: background thread Status <== INACTIVE

      04:56:21,402 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) PeriodicRecovery: background thread backing off

       

       

       

       

      please anyone help me..

       

      thank u for advance..

        • 1. Re: Periodic Recovery runs at every 2 minutes in jboss as 7.1 (com.arjuna.ats.arjuna)
          sfcoy

          You can change the timeout using the default-timeout parameter in the coordinator-environment element.

           

          If your code is running from an EJB you can use the javax.ejb.AccessTimeout annotation instead. (EJBs are super easy these days so it's worth checking this out - an EJB can even be in your webapp if you want).

           

          But really, you should try to avoid these "stop the world" scenarios. If you tell us what you're trying to do maybe we can can come up with a better solution.

          • 2. Re: Periodic Recovery runs at every 2 minutes in jboss as 7.1 (com.arjuna.ats.arjuna)
            rameshchokkapu

            Thank u very much for reply sir,

             

            <coordinator-environment default-timeout="300"/> (5 minutes). this already existed defaultly in my standalone.xml

            but PeroidicRecovery Running in every  2 minitues... where is that 2 minitues configuration in my server, i didnt get still..

            but i have checked in source code in com.arjuna.ats.arjuna.coordinator.TransactionReaper class

            In that code declared as "public static final long defaultCheckPeriod = 120000; // in milliseconds".

            i want to change this time.

             

            how can i achieve from this? i need this Recovery in every 1 hour..

            (or)

            how can i disable this Periodic Recovery Completely in my jboss as 7 server?

             

            please help me sir..

            • 3. Re: Periodic Recovery runs at every 2 minutes in jboss as 7.1 (com.arjuna.ats.arjuna)
              jaikiran

              I don't think the transaction recovery thread should cause "stop the world" problem. Have you taken thread dumps and did other diagnostics tests to see what's causing this "stop the world" problem? The recovery itself taking 30 to 120 seconds for each run looks wrong.

              • 4. Re: Periodic Recovery runs at every 2 minutes in jboss as 7.1 (com.arjuna.ats.arjuna)
                jaikiran

                I've moved this thread to the JBoss Transactions forum, at least for now, since someone from that team might have a better understanding of what's going on.

                • 5. Re: Periodic Recovery runs at every 2 minutes in jboss as 7.1 (com.arjuna.ats.arjuna)
                  rameshchokkapu

                  Thank u very much  sir,

                   

                  how can i disable this Periodic Recovery Completely in my jboss as 7 server?

                  (or)

                  i need this Recovery in every 1 hour.. ?

                   

                  please help me...

                  Thank u

                  • 6. Re: Periodic Recovery runs at every 2 minutes in jboss as 7.1 (com.arjuna.ats.arjuna)
                    tomjenkinson

                    Hi Ramesh,

                     

                    If you look at your log in your original post periodic recovery is not taking anywhere near 30 seconds.

                    04:56:11,392 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) PeriodicRecovery: background thread Status <== SCANNING

                    04:56:21,402 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) PeriodicRecovery: background thread backing off

                     

                    Its taking 10 seconds, which corresponds nicely to the 10 second sleep which takes place between:

                    04:56:11,397 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) 

                    04:56:21,398 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) Periodic recovery second pass at Thu, 30 May 2013 04:56:21

                     

                    Overall this is not causing your "stop the world" problem, it may coincide with another event that happens then but as I see it recovery is doing almost nothing.

                     

                    If you want to change the values related to recovery intervals:

                    -DRecoveryEnvironmentBean.periodicRecoveryPeriod=<default is 120> -DRecoveryEnvironmentBean.recoveryBackoffPeriod=<default is 10>

                     

                    Hope that helps,

                    Tom

                    • 7. Re: Periodic Recovery runs at every 2 minutes in jboss as 7.1 (com.arjuna.ats.arjuna)
                      rameshchokkapu

                      Thank u very much to Tom sir and all,

                       

                      -DRecoveryEnvironmentBean.periodicRecoveryPeriod was very helpful to me.. thank you alot..

                       

                      but

                       

                      please look at my log below TIME before "<== SCANNING"  and "after backing off" taking long time.. why it was taking that much time for SCANNING and backing off.

                      INFO [stdout] are my System.out.println statements.

                       

                      04:58:59,314 INFO  [stdout] (pool-10-thread-3)  Dubug @ stmt :: in RQ

                      05:00:07,719 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) PeriodicRecovery: background thread Status <== SCANNING

                      ---------------------------------------

                      ---------------------------------------

                      05:00:17,898 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) PeriodicRecovery: background thread backing off

                      05:01:35,796 INFO  [stdout] (pool-10-thread-1) Debug @ stmt Do HandShake hostName:port

                       

                      server.log :

                       

                      04:58:59,314 INFO  [stdout] (pool-10-thread-3)  Dubug @ stmt :: in RQ

                      05:00:07,719 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) PeriodicRecovery: background thread Status <== SCANNING

                      05:00:07,744 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) PeriodicRecovery: background thread scanning

                      05:00:07,744 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) Periodic recovery first pass at Fri, 31 May 2013 05:00:07

                      05:00:07,745 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) AtomicActionRecoveryModule first pass

                      05:00:07,892 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) processing /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction transactions

                      05:00:07,893 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) 

                      05:00:07,893 DEBUG [com.arjuna.ats.txoj] (Periodic Recovery) TORecoveryModule - first pass

                      05:00:07,893 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) 

                      05:00:07,894 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) Local XARecoveryModule - first pass

                      05:00:07,894 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) 

                      05:00:17,894 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) Periodic recovery second pass at Fri, 31 May 2013 05:00:17

                      05:00:17,895 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) AtomicActionRecoveryModule second pass

                      05:00:17,895 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) 

                      05:00:17,895 DEBUG [com.arjuna.ats.txoj] (Periodic Recovery) TORecoveryModule - second pass

                      05:00:17,896 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) 

                      05:00:17,896 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) Local XARecoveryModule - second pass

                      05:00:17,896 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) Local XARecoveryModule.transactionInitiatedRecovery completed

                      05:00:17,897 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) Local XARecoveryModule.resourceInitiatedRecovery completed

                      05:00:17,897 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) 

                      05:00:17,898 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) PeriodicRecovery: background thread Status <== INACTIVE

                      05:00:17,898 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) PeriodicRecovery: background thread backing off

                      05:01:35,796 INFO  [stdout] (pool-10-thread-1) Debug @ stmt Do HandShake hostName:port

                       

                      thank u for advance..

                      • 8. Re: Periodic Recovery runs at every 2 minutes in jboss as 7.1 (com.arjuna.ats.arjuna)
                        tomjenkinson

                        Hi Ramesh,

                         

                        I can confirm that the line immediately after the one that is printed is a sleep for "RecoveryEnvironmentBean.periodicRecoveryPeriod" seconds.

                         

                        I would therefore assume that there is some other activity in play that is causing the sleep. Why don't you get a few thread dumps imediately after:

                         

                        05:00:17,898 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) PeriodicRecovery: background thread backing off

                         

                        To see what is happening?

                         

                        Tom