How do I cleanly shutdown JBossTS when running outside of JBoss AS? (e.g. Tomcat)

[talawahdotnet]

I am trying to run JBossTS on a standalone Tomcat instance.  I want to use it as the JTA provider for Hibernate (more questions around that in a separate post).  How do I cleanly startup and shutdown all JBossTS related services/threads?

Right now I just have a simple ServletContextListener that is referenced in my web.xml

public class TransactionManagerListener implements ServletContextListener {

          final static Logger logger = LoggerFactory.getLogger(TransactionManagerListener.class);

          @Override
          public void contextInitialized(ServletContextEvent sce) {
                    logger.info("Starting Transaction Manager Threads");
                    TransactionReaper.instantiate();
                    RecoveryManager.manager().startRecoveryManagerThread();
                    TxControl.enable();
          }

          @Override
          public void contextDestroyed(ServletContextEvent sce) { 
                    logger.info("Stopping Transaction Manager Threads");
                    TxControl.disable();
                    RecoveryManager.manager().terminate();
                    TransactionReaper.terminate(true)
          }
}

Everything seems to start up fine but I do have some issues

     (a) Is this the right/safe orders of calls to make during startup...and even more importantly, shutdown.

     (b) One threads doesn't seem to get properly shutdown and Tomcat complains:

SEVERE: The web application [/cloud] appears to have started a thread named [Listener:1732] but has failed to stop it. This is very likely to create a memory leak.

I am running Tomcat 7.0.27 and everything is being deployed as a .war.  I know that ideally the Transaction Manager should be started as part of the server process rather than the context, but I am trying to deploy to a cloud environment so I have some limitations.  I have done a bit of Googling to find what I have so far, but now I am stuck.  If someone could point me in the right direction it would be greatly appreciated.

[talawahdotnet]

I have managed to get a little further. I took a peak at the source code and realized that calling TxControl.disable(true) will shut down the Transaction Status Manager and hence kill the Listener thread. So now my code looks like this:

          @Override
          public void contextDestroyed(ServletContextEvent sce) { 
                    logger.info("Stopping Transaction Manager Threads");
                    TxControl.disable();
                    RecoveryManager.manager().terminate();
                    TransactionReaper.terminate(true);
                    TxControl.disable(true);


          }

I do the first call to disable new transactions, shutdown the other components, then do a final call to kill the rest. So now the Listener thread gets killed, and Tomcats stops complaining, but I am still not sure about the order, furthermore I am wondering how I should factor the Hibernate session factory shutdown in the mix.  Right now app code and hibernate shutdown happen in contextDestroyed methods that get called before this one, so the order is effectively:

     // Shut down application batch jobs (most db calls done here)
     // Shut down application daemon services (rest of db calls done here)

     HibernateUtil.shutdown() // Calls SessionFactory.close()

     TxControl.disable();
     RecoveryManager.manager().terminate();
     TransactionReaper.terminate(true);
     TxControl.disable(true);

The expectation is that my application components will close off DB related transaction when they are shut down, but we all know things don't always go as expected.  Given my inexperience in the XA Transaction Management world, particularly when it comes to things like recovery, I was hoping to get some advice from somebody more seasoned.  For example is the any value to making the first cal to TxControl.disable() before calling SessionFactory.close() ?

[tomjenkinson]

Hi,

It depends on what you are wanting to achieve. An out of process recovery manager? JTS?

Some useful classes to start the recovery manager:

JTS: com/arjuna/ats/jbossatx/jts/RecoveryManagerService.java

JTA: com/arjuna/ats/jbossatx/jta/RecoveryManagerService.java

First, you must have either the jbossts-properties.xml configured appropriately or if you intend to programmatically configure the transaction/recovery manager then that must be done before you call any other JBTM API.

Next, to launch the recovery manager you could do:

     recoveryManagerService = new RecoveryManagerService();

     recoveryManagerService.create();

     recoveryManagerService.start();/stop();

To launch the transaction manager you could use your line or just this one:

     TransactionReaper.transactionReaper();

The shutdown order you have suggested seems relatively sane for what you are trying to achieve. Although I would be tempted to move to shutting down your transactions in an orderly manner first then:

App:down

Hibernate::down

Reaper::down

RecoveryManager::down

TXControl::down(true)

Hope that helps,

Tom

[talawahdotnet]

Hi Tom,

Thanks for responding.

Tom Jenkinson wrote:

Hi,

It depends on what you are wanting to achieve. An out of process recovery manager? JTS?

No JTS or out of process recovery necessary.  As a matter of fact, I'm not even sure any kind of recovery is needed in my use case. The only reason I am trying to deploy a transaction manager is to facilitate Hibernate/Infinispan integration (which seems like complete overkill). As am sure you are aware, trying to do this on Tomcat instead of JBoss (I am using Amazon's ElasticBeanstalk) turned out to be way more than I bargained for, but I figured it was a good chance to learn something new.  Given that Infinispan now uses tranasaction synchronization instead of enlisting itself as an XA Resource, is there much value to the recovery side of things if I am just connecting to a single MySQL server?  Can I disabe recovery altogether of is it needed to facilitate basic rollbacks?

Some useful classes to start the recovery manager:

JTS: com/arjuna/ats/jbossatx/jts/RecoveryManagerService.java

JTA: com/arjuna/ats/jbossatx/jta/RecoveryManagerService.java

Thanks. Those classes aren't in the maven artifact I am using (narayana-jta) but the source should be a good enough guide.

First, you must have either the jbossts-properties.xml configured appropriately or if you intend to programmatically configure the transaction/recovery manager then that must be done before you call any other JBTM API.

I am now using the default narayana-jta jbossts-properties.xml file. I configured the ObjectStoreEnvironmentBean.objectStoreDir to use ${java.io.tmpdir} and added a similar entry for com.arjuna.ats.arjuna.objectstore.objectStoreDir. Any other suggestions for my scenario?

[tomjenkinson]

Hi,

As you suggest, if you have a single XA resource and a single synchronization then that scenario won't create a recoverable transaction in the log. None-the-less you do have to configure an object store as certain other data is stored in it (EIS name is persisted here). It might be worth asking about the recovery requirement in the infinispan forum as I wouldn't want to second guess its requirements in terms of recovery, perhaps some flows still enlist it as an XAResource.

Did you manage to get your application up and running in a way you think it is working?

Tom

[talawahdotnet]

Tom Jenkinson wrote:

Hi,

As you suggest, if you have a single XA resource and a single synchronization then that scenario won't create a recoverable transaction in the log. None-the-less you do have to configure an object store as certain other data is stored in it (EIS name is persisted here). It might be worth asking about the recovery requirement in the infinispan forum as I wouldn't want to second guess its requirements in terms of recovery, perhaps some flows still enlist it as an XAResource.

According to the docs since Infinispan 5/Hibernate 4, infinispan registers itself as a synchronization by default when used as the 2nd level cache.  It is possible to change it, but I have no plans to.  I have to run my app in both modes with trace level logging turned on for the JBossTM classes and confirmed that enlistment does not happen by default.

So assuming no recovery is needed, are there any processes/services that can be disabled through configuration?

Did you manage to get your application up and running in a way you think it is working?

Good question, emphasis on *think* lol.  I think I have the basics working, but need to write some tests.  I also just posted another question in the forum re connection pooling with TransactionalDriver.

Message was edited by: talawahdotnet

[tomjenkinson]

Hi Talawah,

If I am following you correctly and you only enlist a single XAResource in any transaction (plus Synchronizations) and assuming you haven't disabled one phase commit then running the recovery manager isn't necessary.

Tom