Replacing TransactionalDriver with IronJacamar

[tomjenkinson]

We have an outstanding Jira (https://issues.jboss.org/browse/JBTM-809) to deprecate the TransactionalDriver code in preference to utilising the IronJacamar JCA implementation in embedded mode. Doing so should provide many benefits for users of datasources as the IronJacamar container is under active development where new enhancements/bug fixes are regularly added by their community.

The initial part of this work was to develop an example showing how we see IronJacamar being used in practice (https://issues.jboss.org/browse/JBTM-1479). We have now merged this into the quickstart repositories over here: https://github.com/jbosstm/quickstart/tree/master/jca-and-tomcat and are therefore at the point where we would like to solicit feedback from users of the transactional driver whether as it stands, or with any modifications that may be suggested, making this connection manager change will prove to be an issue for anyone.

Thanks for any suggestions/comments,

Tom

[mauromol]

Hi Tom, I had a look to the quickstart project. Thank you for providing it and ask me for feedback.

Here are my first impressions. First of all, consider I never used IronJacamar before, so I have no experience with it. As far as I understand it, you are doing something like this:

- "deploying" a RAR (resource archive?) containing some code that handles the instantiation of the JDBC driver XA Data Source through IronJacamar API

- "deploying" an XML that specifies the properties to pass to IronJacamar to instantiate the XA Data Source

I suspect the RAR file also takes care of registering the XA Data Source with JBoss Transactions, otherwise I can't see where the registration of the transaction manager is performed (this was done by the transactional driver itself before: getting a transactional driver connection caused this to get an XA connection, register it with JBTM and return a physical JDBC connection to the client code).

Issue that I see:

- I must create a JNDI repository just to get my data source... In a plain JSE environment, this can be problematic. Unit tests immediately come into my mind. In our case, we should use Spring to mimic a real JNDI repository just to get our data source. With the transactional driver, the "dynamic class" mechanism was avoiding this. A way to retrieve the data source prepared by IronJacamar without querying a JNDI repository would be appreciated

- the need to deploy a specific XML file forces you to put the datasource configuration into that file. If you use Spring to handle your configuration this can be a problem, unless you build that XML "on the fly"; another way to pass in configuration properties programmatically could help here

Anyway, I would have to do some tests to understand the whole thing better. For instance, we now use our own XA data source implementation which actually behaves as a proxy to the actual JDBC driver XA data source and performs XA connection pooling. This XA data source proxy takes care of intantiating and configuring the actual XA data source. In this scenario, we may simply have a "standard" proxy-xa-ds.xml which just configures our own proxy and then delegate the proxy to configure the actual data source as it does right now (i.e.: using Spring injected configuration properties). This should work, but I would have to test.

So my only "concerns" right now are about the need to have a JNDI repository and the .rar file thing, which I don't understand if it's just a file that I should take as-is (from the IronJacamar distribution? From elsewhere?) and simply deploy together with my application...

[tomjenkinson]

Hi Mauro,

To answer the first part of the message:

jdbc-xa.rar is a JCA resource adapter which is responsible for integrating XA JDBC data sources. It automatically registers data sources with Narayana once they are deployed. jdbc-xa.rar is available with IronJacamar distribution and can be used directly from the latest version of IronJacamar.

JNDI question:

To the best of our knowledge, IronJacamar does use JNDI under the bonnet, so it is required at some level. In this quickstart Gytis used an isolated standalone JNDI server org.jboss.naming:jnpserver. It just requires the two lines of configuration you can see in the quickstart: https://github.com/jbosstm/quickstart/blob/master/jca-and-tomcat/src/main/java/org/jboss/narayana/quickstart/jca/listener/ServletContextListenerImpl.java#L72 plus the required Jars.

XML question:

There are three ways to deploy to the IronJacamar embedded container: using file's URL (just like in the quickstart) or implementations of either org.jboss.shrinkwrap.descriptor.api.Descriptor or org.jboss.shrinkwrap.api.spec.ResourceAdapterArchive interfaces. One of the last two options should help you out to deploy data source without having physical configuration file.

Hopefully that helps, if you have further questions (for example on JNDI), please can you clarify what problems you might have with the following snippet of code:

	System.setProperty("java.naming.factory.initial", "org.jnp.interfaces.NamingContextFactory");
	System.setProperty("java.naming.factory.url.pkgs", "org.jboss.naming:org.jnp.interfaces");

You could also put those in a JNDI properties file instead.

Thanks for getting back to us,

Tom

[mauromol]

Thank you for the clarification on the .rar and .xml file deploying.

Regarding the JNDI thing: I understand it may not a blocking issue, however the need to have a JNDI repository just to get an instance of a Java object (the data source) in a plain Java SE environment doesn't sound "polite" to me and makes the whole "JBoss TS embedded" setup seem like a sort of "workaround", rather than an explicitely supported option (like it were with the Dynamic Class mechanism of the transactional driver before). At this point, I understand it's not a JBoss TS limit, but rather an IronJacamar one, however from my point of view providing an alternative way of requesting the datasouce (using a singleton or a factory) would be an improvement.

Maybe an alternative approach would be to get rid of IronJacamar completely and take care of the XA data source registration by our own. But I don't know what this would mean.

Thanks again,

Mauro

P.S.: are there other noteworthy things to know to upgrade from JBoss TS 4.x to 5.x?

[tomjenkinson]

Hi Mauro,

Thanks for the response and I see now where you are going on the JNDI front.

Moving to IronJacamar actually provides us with many benefits as it has excellent connection management facilities and is very well tested as it is the datasource management facility within WildFly so very heavily used. It also means that we can connect easily to other types of datasources such as JMS etc. Whether IJ provides alternate ways to resolve the datasource, I have asked Jesper to hop onto this thread to respond to that and your general queries around the JNDI requirement.

In terms of your "PS", there are many new features in Narayana. We have support for JTA 1.2 and a new compensations framework. We have merged in an XA wrapper for file systems and we are working on a Software Transactional Memory feature. Most of the new features we discuss over on our blog too: http://jbossts.blogspot.co.uk/

Best regards,

Tom

[marklittle]

It would be a good idea to create a list of pros and cons of this suggested move to IronJacamar for the community to understand and comment.

[tomjenkinson]

Sure, I will take a look at putting something together

[jesper.pedersen]

IronJacamar uses a JNDI server to binds its ConnectionFactory instances in by default - sort of implicit requirement in the JCA spec.

However, the naming store and the binding policies are pluggable in IronJacamar. So you can just provide your own

https://github.com/ironjacamar/ironjacamar/blob/master/core/src/main/java/org/jboss/jca/core/spi/naming/JndiStrategy.java

implementation, and plug that into deployment chain (see jca.xml typically). There are other objects that needs a naming server (search for "NamingServer"), those would need to be resolved too - or do a bean called "NamingServer" which keeps track of the objects, and the Context lookups.

Deploying the JDBC resource adapter is of course needed in order to support datasources - IronJacamar/Standalone + IronJacamar/Embedded has a datasource deployment chain too (see ds.xml). WildFly uses a .jar approach where the metadata is deployed in code such that the physical .rar file isn't needed.

IronJacamar/Embedded supports ShrinkWrap and ShrinkWrap/Descriptors deployments such that the deployment metadata isn't needed on-disk, see

http://www.ironjacamar.org/doc/userguide/1.1/en-US/html/ch11.html

for an overview.

The IronJacamar team provides two different setups of a JCA environment based on section 3.5 in the JCA specification, however, everything is pluggable as proved by WildFly

Hope that this provides some insight.

[mmusgrov]

Hi Mauro,

If we integrate IJ into the transaction manager some of the details you see in the example quickstart will disappear. In particular we will be responsible for starting IJ and the JNDI server and registering your datasources. Your responsibility is to supply the datasource definition after which you would obtain your datasource with a lookup (new InitialContext().lookup(jndiNameOfDS)). We could wrap that call into a singleton but I not sure if that would provide much value add.

Also, you mentioned an issue with needing to deploy the datasource definition in an XML file. If we were to provide a programatic mechanism then we would simply end up constructing the XML dynamically anyway and it would be preferable if you could do this and we provide an API that accepts either a file or an XML data structure.

Have you had any more thoughts about whether you will be able to adapt your application to use the new approach we are proposing?

[mauromol]

Hi Michael,

first of all thanks again for considering my use case and asking me for opinions :-)

Well, I will try to explain my point of view, but consider that it's limited to the way we use JBoss Transactions (which is OUTSIDE a JavaEE container, since we deploy our application to a plain Tomcat).

After all, JBoss Transactions by itself can be embedded into your application, it's just a bunch of configuration and lifecycle management work. The problem now is to enable your "legacy" application (which just knows about DataSources and JDBC connections) to participate in a JTA-XA transaction. Until now, we used the ability provided by JBoss Transaction called "TransactionalDriver", which means: the application uses a DataSource implementation that pulls connections from a TransactionalDriver instance rather than from a Driver instance provided by the actual JDBC driver in use. Then, the configuration problem is moved below the TransactionalDriver, that is we provide some DynamicClass implementations to the TransactionalDriver to enable it to pull connections from an XADataSource provided by the actual JDBC driver. The TransactionalDriver takes care of registering the XAConnections with the TransactionManager as needed and then pulls plain Connections from the XAConnections whenever the client application is requesting a Connection from the TransactionalDriver. So far so good (unless you have corrections to make to this picture).

Now, the problem is that you are going to not support the TransactionalDriver any longer, so how the whole picture changes?

In an "all-by-myself" approach, I would say: I just need a DataSource implementation that pulls Connections from XAConnections obtained from the XADataSource provided by the JDBC driver; those XAConnections, however, must be correctly registered with the TransactionManager before use. I would say that recent versions of Apache DBCP should do exactly this, although when I looked at it it was not mature enough and some aspects of its configuration were quite obscure to me. I would also say that I could write such DataSource implementation by myself, the only part I would need to learn is the "registration" one (i.e.: "connecting" XAConnections to the TransactionManager, so that the XAResources are then correctly orchestrated by JBoss Transaction).

Now, you're actually suggesting a different approach, that is to use IronJacamar to get the DataSource already "prepared". That sounds good, but my concerns are about the baggage we have to take with us in order to use it. From the information you gave me, I actually need to:

- take IronJacamar libraries and their dependencies (if any)

- deploy a resource archive with the application

- provide an XML configuration file or alternatively supply a couple of interface implementations needed to tell IronJacamar how to build up the DataSource from your actual JDBC driver XADataSource

- query a JNDI repository where IronJacamar places the prepared DataSource

So, going back to the original need, will this enable our "legacy" application to stay untouched (apart from the configuration details) and easily use JBoss Transactions? I think so.

Will this approach be JavaSE-friendly as much as the old one? Well, this is where I have some concerns.

After all, JCA and JNDI are technologies coming from the JEE platform and having to provide configuration through an XML file is less Spring-friendly (well, right now also JBoss Transactions needs jbossts-properties.xml, I discussed about this with Andrew Dinn in the past and I repeatedly went deeply in the JBossTS sources to find a way to inject configuration properties by myself without having to use that XML file, but unfortunately I always got to a dead end). Anyway, the possibility to specify a custom interface implementation to describe the XADataSource would be a good way to overcome the latter point, however, the former could be much annoying.

As I said, I don't like the idea to set up a JNDI context just to get one amongst other dozen objects that I'm currently setting up at configuration time and wiring with just plain Java code or XML Spring definitions/autowiring. Also, the need of a rar archive instead of plain code (either in source or binary form) would be a second exception on how our application is built. And these exceptions would be used to cover a very tiny aspect of the whole application configuration, which is already quite huge and complex.

So, if you're asking me if this is a severe blocker, I would say "no, it isn't", but if you're asking me if I'm going to replace the TransactionalDriver with IronJacamar to set up our embedded use of JBoss Transactions, I would say that I'd like to rather investigate the "all-by-myself" approach and then fall back to the IronJacamar solution only if I find really complex challenges with that approach.

I hope I've been able to explain better my point of view, without too many English errors :-D

Anyway, I would be really glad to hear from you if you find that what I've said is wrong or debatable.

Thanks again,

Mauro.

[jesper.pedersen]

You can't really compare Apache DHCP and IronJacamar. Their focus is very different.

IronJacamar is an implementation of the Java EE Connector Architecture specification, including the standalone profile, which defines a JCA environment outside an application server context. IronJacamar is the only implementation of this profile, which opens up a lot of possibilities for projects like Narayana.

Correct, if you only look at the datasource case many people will find Apache DHCP enough for their needs. However, in the enterprise world access to enterprise information systems, like WebSphereMQ, plays just an important role. Some of these systems even provides even more advanced scenarios that databases could only dream about. That leaves it to companies like Red Hat to "bridge" this gap using a resource adapter. IMHO it is just as important to focus these XA capable systems that aren't "exposed" through the JDBC API.

Using IronJacamar as the transactional driver would allow the Narayana project to focus on improving their implementation, since IronJacamar now handles configuration and lifecycle of datasources, and any possible deployed resource adapters. Yes, of course Narayana now needs to include IronJacamar and its dependencies, but IronJacamar is a modular implementation, and leave choice to the user - in this case Narayana - on how integration to services like JNDI is done.

If Narayana decides to use IronJacamar as the foundation for its transactional driver I'll of course work with the developers on providing an optimal setup that targets its current/future users. IronJacamar provides a set of functionality, but Narayana will decide what and how it is exposed to its users. A scenario could be just to expose a deploy(URL) method, which could deploy an IronJacamar datasource XML definition, and have the rest "hidden".

But lets cross that bridge once we get to it.

[marklittle]

I don't think there can be any doubt that we have to replace TransactionalDriver with something else: I wrote the initial code in 1996/1997 and it hasn't really evolved much in the past 5+ years except where JDBC standards have changed. IronJacamar seems like a good option here, but what we need to be sure is that whatever we do is a good replacement (similar requirements matched, as easy to use outside an application server etc.) I don't know if we are there yet, since I haven't looked at the proposed changes or spoken with the team, but I am confident that we can work together to make this a success, just as Jesper mentions.

[jesper.pedersen]

Although the primary focus for IronJacamar 1.x have been on the JCA/Standalone and JCA/EE profiles it should be possible to create a setup that fits into the desired environment. F.ex. adding support for JCA 1.6 resource adapters using bean validation maybe out of scope for the transactional driver - so that can be null'ed out in the configuration, and the dependencies removed. The maybe bugs, but those can be fixed

[tomjenkinson]

I am considering how much we can sit IronJacamar behind the TransactionalDriver APIs particularly around the configuration of the datasource, obtaining a reference to the connection plus making use of JNDI internal to TransactionalDriver. The immediate thing that springs to mind is although we can lazily launch IJ and hopefully have enough configuration from the properties in the existing API to deploy datasources during calls to java.sql.Driver::getConnection(), I am not so sure how we can shutdown IJ except as a Runtime::addShutdownHook()

I think we might be well placed to set up a quickstart using the old TransactionalDriver and compare this to how the new TransactionalDriver would work, probably outside of even Tomcat in a purely standalone application.

[jesper.pedersen]

http://www.ironjacamar.org/doc/api/embedded/1.1/org/jboss/jca/embedded/Embedded.html#shutdown%28%29

Embedded chapter gives a bit of insight - http://www.ironjacamar.org/doc/userguide/1.1/en-US/html/ch11.html