Creating datasource with the DeploymentService

[ccrouch]

I would like to revisit an issue that was briefly mentioned in this thread (http://www.jboss.com/index.html?module=bb&op=viewtopic&t=58900). Namely that I would like to discuss adding support for the classpath element to *-ds.xml's.

The recommended datasource deployment scenario from thread above is to bundle everything into a .sar file with the following structure:

./deploy/mydatasource.sar/
 META-INF/jboss-service.xml = <?xml version="1.0"?><server></server>
 my-ds.xml
 mydriver.jar

Suggested alternative:

./deploy/my-ds.xml (including classpath element pointing to mydriver.jar)
./data/datasource_drivers/mydriver.jar

I am not suggesting this as a general alternative to the .sar approach, but for one particular scenario, described below, I believe it has some advantages.

1) The Admin Console is going to use the DeploymentService (http://www.jboss.org/wiki/Wiki.jsp?page=DeploymentService) to create, update and delete datasources.

2) There is an advantage in having the datasource driver .jar live outside of a .sar file due to the way the DeploymentService works. Namely that clients of the DeploymentService would not have to keep a reference to the datasource driver .jar, in order to pass it to the DeploymentService when doing an update. Keeping the reference is necessary when everything is bundled into a .sar since there is no way to obtain from a datasource a reference to the driver it is using.

3) Assuming you agree with point 2) it is also a good idea for the datasource driver .jar to live outside of the scanned deployment directories, so that multiple versions of the same driver can be supported. The classpath element is therefore needed to associate the configuration definition of the datasource and the driver.jar file.

4) It is easier to create a DeploymentService template for a -ds.xml file than to create a -service.xml that does the same thing.


So, assuming the scenario above is taken as being sufficient reason to add classpath element support for ds.xml's, how hard would it actually be to implement?

Thanks

[adrian.brock]

I don't like the

<classpath/>

configuration in -service.xml

1) The semantics are not well defined.
2) Classloading dependencies are not defined at all in the current jmx kernel
3) There are some workarounds in the deployment code just to stop users
trying to deploy the same jar twice through classpath elements in different -service.xml
4) The workaround in (3) does not take take into reference counting such that
correct undeployment would work
5) It has no notion of LoaderRepository, i.e. what is the classloading domain
and policy for these classloaders.

There is a ClassPathExtension MBean that can be used to add url classloaders
to a loader repository and that MBean takes part in the service lifecycle.
However, see (2) above.

I need to be convinced that these scheme would work anyway, i.e.
that you can actually redeploy a jdbc driver.
e.g. "uncomment" the jdbc driver redeploy test in the jca part of the testsuite
to see the problem.

[dimitris]

Just a point here: using subdeployments (.jar within .sar) you have the advantage of the deployer making temporary copies of the .jar, .sar, so we avoid the problem of the URLClassLoader locking those files....

[ccrouch]

I don't like the

<classpath/>

configuration in -service.xml

Does not liking it mean:
a) it should not be used because it is unreliable and things break, or
b) we should really change the whole implementation in this area because it is not very nice, or
c) all of the above, or
d) none of the above

I have done some simple tests with the alternative deployment scenario:

./deploy/my-datasource-service.xml (including classpath element pointing to mydriver.jar)
./data/datasource_drivers/mydriver.jar

and other than the locking of mydriver.jar which Dimitris points out, it seems to "work". Where "work" means:
-you can hot deploy a datasource.
-you can update the -service.xml to point to a different driver and after JBoss restarts the datasource will use the new driver.


Thanks

[ccrouch]

Have discussed the various options again with Ivelin, we're going with the .sar bundling the driver.jar and -ds.xml.

Thanks for all the feedback.

[adrian.brock]

But you cannot hotdeploy the driver.

If you don't believe me, try running the
org.jboss.test.jca.test.JDBCStatementTestsConnectonUnitTestCase
from the testsuite (twice).

It works the first time, but fails the second time after the driver is redeployed.
As I said before the problem is in java.sql.Driver and its use of Class.forName().

[ccrouch]

Adrian, thank you for bearing with me. I believe you that the test you mention will fail, when you run it twice. I was being overly optimistic based on limiting testing using mysql drivers:

-I could deploy a datasource with driverA, delete the deployment, redeploy the exact same deployment and the datasource worked as expected.
-I could deploy a datasource with driverA, delete the deployment, redeploy the exact same deployment except use driverB and the datasource worked as expected: using driverA until the server was restarted when it would work using driverB.

The above held true whether I was using the "wrapped approach" of putting the -ds.xml and driver in .sar or the "split approach" having a -service.xml with a classpath entry and the driver jar stored separately. The unit test you mention demonstrates this does not hold true in the general case.

I think the crux of my misunderstanding was based around the meaning of driver hot redeploy "not working". When I saw that statement, I thought it agreed with my results i.e. "The datasource will continue to function but that any driver change will not get picked up until the server is restarted", when the case is actually "in general as soon as you redeploy the the driver, the datasource will be broken (e.g. throw exceptions) until the server is restarted". The former is in my mind is a whole lot more acceptable than the latter.

I then ran another test:
deploy a datasource with a bad password, delete the deployment, redeploy the exact same deployment except with the correct password.

1) When using the "wrapped approach" the datasource behaved as expected. At the end of the test, after the redeploy, the datasource threw a NPE. After restarting the server the datasource worked fine.
2) When using the "split approach" and putting the driver jar in .default/data/datasources/ the same results as in 1) were experienced.
3) When using the "split approach" and putting the driver jar in .default/deploy/datasources/ no exceptions were thrown and at the end of the test a connection to the database could be obtained.

Is result 3) useful, or should I just stop banging my head against this particular wall?

[adrian.brock]

I'm not sure I understand.

The fundamental problem is that once a class looks up a jdbc driver
java.sql.Driver remembers the classloader where the driver was found.

If the driver gets redeployed java.sql.Driver will effectively say the driver
is not in the place I expected (it cached the info) and ignore it when checking
what drivers are visible.

The code is actually intended to stop java applets looking at each other's drivers.

[ccrouch]

I'm sorry if I'm confusing the situation, I was just trying to describe the tests that I had done and why I persevered with this topic.

Fundamentally what I am trying to do is to find a way to manage datasources and drivers which has consistent and reasonable behavior.
If that behavior includes needing to restart JBoss to see a change to a new driver then that is acceptable. What is less good is if the behavior includes exceptions being thrown in the server log and datasources being unavailable. For instance the testcase you described, JDBCStatementTestsConnectionUnitTestCase, indicates that hot redeploying drivers in JBoss will likely lead to class cast exceptions being thrown. Also in my tests, 1) above, I found that when redeploying a .sar file that contained a driver and datasource definition, that changing just the password attribute, caused references to the datasource to throw a NPE. The only scenario which appeared to show any promise was 3) from my previous post.

Going forward I intend to follow a much more conservative approach to managing datasource and drivers. Namely putting drivers in ./lib, and -ds.xmls in./deploy. This will not give us the functionality we were hoping for from the Admin Console but I'm confident it will behave cleanly, e.g. not throw exceptions.

I'm about to post a description of this new approach on the Admin console forum (http://www.jboss.com/index.html?module=bb&op=viewforum&f=209). If you are interested, your input would be valuable.

Thanks