Moving out of the UnifiedClassloaders

[dimitris]

Can we provide a more detailed description and an associated JIRA case what it would require to move out of the Unified Classloaders for AS5 Beta4?

[starksm64]

The discussion related to enabling the VFSClassLoader is here:
http://www.jboss.com/index.html?module=bb&op=viewtopic&t=113494

I'll create a jira issue for setting this up by default and do some testing before making the switch.

[adrian.brock]

Scott, I'm working on making the switch now.

The work required is:
1) Update to the lastest snapshot of the classloader project to fix a bug in getPackage()
2) Change bootstrap-beans.xml
3) Add osgi.jar to the bootstrap-beans.xml classloader
4) Check it doesn't seriously break the testsuite/tck (it will break the old UCL tests in the testsuite - that is expected)
5) Once any issues found are resolved do a proper release of (1) and update
the other configurations - e.g. profile service test and jboss embedded

Using the new classloader is a prerequiste for modularizing the bootstrap-beans.xml
which in turn solves a number of other issues like the @JMX annoation getting
deployed too late.

[starksm64]

Ok.

[adrian.brock]

The new classloader is in.

I've immediately come across one problem:
http://jira.jboss.com/jira/browse/JBAS-5120

I'm not sure if this is a messaging, aop or classloader problem?

[alesj]

The Seam deployments are now failing.
I'm getting different URLs that I used to get.
And the resources from .war files - that are extracted to tmp/deploy - are duplicated.

This is Scanner code from Seam.

 protected void scan()
 {
 Set<String> paths = new HashSet<String>();
 if (resourceName==null)
 {
 if (log.isTraceEnabled())
 log.trace("Scanning URLs from classloader: " + classLoader);

 for ( URL url: getURLsFromClassLoader() )
 {
 String urlPath = url.getFile();
 if ( urlPath.endsWith("/") )
 {
 urlPath = urlPath.substring( 0, urlPath.length()-1 );
 }
 paths.add( urlPath );
 }
 }
 else
 {
 if (log.isTraceEnabled())
 log.trace("Scanning URLs from resource: " + resourceName);

 try
 {
 Enumeration<URL> urlEnum = classLoader.getResources(resourceName);
 while ( urlEnum.hasMoreElements() )
 {
 String urlPath = urlEnum.nextElement().getFile();
 urlPath = URLDecoder.decode(urlPath, "UTF-8");
 if ( urlPath.startsWith("file:") )
 {
 if (useVFS())
 urlPath = "vfs" + urlPath;
 else
 urlPath = urlPath.substring(5);
 }
 if ( urlPath.indexOf('!')>0 )
 {
 urlPath = urlPath.substring(0, urlPath.indexOf('!'));
 }
 else
 {
 File dirOrArchive = new File(urlPath);
 if ( resourceName!=null && resourceName.lastIndexOf('/')>0 )
 {
 //for META-INF/components.xml
 dirOrArchive = dirOrArchive.getParentFile();
 }
 urlPath = dirOrArchive.getParent();
 }
 paths.add(urlPath);
 }
 }
 catch (IOException ioe)
 {
 log.warn("could not read: " + resourceName, ioe);
 return;
 }
 }

 for ( String urlPath: paths )
 {
 try
 {
 log.info("scanning: " + urlPath);
 File file = new File(urlPath);
 if ( file.isDirectory() )
 {
 handleDirectory(file, null);
 }
 else if (useVFS())
 {
 URL url = new URL(urlPath);
 handleArchiveByURL(url);
 }
 else
 {
 handleArchiveByFile(file);
 }
 }
 catch (IOException ioe)
 {
 log.warn("could not read entries", ioe);
 }
 }
 }

I added the useVSF part, which knew how to handle our vfs URLs.
Now I'm getting direct path:
e.g. 23:16:17,890 INFO [Scanner] scanning: C:\projects\jboss5\trunk\build\output\jboss-5.0.0.Beta4\server\default\deploy\jboss-seam-booking.ear\jboss-seam.jar

The duplicated part:
23:16:17,609 INFO [Scanner] scanning: vfsfile:/C:/projects/jboss5/trunk/build/output/jboss-5.0.0.Beta4/server/default/tmp/deploy/jboss-seam-booking2549-exp.war/WEB-INF/lib/jboss-seam-ui.jar
23:16:17,640 INFO [Scanner] scanning: C:\projects\jboss5\trunk\build\output\jboss-5.0.0.Beta4\server\default\deploy\jboss-seam-booking.ear\jboss-seam-booking.war\WEB-INF\lib\jboss-seam-ui.jar

[dimitris]

Some initial results here: (+256 failures)

http://www.jboss.com/index.html?module=bb&op=posting&mode=reply&p=4118715

- Many messaging test failures (as expected, due to the deployment issue)
- Classloader & hibernate ones
- Some jca/jmx/security
- ...

[adrian.brock]

The classloader (old UCL tests) and hibernate (again assumes UCL
to do getURLs() from the classloader) are expected.

The jca ones look like they are testing the jms rar - so same issue as messaging.

I think I know what the bug is with messaging. :-)

The "export all" handling isn't picking up the default package, so the classloader
for jboss-messaging.jar isn't included in the index of classloaders to check
for Thread.currentThread().getContextClassLoader().getResource("VERSION");

The security failure (formauth.ear) looks like the same issue
since it also has resources in the default package.

Although this looks wrong to me. Resources in the root of the ear
shouldn't be seen by the classloader anyway??????

$ jar -tf ~/jboss-head/testsuite/output/lib/form-auth.ear
META-INF/
META-INF/application.xml
META-INF/jboss-app.xml
META-INF/security-config.xml
form-auth-users.properties
form-auth-roles.properties
form-auth.war
jboss-service.xml

[adrian.brock]

"alesj" wrote:

This is Scanner code from Seam.

 protected void scan()
 {

...
 for ( URL url: getURLsFromClassLoader() )

What is getURLsFromClassLoader()?
The new Classloader is not a URLClassLoader so there is no getURLs().

Looks a wrong/bad assumption to me. :-)

This looks like the same issue as was raised for Hibernate
in the context of the embedded server.

The way to scan/visit the deployment structure needs to be pluggable.
You can't just assume it is a jar or file structure if we want to handle abitrary
virtual files.

[alesj]

What is getURLsFromClassLoader()?
The new Classloader is not a URLClassLoader so there is no getURLs().

In our case the resourceName != null, so that part was obviously not used. :-)
But I still don't see where the non vfs URLs came from, or the duplicates.

Looks a wrong/bad assumption to me. :-)

A deprecated assumption. ;-)

This looks like the same issue as was raised for Hibernate
in the context of the embedded server.

Only in embedded or the whole AS5?
Is this about the Hibernate core or AS's Hibernate-int?
I already did a simple VFS introduction into Hibernate class, but that also needs fixing, since it also assumes URLCL.

The way to scan/visit the deployment structure needs to be pluggable.
You can't just assume it is a jar or file structure if we want to handle abitrary
virtual files.

I was able to do minor 'hacks' for beta3 to make it work.
But I agree, and so does Pete, this needs rewriting into pluggable impl.

[starksm64]

Dimitris said we are pulling in the osgi jars just to use its version class. Can we move to our own compatible interface to avoid this?

[adrian.brock]

Dimitris said we are pulling in the osgi jars just to use its version class. Can we move to our own compatible interface to avoid this?

We do have an interface,
http://viewvc.jboss.org/cgi-bin/viewvc.cgi/jbossas/projects/microcontainer/trunk/deployers-structure-spi/src/main/org/jboss/deployers/structure/spi/classloading/Version.java?revision=64469&view=markup
but you still need an implementation.
Currently, we use the OSGi version class as our implementation:
http://viewvc.jboss.org/cgi-bin/viewvc.cgi/jbossas/projects/microcontainer/trunk/deployers-structure-spi/src/main/org/jboss/deployers/structure/spi/classloading/helpers/VersionImpl.java?revision=64482&view=markup

There was a previous discuss on this problem.
The issue is basically if we don't use the OSGi version then we have
to come up with some complicated code that handles comparisons between
the different implementations when doing compareTo().
http://viewvc.jboss.org/cgi-bin/viewvc.cgi/jbossas/projects/microcontainer/trunk/deployers-structure-spi/src/main/org/jboss/deployers/structure/spi/classloading/VersionRange.java?revision=64469&view=markup

Of course, its probably not a real problem in practice since it is unlikely that
foo:1.0.0 uses a different version implementation to foo:1.0.1 or foo:2.0.0 :-)

[adrian.brock]

I think I know what the bug is with messaging. :-)

The "export all" handling isn't picking up the default package, so the classloader
for jboss-messaging.jar isn't included in the index of classloaders to check
for Thread.currentThread().getContextClassLoader().getResource("VERSION");

I fixed the "default package" problem, but it is still failing to deploy jms.
Strangely, it doesn't always fail? But it does more often than not.

Having looked at the code, I don't think this is a classloader issue
unless it is a problem with the new aop integration that Kabir did
i.e. it is not applying the correct advices.

[starksm64]

We do have an interface,
...
There was a previous discuss on this problem.
The issue is basically if we don't use the OSGi version then we have
to come up with some complicated code that handles comparisons between
the different implementations when doing compareTo().
...
Of course, its probably not a real problem in practice since it is unlikely that
foo:1.0.0 uses a different version implementation to foo:1.0.1 or foo:2.0.0 :-)

VersionRange could accept a Comparator to allow for externalization of the Version compareTo.

[starksm64]

I fixed the "default package" problem...

This is the cause of some of the security test failures as they are not finding "users.properties" resources. I would expect a fair number of the 250 extra failures are related to this.