Embbedded - broken - some fixes - still not working

[adrian.brock]

I tried to get the ide test working for embedded inside Eclipse
to understand how Bill's assembly vfs stuff works.
(Aside - the public interface for these classes shouldn't be in org.jboss.vfs.plugins :-)

Problems found so far:
1) Resources from the ejb3 testsuite were leaking into the embeded project.
In particular jndi.properties
2) The ejb3-interceptors-aop.xml is out-of-date
3) The default ejb3 persistence properties was missing
4) jboss-local-jdbc.rar was missing
5) The jca stylesheet was in the wrong place
6) There were many missing jars from the classpath

I've managed to fix all these (at least as they relate to the VFS test in the IDE)
although this shows just how stupid this solution is.

Why do we need to maintain multiple copies of configurations files and implementation
details all over the place. It's an impossible task to keep them in synchronization.

Finally, it's still broken, because Hibernate can't find the entities.

WARN 21-11 15:44:14,717 (?:?:?) -Unable to find file (ignored): vfs://11956562541581/
java.lang.RuntimeException: Cannot open stream
 at org.jboss.virtual.plugins.context.vfs.AssembledDirectoryHandler.openStream(AssembledDirectoryHandler.java:117)
 at org.jboss.virtual.VirtualFile.openStream(VirtualFile.java:216)
 at org.jboss.virtual.plugins.vfs.VirtualFileURLConnection.getInputStream(VirtualFileURLConnection.java:117)
 at java.net.URL.openStream(URL.java:1007)
 at org.hibernate.ejb.packaging.InputStreamZippedJarVisitor.doProcessElements(InputStreamZippedJarVisitor.java:33)
 at org.hibernate.ejb.packaging.JarVisitor.getMatchingEntries(JarVisitor.java:240)
 at org.hibernate.ejb.Ejb3Configuration.addScannedEntries(Ejb3Configuration.java:275)

It doesn't undertand the virtual file system and assumes its a zipped jar
which it fails to open - obviously.
This behaviour is not pluggable in hibernate. And the choice of how to scan
the jars/filesystem is even in a private method.

A policy decision in a private non-overridable method? :-)

Finally the test itself uses a jboss image in the main src checkout.
Which SVN tells me should be checked in! ;-)

[ejort@warjort embedded]$ svn status | grep \?
? src/main/resources/javase/tmp
? src/main/resources/javase/data

[sebersole]

1) What exactly does a vfs:// url represent? Any type of file resource?

2) Are we (Hibernate) being handed an actual java.net.URL here? If so, is this a vfs-specific URL implementation? In other words could we rely on using the URLConnection/ContentHandler mechanism and have that be handled correctly by the underlying vfs?

[maxandersen]

1) its a jboss virtual file system. Afaik only accessible through the jboss vfs api i think (would be cool though if it could be mapped behind a file system in java's api ;)

2) that is what i thought so too and if you look in embedded source code there are some attempts on doing it via HandlerHack's etc. But from what I understood from the code hibernate assumes its either a file or a jar connection and the VFS is neither of these two.....but if jboss can traverse it we should also be able to map it to something that the current traversal code can handle....

[adrian.brock]

1) What exactly does a vfs:// url represent? Any type of file resource?

Its an abstract directory structure. The underlying resources are scattered
around the file system. i.e. no need to package a jar (not even as an
expanded filesystem that looks like a jar).
See the VFSTestCase in embedded.

2) Are we (Hibernate) being handed an actual java.net.URL here? If so, is this a vfs-specific URL implementation? In other words could we rely on using the URLConnection/ContentHandler mechanism and have that be handled correctly by the underlying vfs?

Yes, but you don't use it. Your logic is something like:
if (fileURL()) then use java.io.File
else assume a jar file.

This is neither of those. You need to define your own simple VFS api
that we can plug into. In fact, you should really be defining something
more high level, since all you really want is a list of relevant annotated class URLs
(the real ones) which we can give you.

[sebersole]

Yes, but you don't use it. Your logic is something like:
if (fileURL()) then use java.io.File
else assume a jar file.

Yes, I understand that much; I am a dolt, not a *complete* dolt ;)

The question is whether if we did use that approach the vfs URL would actually do TheRightThing...

This is neither of those. You need to define your own simple VFS api
that we can plug into. In fact, you should really be defining something
more high level, since all you really want is a list of relevant annotated class URLs
(the real ones) which we can give you.

You can, sure. But what about other app severs/environments... If the URLConnection/ContentHandler approach is more "correct" and the vfs stuff will really handle it correctly, then whats the issue with using that? Seems like a more universally applicable solution. But of course, I'm a dolt (^^) so please enlighten me ;)

[adrian.brock]

You can, sure. But what about other app severs/environments... If the URLConnection/ContentHandler approach is more "correct" and the vfs stuff will really handle it correctly, then whats the issue with using that? Seems like a more universally applicable solution. But of course, I'm a dolt (^^) so please enlighten me ;)

Where in the URL api does it have a listChildren(), etc.? :-)

You have to use a specific api, e.g. java.io.File or java.util.jar.JarFile
before you can look at structure.

This adhoc processing (usually done wrong in multiple places :-) is one of
the reasons we introduced the VFS, so we can do it correctly in one place
and extend it.

All I'm asking is for a way to plugin where currently hibernate takes a
non-overriable decision. If other appservers want to plugin as well
then ok. Even something as simple as:

package org.hiberante.vfs

public interface URLLister
{
 URL[] getChildren(URL parent);
}

would mean we could do:
-Dorg.hibnerate.vfs.URLLister.vfs=org.jboss.hibernate.vfs.VFSURLLister
i.e. a system/hibernate property org.hiberate.vfs.URLLister.{protocol}

I'm sure you can come up with something better :-)

I'd prefer something at a higher level. Mainly because
we are already scanning classes for annotations for other reasons so we
can optimize your search for those that are marked @Entity or whatever
filtering you need.

[sebersole]

I guess it comes down to the definition of "correctness" I asked about before. I should have been more specific. Concretely I was thinking of URLConnection.getContentType(), and wondering what would get returned here for the vfs URL impl. But based on your answers I have to assume it is something non-useful like "vfs' or some-such.

I have not seen Emmanuel's code for this discovery, so I cannot give the best answer. But I would assume something like this registerable with the Hibernate Configuration. Probably even the discovery itself could get encapsulated into a registerable contract which you could then provide. A contact like:

public interface EntityScanner {
 /**
 * Note sure of the most appropriate return type here, but
 * it seems best to stay away from URL on vfs ;)
 */
 File[] getAnnotatedEntities(URL root);
}

[adrian.brock]

I guess it comes down to the definition of "correctness" I asked about before. I should have been more specific. Concretely I was thinking of URLConnection.getContentType(), and wondering what would get returned here for the vfs URL impl. But based on your answers I have to assume it is something non-useful like "vfs' or some-such.

Well it should return the underlying real answer. But that
depends upon how good the protocol handler is.
In this case, Bill wrote it, so its a NYI :-)

I have not seen Emmanuel's code for this discovery, so I cannot give the best answer. But I would assume something like this registerable with the Hibernate Configuration. Probably even the discovery itself could get encapsulated into a registerable contract which you could then provide. A contact like:

public interface EntityScanner {
 /**
 * Note sure of the most appropriate return type here, but
 * it seems best to stay away from URL on vfs ;)
 */
 File[] getAnnotatedEntities(URL root);
}

URL[] is better than file. e.g. it could be a jar entry.

But like I said before, this doesn't really optimize it. e.g. at least in JBoss
we'll already have a cache of the annotations so there would be no
need for you to re-read them. e.g. you currently take the stream
and parse the byte code (we've already done that looking for other annotations)

Anyway, let's keep the two issues seperate. Just being able
to tell you what the real class and META-INF/persistence.xml URLs are would be a start. :-)

[adrian.brock]

URL[] is better than file. e.g. it could be a jar entry.

We'll actually

Iterator<URL>

will scale better when there are lots of classes.

[sebersole]

But like I said before, this doesn't really optimize it. e.g. at least in JBoss
we'll already have a cache of the annotations so there would be no
need for you to re-read them. e.g. you currently take the stream
and parse the byte code (we've already done that looking for other annotations)

The idea is that whatever implementation of the EntityScanner contract was registered would be the one "finding" annotated classes. In your case, as you said, that work has already been done, so your impl could be quite optimized ;)

Iterator<URL>

Personally, I like this:

interface EntityProcessor {
 void processAnnotatedClass(Class clazz);
 ...
}
interface EntityScanner {
 void findAnnotatedEntities(EntityProcessor processor);
}

But like I said, I'll leave this up to Emmanuel...

[epbernard]

Hey guys,
There are some reasons why I assume either a JAR structure or a directory from these URLs.
The contract between the app server and the persistence provider is defined by javax.persistence.spi.PersistenceUnitInfo

When the container creates a PU, it calls

/**
 * Called by the container when an EntityManagerFactory
 * is to be created.
 *
 * @param info Metadata for use by the persistence provider
 * @param map A Map of integration-level properties for use
 * by the persistence provider (may be null if no properties
 * are specified).
 * @return EntityManagerFactory for the persistence unit
 * specified by the metadata
 */
 public EntityManagerFactory createContainerEntityManagerFactory(PersistenceUnitInfo info, Map map);

Specifically the interesting bits of the contract are

/**
 * Interface implemented by the container and used by the
 * persistence provider when creating an EntityManagerFactory.
 */
public interface PersistenceUnitInfo {
 [...]

 /**
 * @return The list of JAR file URLs that the persistence
 * provider must examine for managed classes of the persistence
 * unit. Each jar file URL corresponds to a named <jar-file>
 * element in the persistence.xml file.
 */
 public List<URL> getJarFileUrls();

 /**
 * @return The URL for the jar file or directory that is the
 * root of the persistence unit. (If the persistence unit is
 * rooted in the WEB-INF/classes directory, this will be the
 * URL of that directory.)
 */
 public URL getPersistenceUnitRootUrl();

 [...]
}

The reading of the JavaDoc (and the intend of the spec) is that the persistence provider expect either a directory or a JAR file.

I scan the root URL for 3 reasons:
- get the annotated classes
- get hbm.xml files
- get ORM.xml files

2 is proprietary to Hibernate of course.

Even if we define a proprietary abstraction implemented by JBoss AS and passed as an argument to createContainerEntityManagerFactory (in the Map), it does not solves the whole problem.
JBoss AS has to conform to the PersistenceUnitInfo contract so that other providers can bootstrap :)

[maxandersen]

emmanuel - Isn't this way of starting (with PersistenceUnitInfo) different from what happens when hibernate scans for entities "on its own" in j2se scenario ?

[adrian.brock]

"epbernard" wrote:

Even if we define a proprietary abstraction implemented by JBoss AS and passed as an argument to createContainerEntityManagerFactory (in the Map), it does not solves the whole problem.
JBoss AS has to conform to the PersistenceUnitInfo contract so that other providers can bootstrap :)

I'm not saying don't support the spec. I'm saying we can optimize it.

There are really two seperate issues:

1) The embedded stuff where we don't require a packaging step to run tests,
we create a VFS context that collects the disparate resources from across
the IDE project. A sort of in memory symbolic link.
See the VFSTestCase in the embedded project.

2) Optimizing the scanning/parsing of the class files for annotations.
Scanning we can trim the list the list to speed things up
since we already did the scan and know which are @Entity.
And we've already parse those classes that do match.

The issue that started this thread was (1).

(2) is a nice to have. At the moment we have EJB3, Hibernate, Tomcat, AOP
and in the future the MC all needing to scan packages for annotated classes.
Doing it N times for N different modules is just a waste.

[maxandersen]

...but why is this concern of where to get resources from leaked out via the urls passed down to hibernate via the urls it gets ? Can't we avoid that somehow?

[adrian.brock]

...but why is this concern of where to get resources from leaked out via the urls passed down to hibernate via the urls it gets ? Can't we avoid that somehow?

I don't understand your question? You need some identifier.
JavaEE generally uses URLs (but they are not navigable
there is no generic getChildURLs()).

Currently hibernate gets the top level url and barfs because it only knows
how to navigate file:, jar:
There is no way to make it understand anything else.

We're discussing how to go beyond these for reasons of optimization
and ease of use/development.

If you want to avoid the problem then we say you always have to package
as a jar or exploded archive and we will always scan everything (multiple times :-)