VFS and directory handling

[alesj]

Another VFS issue I stumbled upon.
I guess this was there for quite some time, we just never deployed things in this way to see it. :-)

The issue is that when I deploy an artifact that contains persistence.xml as exploded jar, this is what I get:

2008-05-23 18:08:35,578 DEBUG [org.hibernate.ejb.packaging.AbstractJarVisitor] (HDScanner) Searching mapped entities in jar/par: vfsfile:/C:/projects/jboss5/trunk/build/output/jboss-5.0.0.CR1/server/default/deploy/service-test.jar/
2008-05-23 18:08:35,578 WARN [org.hibernate.ejb.packaging.InputStreamZippedJarVisitor] (HDScanner) Unable to find file (ignored): vfsfile:/C:/projects/jboss5/trunk/build/output/jboss-5.0.0.CR1/server/default/deploy/service-test.jar/
java.io.FileNotFoundException: C:\projects\jboss5\trunk\build\output\jboss-5.0.0.CR1\server\default\deploy\service-test.jar (Access is denied)
 at java.io.FileInputStream.open(Native Method)
 at java.io.FileInputStream.<init>(FileInputStream.java:106)
 at org.jboss.net.protocol.file.FileURLConnection.getInputStream(FileURLConnection.java:106)
 at java.net.URL.openStream(URL.java:1007)
 at org.jboss.virtual.plugins.context.AbstractURLHandler.openStream(AbstractURLHandler.java:131)
 at org.jboss.virtual.VirtualFile.openStream(VirtualFile.java:216)
 at org.jboss.virtual.plugins.vfs.VirtualFileURLConnection.getInputStream(VirtualFileURLConnection.java:117)
 at java.net.URL.openStream(URL.java:1007)
 at org.hibernate.ejb.packaging.InputStreamZippedJarVisitor.doProcessElements(InputStreamZippedJarVisitor.java:38)
 at org.hibernate.ejb.packaging.AbstractJarVisitor.getMatchingEntries(AbstractJarVisitor.java:139)
 at org.hibernate.ejb.Ejb3Configuration.addScannedEntries(Ejb3Configuration.java:287)
 at org.hibernate.ejb.Ejb3Configuration.scanForClasses(Ejb3Configuration.java:614)
 at org.hibernate.ejb.Ejb3Configuration.configure(Ejb3Configuration.java:360)
 at org.hibernate.ejb.HibernatePersistence.createContainerEntityManagerFactory(HibernatePersistence.java:131)
 at org.jboss.ejb3.entity.PersistenceUnitDeployment.start(PersistenceUnitDeployment.java:280)

Response from HEM team:

"emmanuel" wrote:

You guys do not expose a directory as a directory.
So the alternative is to go read the InputStream. I don't see why VFS could not expose a directory as an InputStream (just like a zip file does). What else can I "assume"?

The proper solution is a real integration with VFS but I lack time at the moment.

No to blame them.
But I guess we're gonna see a lot of this eventaully.
So, this needs more generic solution, which doesn't expect all external libs to adopt VFS.

[dimitris]

That's exactly the point, we cannot expect external libraries to understand jboss-vfs, we need a generic solution or some sort of compatibility mode.

[starksm64]

This is the same situation that exists without the vfs for exploded vs packed jars. The only difference without the vfs is that the url prefix is probably jar: for packed, file: for exploded. There is no generic solution I can see, but this interaction between ejb3 and hibernate is part of the javax.persistence.spi.PersistenceUnitInfo where there is a contract defined to either return a file: url, or an InputStream to the jar. Its the PersistenceUnitInfo implementation that needs to be vfs aware, not hibernate.

[wolfc]

It's PersistenceUnitInfo that has the responsibility to be spec compliant, but it's VFS that has to provide the facilities to enable it to do so.

[alesj]

"wolfc" wrote:
It's PersistenceUnitInfo that has the responsibility to be spec compliant, but it's VFS that has to provide the facilities to enable it to do so.

What does JPA expect to get in the case of nested jar?
It's neither file or jar.
OK, not file is obvious.

From the spec:

.. other URL from which an InputStream in jar format can be obtained.

In our case this could be it.
If they then did this

 ZipInputStream zis;
 if (is instanceof ZipInputStream)
 zis = (ZipInputStream)is;
 else
 zis = new ZipInputStream(is);

But this would mean that the URL returned had to be VFS based - so that VFSURLConnection would kick in, pointing to nested jar. Which then returns itself as ZipInputStream.

But there is another catch.
Imagine you have that jar in exploded form.
How do you then know weatehr it's a nested jar or just plain dir?
Since for plain dir this does the trick:

 String urlString = url.toExternalForm();
 if (urlString.startsWith("vfs"))
 url = new URL(urlString.substring(3));

And we're back to Scott's description:

This is the same situation that exists without the vfs for exploded vs packed jars. The only difference without the vfs is that the url prefix is probably jar: for packed, file: for exploded. There is no generic solution I can see

[starksm64]

The only more generic way that would allow for compatibility modes would be to rely on the content-type rather than the url. However, no one uses that. They look at the url scheme or the file suffix and make assumptions about what the url points to, or its InputStream.

There are standard content-type descriptions for a zip input stream, but I don't know about a java.io.File object. Maybe there is an application/x-java-object-java.io.File type defined somewhere.

A nested jar should simply be seen as a JarInputStream.

"http://java.sun.com/javaee/5/docs/api/javax/persistence/spi/PersistenceUnitInfo.html" wrote:

getJarFileUrls

List<URL> getJarFileUrls()

Returns a list of URLs for the jar files or exploded jar file directories that the persistence provider must examine for managed classes of the persistence unit. Each URL corresponds to a named element in the persistence.xml file. A URL will either be a file: URL referring to a jar file or referring to a directory that contains an exploded jar file, or some other URL from which an InputStream in jar format can be obtained.

Returns:
a list of URL objects referring to jar files or directories.

[alesj]

A nested jar should simply be seen as a JarInputStream.

For NoCopy handling, I can't see any other way then doing either this

 // hack the JPA url
 VirtualFile root = di.getRootFile();
 URL url = root.toURL();
 String urlString = url.toExternalForm();
 if (urlString.startsWith("vfs"))
 {
 // unpack, since that's the only way
 // to make sure we point to file
 VirtualFile file = VFSUtils.unpack(root);
 url = file.toURL();
 urlString = url.toExternalForm();
 if (urlString.startsWith("vfs"))
 url = new URL(urlString.substring(3));
 }
 pi.setPersistenceUnitRootUrl(url);

or like I've already said, having this check

ZipInputStream zis;
 if (is instanceof ZipInputStream)
 zis = (ZipInputStream)is;
 else
 zis = new ZipInputStream(is);

Since probably if you include zis into zis, you don't get anything.
At least that's what I've encountered recently when hacking facelets resource lookup.

[alesj]

"alesj" wrote:

For NoCopy handling, I can't see any other way then doing either this

 // hack the JPA url
 VirtualFile root = di.getRootFile();
 URL url = root.toURL();
 String urlString = url.toExternalForm();
 if (urlString.startsWith("vfs"))
 {
 // unpack, since that's the only way
 // to make sure we point to file
 VirtualFile file = VFSUtils.unpack(root);
 url = file.toURL();
 urlString = url.toExternalForm();
 if (urlString.startsWith("vfs"))
 url = new URL(urlString.substring(3));
 }
 pi.setPersistenceUnitRootUrl(url);

This is probably bad as well - calling VFSUtils.unpack after classpath has been set.
Since pushing something to JPA to do some resources lookup, and that this is NOT part of classloader is surely wrong. :-(

Currently I'm out of ideas ...

[starksm64]

The only way I see is to add yet another custom url/handler that returns the proper ZipInputStream that can navigate whatever structure is.

 if (urlString.startsWith("vfs"))
 {
 // vfsnjar, vfsdir ...
 url = new URL(...)
 }
 pi.setPersistenceUnitRootUrl(url);

Isn't the PersistenceUnitRootUrl always pointing to the contents containing the persistence-info.xml so that there really never is any navigation of ZipInputStreams within ZipInputStreams?

[alesj]

Isn't the PersistenceUnitRootUrl always pointing to the contents containing the persistence-info.xml so that there really never is any navigation of ZipInputStreams within ZipInputStreams?

Yes.
It should be / it is pointing to the module (jar or 'exploded' jar directory) that contains that persistence.xml.

So, I'm probably reading your quote wrong

A nested jar should simply be seen as a JarInputStream.

If by that you mean that URL of nested jar should return JIS, then we're almost there - we return ZIS.

Off topic:
I remember having problems finding MANIFEST.MF file if the IS impl in nested jar handling was JIS, instead of ZIS.

[starksm64]

"alesj" wrote:

So, I'm probably reading your quote wrong

A nested jar should simply be seen as a JarInputStream.

If by that you mean that URL of nested jar should return JIS, then we're almost there - we return ZIS.

Yes, we should be there if a JarInputStream is returned.

"alesj" wrote:

Off topic:
I remember having problems finding MANIFEST.MF file if the IS impl in nested jar handling was JIS, instead of ZIS.

Having accessors for the Manifest is basically all JarInputStream adds to the ZipInputStream. Really, from "URL from which an InputStream in jar format can be obtained. ", I don't think a JarInputStream is required as ZipInputStream is in the jar format. Can you validate with the hibernate team that this works for them?

[alesj]

Can you validate with the hibernate team that this works for them?

It's probably more of a global issue, e.g. having any kind of PersistenceProvider work on our PersistenceDeployer.

What we should do now is differentiate between this:
1) exploded inner dir containing persistence.xml
2) nested jar containing persistence.xml

For 1) we strip the vfs prefix off,
for 2) we return VFS URL, which will lead to VFSURLConnection --> VirtualFile.openStream --> ZipInputStream.

I can add some VFSUtils.isNestedJar method which knows how diff between 1) and 2) based on the handler impl.

OK?

[starksm64]

Sounds fine as long as it works.