Heavy usage of VirtualFileURLConnection

[alesj]

When dealing with VFS URLs pointing to some nested resources,
things get horribly slow due to huge VirtualFileURLConnection usage.
Here is where things are explained a bit more on real Seam example:
- http://www.jboss.com/index.html?module=bb&op=viewtopic&p=4184779#4184779

The possible solutions I see.

1) try to impl those Seam use cases directly on VFS - same way we do it for Deployment and CL
Q1: can we really do it for all cases?
Q2: what about other non-integrated frameworks in JBoss5?

2) change how we handle VFS URLs - same way we do it in pre-JBoss5
Q1: what's the reason we don't return VFS URLs with temp paths?

3) implement some simple VFS caching
Q1: use real cache, e.g. LRU, or is SoftRef enough?
Q2: we need to cache intermediate paths - getting different resources from same parent faster (this requires VFURLConnection impl change)
Q3: transparent AOP usage - caching is an aspect?

Any other ideas?

[mstruk]

"alesj" wrote:

2) change how we handle VFS URLs - same way we do it in pre-JBoss5
Q1: what's the reason we don't return VFS URLs with temp paths?

VFS URL handling delegates to VFS handlers and contexts. The existence of a temp path is currently not guaranteed - depends on vfs settings and is an implementation detail. If contexts (handlers) were reused between URLs then there would be no performance problem. But as things are now, every getInputStream() request creates a new context.

"alesj" wrote:

3) implement some simple VFS caching
Q1: use real cache, e.g. LRU, or is SoftRef enough?

What would you cache? The most sensible thing to cache would be VFSContext ...

"alesj" wrote:

Q2: we need to cache intermediate paths - getting different resources from same parent faster (this requires VFURLConnection impl change)

As long as you hold a context 'open', intermediate paths are automatically kept 'open' for fast access.

"alesj" wrote:

Q3: transparent AOP usage - caching is an aspect?

We would have to identify the most appropriate join points for this. This opens another question - do we want to put cache interception only on URL VFS integration code or at VFS API level itself.

- marko

[alesj]

I've implemented simple vfs cache:

public interface VFSCache
{
 /**
 * Get the file.
 *
 * @param uri the file's uri
 * @return virtual file instance
 * @throws IOException for any error
 */
 VirtualFile getFile(URI uri) throws IOException;

 /**
 * Get the file.
 *
 * @param url the file's url
 * @return virtual file instance
 * @throws IOException for any error
 */
 VirtualFile getFile(URL url) throws IOException;

 /**
 * Put vfs context to cache.
 *
 * @param context the vfs context
 */
 void putContext(VFSContext context);

 /**
 * Remove vfs context from cache.
 *
 * @param context the vfs context
 */
 void removeContext(VFSContext context);

 /**
 * Start the cache.
 *
 * @throws Exception for any error
 */
 void start() throws Exception;

 /**
 * Stop the cache.
 */
 void stop();
}

As you can see it caches VFSContex.
Every time VFSContext is created, it is put into cache.
This creation only happens - if you use VFS API - in VFS class.

When you demand VirtualFile for given URI/URL,
I match VFSContex's rootURI to get the right context for URI/URL.
Then it's plain navigation from VFSContext to the actual relative VirtualFile.
Relative path == URI/URL path - context's path.

VFSCache comes in different impls: LRU, Timed, Soft, Weak.
It's all configurable over jboss.vfs.cache System property.

There is also optional CacheStatistics interface,
which knows how to return:
- cached contexts (unfortunately not for CachePolicy based cache)
- cache size
- last insert

I expect only a dozen contexts in this cache when running JBossAS.
e.g. deployers and deploy dir should be among them

[slaboure]

and does it improve performance? how much?

[alesj]

and does it improve performance?

No, it's there just for fun. :-)

The way it's designed it should definitely improve performance.
It's aimed at URL usage outside direct VFS usage.
e.g. 3rd party libs that mostly work on CL::getResource(s)
will now re-use existing VFSContext --> existing VirtualFile instances
In simpler words, there won't be duplicated un-packaging. ;-)

how much?

The original cause for this impl - Seam example redeploy,
now takes ~10sec instead of previous 2min+.
Is that good enough? ;-)

[slaboure]

it is a start :)

[alesj]

I added JMX statistics about this to JBossAS.
And I'm seeing some minor issue - nothing than cannot be fixed. ;-)

VFSContext Root URI
conf file:/C:/projects/jboss5/trunk/build/output/jboss-5.0.0.GA/server/default/conf/
jboss-aop-asintegration-core.jar file:/C:/projects/jboss5/trunk/build/output/jboss-5.0.0.GA/lib//jboss-aop-asintegration-core.jar
lib file:/C:/projects/jboss5/trunk/build/output/jboss-5.0.0.GA/server/default/lib/
jboss-deployers-vfs.jar file:/C:/projects/jboss5/trunk/build/output/jboss-5.0.0.GA/lib//jboss-deployers-vfs.jar
jboss-mbeans.jar file:/C:/projects/jboss5/trunk/build/output/jboss-5.0.0.GA/lib//jboss-mbeans.jar
jboss-aop-asintegration-mc.jar file:/C:/projects/jboss5/trunk/build/output/jboss-5.0.0.GA/lib//jboss-aop-asintegration-mc.jar
deploy file:/C:/projects/jboss5/trunk/build/output/jboss-5.0.0.GA/server/default/deploy/
jboss-bindingservice.jar file:/C:/projects/jboss5/trunk/build/output/jboss-5.0.0.GA/server/default/lib//jboss-bindingservice.jar
jboss-system-jmx.jar file:/C:/projects/jboss5/trunk/build/output/jboss-5.0.0.GA/lib//jboss-system-jmx.jar
jaxb-impl.jar file:/C:/projects/jboss5/trunk/build/output/jboss-5.0.0.GA/lib//jaxb-impl.jar
jboss-service.xml file:/C:/projects/jboss5/trunk/build/output/jboss-5.0.0.GA/server/default/conf/jboss-service.xml
jboss-deployers-spi.jar file:/C:/projects/jboss5/trunk/build/output/jboss-5.0.0.GA/lib//jboss-deployers-spi.jar
jboss-aop-asintegration-jmx.jar file:/C:/projects/jboss5/trunk/build/output/jboss-5.0.0.GA/lib//jboss-aop-asintegration-jmx.jar
jboss-deployers-impl.jar file:/C:/projects/jboss5/trunk/build/output/jboss-5.0.0.GA/lib//jboss-deployers-impl.jar
jboss-deployers-client-spi.jar file:/C:/projects/jboss5/trunk/build/output/jboss-5.0.0.GA/lib//jboss-deployers-client-spi.jar
wstx.jar file:/C:/projects/jboss5/trunk/build/output/jboss-5.0.0.GA/lib//wstx.jar
jboss-deployers-structure-spi.jar file:/C:/projects/jboss5/trunk/build/output/jboss-5.0.0.GA/lib//jboss-deployers-structure-spi.jar
jboss-system.jar file:/C:/projects/jboss5/trunk/build/output/jboss-5.0.0.GA/lib//jboss-system.jar
deployers file:/C:/projects/jboss5/trunk/build/output/jboss-5.0.0.GA/server/default/deployers/
jboss-deployers-core-spi.jar file:/C:/projects/jboss5/trunk/build/output/jboss-5.0.0.GA/lib//jboss-deployers-core-spi.jar
jboss-j2se.jar file:/C:/projects/jboss5/trunk/build/output/jboss-5.0.0.GA/lib//jboss-j2se.jar
jboss-jmx.jar file:/C:/projects/jboss5/trunk/build/output/jboss-5.0.0.GA/lib//jboss-jmx.jar
jboss-deployers-client.jar file:/C:/projects/jboss5/trunk/build/output/jboss-5.0.0.GA/lib//jboss-deployers-client.jar
jboss-deployers-vfs-spi.jar file:/C:/projects/jboss5/trunk/build/output/jboss-5.0.0.GA/lib//jboss-deployers-vfs-spi.jar
dom4j.jar file:/C:/projects/jboss5/trunk/build/output/jboss-5.0.0.GA/lib//dom4j.jar
jboss-deployers-core.jar file:/C:/projects/jboss5/trunk/build/output/jboss-5.0.0.GA/lib//jboss-deployers-core.jar
jboss-aop-jboss5.jar file:/C:/projects/jboss5/trunk/build/output/jboss-5.0.0.GA/lib//jboss-aop-jboss5.jar

Just need to find out where that double // is coming in.
Otherwise it looks good == behaves as expected. :-)

[alesj]

"alesj" wrote:

Just need to find out where that double // is coming in.
Otherwise it looks good == behaves as expected. :-)

Down to just this:

VFSContext - root URI
file:/C:/projects/jboss5/trunk/build/output/jboss-5.0.0.GA/lib/
file:/C:/projects/jboss5/trunk/build/output/jboss-5.0.0.GA/server/default/conf/
file:/C:/projects/jboss5/trunk/build/output/jboss-5.0.0.GA/server/default/deploy/
file:/C:/projects/jboss5/trunk/build/output/jboss-5.0.0.GA/server/default/deployers/
file:/C:/projects/jboss5/trunk/build/output/jboss-5.0.0.GA/server/default/lib/

;-)