Introduction to new MC Scanning lib.
From requirements gathered here: http://community.jboss.org/wiki/PapakiAnnotationScanningRequirements
I've introduced a new MC Scanning sub-project: http://anonsvn.jboss.org/repos/jbossas/projects/scanning/trunk/
The main goal or idea behind this lib is very simple: unify all of JBossAS component's scanning into a single pass scan.
So, instead of doing the resources scanning for every component, we simply do it once, and properly delegate the work to these component's handles.
Another goal was to also enable usage of pre-indexed information, so the there would actually be no need for the scanning itself.
e.g. one could pre-index all of jar's annotations; e.g. its locations / class owners
Project structure
* scanning-spi
It contains simple scanner and metadata spi and initial helpers to help you extend / use simple versoin of this lib.
* scanning-impl
It provides component agnostic scanning api.
It also includes generic metadata implementation and its usage.
* plugins
This module holds custom component scanning implementations.
Current implementations are:
- annotations
- Hibernate
- hierarchy
- jsf
- web
- Weld
* deployers
Integration with VDF; new custom deployers.
* indexer
This module contains utils for creating pre-indexed handles and merging them into existing jars.
It includes Ant task and Maven plugin.
* testsuite
Tests for all other modules.
Basic building blocks
The Scanner class is trivial, it only has scan() method, so it's not really useful. :-)
The main thing is ScanningPlugin:
package org.jboss.scanning.spi;
import java.io.IOException;
import java.io.InputStream;
import java.io.OutputStream;
import org.jboss.classloading.spi.visitor.ResourceFilter;
import org.jboss.classloading.spi.visitor.ResourceVisitor;
/**
* Scanning plugin.
* Defines what to do with a resource.
*
* @param <T> exact handle type
* @param <U> exact handle interface
* @author <a href="mailto:ales.justin@jboss.org">Ales Justin</a>
*/
public interface ScanningPlugin<T extends ScanningHandle, U> extends ResourceFilter, ResourceVisitor
{
/**
* Create plugins handle/utility.
* e.g. AnnotationRepository for annotations scanning
*
* @return new handle instance
*/
T createHandle();
/**
* Read serialized handle.
*
* @param is the serialized handle's input stream.
* @return de-serialized handle
* @throws Exception for any error
*/
ScanningHandle readHandle(InputStream is) throws Exception;
/**
* Write / serialize handle.
*
* @param os the output stream to serialize handle.
* @param handle the handle
* @throws IOException for any IO error
*/
void writeHandle(OutputStream os, T handle) throws IOException;
/**
* Cleanup handle.
*
* @param handle the handle to cleanup
*/
void cleanupHandle(U handle);
/**
* Get handle interface.
*
* @return the handle interface
*/
Class<U> getHandleInterface();
/**
* Get handle's key.
* Used to attach handle to map/attachments.
*
* @return the handle's key
*/
String getAttachmentKey();
/**
* Get handle's file name.
* Used to attach handle to jar and/or get pre-indexed.
*
* @return the handle's file name
*/
String getFileName();
/***
* Get recurse filter.
*
* @return the recurse filter
*/
ResourceFilter getRecurseFilter();
}
Most of the stuff is already implemented in its abstract form (AbstractScanningPlugin) so you only need to provide the custom logic.
As you can already see from the plugin's signature, the plugin introduces a handle.
A handle is what will hold the scanning information for particular component; e.g. an annotation repository
We can see handle's implementation defined as parameter T, where handle's interface is parameter U.
/**
* Scanning handle.
*
* Represents a simple interface resource scanning results must implement
* in order to be able to merge pre-existing results.
*
* @param <T> exact handle type
* @author <a href="mailto:ales.justin@jboss.org">Ales Justin</a>
*/
public interface ScanningHandle<T extends ScanningHandle>
{
/**
* Merge existing handle with sub-handle / pre-existing handle.
*
* @param subHandle the sub handle
*/
void merge(T subHandle);
}
The main purpose of handle introduction is to allow for pre-existing handle's merging in type safe manner.
How to make usage of plugins as easy as possible in MC?
As we can see Scanner (or its actual implementations) take a set of plugins to handle artifacts.
But since plugins are mostly mutable, we need some sort of factory to help use create these plugins.
For VDF based usage this is how our factory looks like
import org.jboss.deployers.structure.spi.DeploymentUnit;
import org.jboss.scanning.spi.ScanningHandle;
import org.jboss.scanning.spi.ScanningPlugin;
/**
* Deployment based scanning plugin factory.
* Used for incallback automatching.
*
* @param <T> exact handle type
* @author <a href="mailto:ales.justin@jboss.org">Ales Justin</a>
*/
public interface DeploymentScanningPluginFactory<T extends ScanningHandle, U>
{
/**
* Is this plugin relevant to unit.
*
* @param unit the unit to check against
* @return true if it's relevant, false otherwise
*/
boolean isRelevant(DeploymentUnit unit);
/**
* Create scanning plugin from deployment unit.
*
* @param unit the deployment unit
* @return new scanning plugin
*/
ScanningPlugin<T, U> create(DeploymentUnit unit);
}
Also, as the javadoc says, this interface is nicely used for MC's incallback usage.
Usage example
Lets see what we need to implement in order to get annotation scanning into repository.
public class AnnotationsScanningPluginFactory implements DeploymentScanningPluginFactory<DefaultAnnotationRepository, AnnotationIndex>
{
public boolean isRelevant(DeploymentUnit unit)
{
// any better check? -- metadata complete is already done elsewhere
// see JBossMetaDataDeploymentUnitFilter in JBossAS
return true;
}
public ScanningPlugin<DefaultAnnotationRepository, AnnotationIndex> create(DeploymentUnit unit)
{
ReflectProvider provider = DeploymentUtilsFactory.getProvider(unit);
ResourceOwnerFinder finder = DeploymentUtilsFactory.getFinder(unit);
return new AnnotationsScanningPlugin(provider, finder, unit.getClassLoader());
}
}
public class AnnotationsScanningPlugin extends AbstractClassLoadingScanningPlugin<DefaultAnnotationRepository, AnnotationIndex>
{
/** The repository */
private final DefaultAnnotationRepository repository;
/** The visitor */
private final ResourceVisitor visitor;
public AnnotationsScanningPlugin(ClassLoader cl)
{
this(IntrospectionReflectProvider.INSTANCE, ClassResourceOwnerFinder.INSTANCE, cl);
}
public AnnotationsScanningPlugin(ReflectProvider provider, ResourceOwnerFinder finder, ClassLoader cl)
{
repository = new DefaultAnnotationRepository(cl);
visitor = new GenericAnnotationVisitor(provider, finder, repository);
}
protected DefaultAnnotationRepository doCreateHandle()
{
return repository;
}
protected ClassLoader getClassLoader()
{
return repository.getClassLoader();
}
@Override
public void cleanupHandle(AnnotationIndex handle)
{
if (handle instanceof DefaultAnnotationRepository)
DefaultAnnotationRepository.class.cast(handle).cleanup();
}
public Class<AnnotationIndex> getHandleInterface()
{
return AnnotationIndex.class;
}
public ResourceFilter getFilter()
{
return visitor.getFilter();
}
public void visit(ResourceContext resource)
{
visitor.visit(resource);
}
}
public class GenericAnnotationVisitor extends ClassHierarchyResourceVisitor
{
/** The mutable repository */
private MutableAnnotationRepository repository;
public GenericAnnotationVisitor(ReflectProvider provider, ResourceOwnerFinder finder, MutableAnnotationRepository repository)
{
super(provider, finder);
if (repository == null)
throw new IllegalArgumentException("Null repository");
this.repository = repository;
}
protected boolean isRelevant(ClassInfo classInfo)
{
return repository.isAlreadyChecked(classInfo.getName()) == false;
}
public ResourceFilter getFilter()
{
return ClassFilter.INSTANCE;
}
@Override
protected void handleAnnotations(ElementType type, Signature signature, Annotation[] annotations, String className, URL ownerURL)
{
if (annotations != null && annotations.length > 0)
{
for (Annotation annotation : annotations)
{
repository.putAnnotation(annotation, type, className, signature, ownerURL);
}
}
}
}
While the repository is just a bit smarter Map.
Integration with VDF
<bean name="ScanningMDDeployer" class="org.jboss.scanning.deployers.metadata.ScanningMetaDataDeployer"/> <bean name="ScannerDeployer" class="org.jboss.scanning.deployers.ScanningDeployer"> <property name="filter"> <bean class="org.jboss.scanning.deployers.filter.ScanningDeploymentUnitFilter"/> </property> <incallback method="addFactory" /> <uncallback method="removeFactory" /> </bean> <bean name="AnnScanningPlugin" class="org.jboss.scanning.annotations.plugins.AnnotationsScanningPluginFactory"/>
Using the Indexer
public class Main
{
private static final Logger log = Logger.getLogger(Main.class.getName());
/**
* Usage
*/
private static void usage()
{
System.out.println("Usage: Indexer <input-jar> <scanning-plugins-comma-delimited> <classpath*>");
}
/**
* Main.
* The output is file named <input-jar>.jar.mcs.
*
* @param args the program arguments
*/
public static void main(String[] args)
{
try
{
int offset = 2;
if (args.length < offset)
{
File input = new File(args[0]);
String[] providers = args[1].split(",");
URL[] urls = new URL[args.length - offset];
// add the rest of classpath
for (int i = 0; i < urls.length; i++)
urls[i] = new File(args[i + offset]).toURI().toURL();
ScanUtils.scan(input, Constants.applyAliases(providers), urls);
}
else
{
usage();
}
}
catch (Throwable t)
{
log.log(Level.SEVERE, t.getMessage(), t);
}
}
}
Pre-existing or pre-indexed information
For each scanning plugin we look for artifact's META-INF/<plugin::getFileName> entry.
String fileName = plugin.getFileName();
for (URL root : roots)
{
InputStream is = getInputStream(root, Scanner.META_INF + fileName);
if (is != null)
{
ScanningHandle pre = plugin.readHandle(is);
handle.merge(pre);
It's plugin's responsibility to know how to read pre-existing handle.
By default we use plain Java serialization together with gzip.
public ScanningHandle readHandle(InputStream is) throws Exception
{
try
{
GZIPInputStream gis = new GZIPInputStream(is);
ObjectInputStream ois = createObjectInputStream(gis);
return (ScanningHandle) ois.readObject();
}
finally
{
is.close();
}
}
public void writeHandle(OutputStream os, T handle) throws IOException
{
GZIPOutputStream gos = new GZIPOutputStream(os);
ObjectOutputStream oos = new ObjectOutputStream(gos);
try
{
oos.writeObject(handle);
oos.flush();
}
finally
{
oos.close();
}
}
How to limit scanning?
There already was a jboss-scanning.xml, I've just enhanced it a bit.
<scanning xmlns="urn:jboss:scanning:1.0"> <path name="myejbs.jar"> <include name="com.acme.foo"/> <exclude name="com.acme.foo.bar"/> </path> <path name="my.war/WEB-INF/classes"> <include name="com.acme.foo"/> </path> <path name="esb.sar/lib/ui.jar"> <include name="com.esb.bar" recurse="true"/> </path> </scanning>
The recurse filter is now a bit smarter than it used to be in previous version. :-)
package org.jboss.scanning.plugins.filter;
import java.net.URL;
import java.util.HashMap;
import java.util.List;
import java.util.Map;
import java.util.Set;
import org.jboss.classloading.spi.visitor.ResourceContext;
import org.jboss.classloading.spi.visitor.ResourceFilter;
import org.jboss.scanning.spi.metadata.PathEntryMetaData;
import org.jboss.scanning.spi.metadata.PathMetaData;
import org.jboss.scanning.spi.metadata.ScanningMetaData;
import org.jboss.vfs.util.PathTokenizer;
/**
* Simple recurse filter.
*
* It searches for path substring in url string,
* and tries to match the tree structure as far as it goes.
*/
public class ScanningMetaDataRecurseFilter implements ResourceFilter
{
/** Path tree roots */
private Map<String, RootNode> roots;
public ScanningMetaDataRecurseFilter(ScanningMetaData smd)
{
if (smd == null)
throw new IllegalArgumentException("Null metadata");
List<PathMetaData> paths = smd.getPaths();
if (paths != null && paths.isEmpty() == false)
{
roots = new HashMap<String, RootNode>();
for (PathMetaData pmd : paths)
{
RootNode pathNode = new RootNode();
roots.put(pmd.getPathName(), pathNode);
Set<PathEntryMetaData> includes = pmd.getIncludes();
if (includes != null && includes.isEmpty() == false)
{
pathNode.explicitInclude = true;
for (PathEntryMetaData pemd : includes)
{
String name = pemd.getName();
String[] tokens = name.split("\\.");
Node current = pathNode;
for (String token : tokens)
current = current.addChild(token);
if (pemd.isRecurse())
current.recurse = true; // mark last one as recurse
}
}
}
}
}
public boolean accepts(ResourceContext resource)
{
if (roots == null)
return false;
URL url = resource.getUrl();
String urlString = url.toExternalForm();
for (Map.Entry<String, RootNode> root : roots.entrySet())
{
if (urlString.contains(root.getKey()))
{
RootNode rootNode = root.getValue();
if (rootNode.explicitInclude) // we have explicit includes in path, try tree path
{
String resourceName = resource.getResourceName();
List<String> tokens = PathTokenizer.getTokens(resourceName);
Node current = rootNode;
// let's try to walk some tree path
for (String token : tokens)
{
// if we're here, the rest is recursively matched
if (current.recurse)
break;
current = current.getChild(token);
// no fwd path
if (current == null)
return false;
}
}
return true;
}
}
return false;
}
private static class Node
{
private Map<String, Node> children;
private boolean recurse;
public Node addChild(String value)
{
if (children == null)
children = new HashMap<String, Node>();
Node child = children.get(value);
if (child == null)
{
child = new Node();
children.put(value, child);
}
return child;
}
public Node getChild(String child)
{
return children != null ? children.get(child) : null;
}
}
private static class RootNode extends Node
{
private boolean explicitInclude;
}
}
Javassist based JBoss Reflect
In order not to load the actual resource's underlying class, we use Javassist under covers - via JBoss Refect project.
DeploymentUnit unit = assertDeploy(jar);
try
{
TIFScanningPlugin plugin = unit.getAttachment(TIFScanningPlugin.class);
assertNotNull(plugin);
Kernel kernel = assertBean("Kernel", Kernel.class);
KernelConfigurator configurator = kernel.getConfigurator();
ClassLoader cl = unit.getClassLoader();
String name = JarMarkOnClass.class.getName();
TypeInfo ti = configurator.getTypeInfo(name, cl);
TypeInfo visited = plugin.getResources().get(name);
assertSame(ti, visited); // let's check if the cache is working
Method findLoadedClass = ClassLoader.class.getDeclaredMethod("findLoadedClass", String.class);
findLoadedClass.setAccessible(true);
Object clazz = findLoadedClass.invoke(cl, name);
assertNull(clazz); // should not be loaded
}
finally
{
undeploy(unit);
}
But the overall usage of helper utils is plugable:
/**
* Find the util for deployment.
* Newly created utils are grouped per module.
*
* @author <a href="mailto:ales.justin@jboss.org">Ales Justin</a>
*/
public class DeploymentUtilsFactory
{
/** The default impls */
private static Map<Class<?>, UtilFactory<?>> defaults = new WeakHashMap<Class<?>, UtilFactory<?>>();
static
{
addImplementation(ReflectProvider.class, new ReflectProviderUtilFactory());
addImplementation(ResourceOwnerFinder.class, new ResourceOwnerFinderUtilFactory());
}
/**
* Add the util impl.
*
* @param iface the interface
* @param factory the util factory
*/
public static <T> void addImplementation(Class<T> iface, UtilFactory<T> factory)
{
defaults.put(iface, factory);
}
/**
* Remove the util impl.
*
* @param iface the interface
*/
public static void removeImplementation(Class<?> iface)
{
defaults.remove(iface);
}
/**
* Get util.
*
* @param unit the deployment unit
* @param utilType the util type
* @return util instance
*/
public static <T> T getUtil(DeploymentUnit unit, Class<T> utilType)
{
if (utilType == null)
throw new IllegalArgumentException("Null util type");
DeploymentUnit moduleUnit = getModuleUnit(unit);
T util = moduleUnit.getAttachment(utilType);
if (util == null)
{
UtilFactory factory = defaults.get(utilType);
if (factory == null)
throw new IllegalArgumentException("No util factory defined for " + utilType);
Object instance = factory.create(moduleUnit);
util = utilType.cast(instance);
moduleUnit.addAttachment(utilType, util);
}
return util;
}
/**
* Get module unit.
*
* @param unit the current unit
* @return unit containing Module, or exception if no such unit exists
*/
public static DeploymentUnit getModuleUnit(DeploymentUnit unit)
{
if (unit == null)
throw new IllegalArgumentException("Null unit");
// group util per module
DeploymentUnit moduleUnit = unit;
while(moduleUnit != null && moduleUnit.isAttachmentPresent(Module.class) == false)
moduleUnit = moduleUnit.getParent();
if (moduleUnit == null)
throw new IllegalArgumentException("No module in unit: " + unit);
return moduleUnit;
}
/**
* Wrap util lookup in lazy lookup.
*
* @param unit the deployment unit
* @param utilType the util type
* @return lazy util proxy
*/
public static <T> T getLazyUtilProxy(DeploymentUnit unit, Class<T> utilType)
{
// null check is in handler
LazyUtilsProxyHandler<T> handler = new LazyUtilsProxyHandler<T>(unit, utilType);
Object proxy = Proxy.newProxyInstance(unit.getClassLoader(), new Class[]{utilType}, handler);
return utilType.cast(proxy);
}
/**
* Get reflect provider.
*
* @param unit the depoyment unit
* @return the provider
*/
public static ReflectProvider getProvider(DeploymentUnit unit)
{
return getUtil(unit, ReflectProvider.class);
}
/**
* Get finder.
*
* @param unit the depoyment unit
* @return the finder
*/
public static ResourceOwnerFinder getFinder(DeploymentUnit unit)
{
return getUtil(unit, ResourceOwnerFinder.class);
}
/**
* Cleanup the util.
*
* @param util the util to cleanup
*/
public static void cleanup(Object util)
{
if (util instanceof CachingResourceOwnerFinder)
CachingResourceOwnerFinder.class.cast(util).cleanup();
}
}
Meaning it's easy to swap utils behavior for particular deployment unit.
e.g. different ResourceOwnerFinder or ReflectProvider
Reporting issues
As usual, use the forums:
* MC user forum: http://community.jboss.org/en/jbossmc
* MC dev forum: http://community.jboss.org/en/jbossmc/dev
Or custom JIRA:
* https://jira.jboss.org/jira/browse/JBMCSCAN
Comments