-
15. Re: Generated Classes not found if they do not match any of
"kabir.khan@jboss.com" wrote:
"adrian@jboss.org" wrote:
You haven't answered why it is defined against the user classloader?
See my point about classloader leaks.
It is defined against the user classloader since as far as I can tell:
-We cannot use the system classloader. The proxies use interfaces from the aop classloader (org.jboss.aop.proxy.container.AspectManaged and org.jboss.aop.proxy.container.Delegate) which are not visible from the system classloader.
-We cannot use the aop classloader. The proxies might have interfaces introduced by the deployment, whose classloader might be in a child domain of the aop classloader's domain. Those interfaces would not be visible from the aop classloader.
At the moment we are caching the generated proxies by deployment classloader. The classloader is the key into a WeakHashMap. If deploymentA wants to create a proxy for ArrayList with no interface introductions, we generate the proxy class using deploymentA's classloader. If deploymentB wants to create a similar proxy for ArrayList, we generate the proxy class using deploymentB's classloader. Subsequent attempts to get a plain ArrayList proxy from deploymentA's classloader will use deploymentA's cached class, and deploymentB will use deploymentB's cached class.
I guess I'm not understanding what you are doing. But what you described above
doesn't match what you originally said.
Above you have deploymentB using the proxy class from its classloader
but earlier you want it to see deploymentA's class?
If the proxy uses a deployment class then obviously it should be generated
against that deployments classloader. Other deployments then become dependent
upon that deployment and must be redeployed.
I don't see why the class can't use the deployment class's package in that case?
But that doesn't sound like your original requirement which was to do "instrumenting
JDK classes" by creating classes in the base package.
Doesn't your point about classloader leaks hold for any deployment?
Yes, that's why you need to be careful with it. e.g. ejb3 had a big memory leak
while we were developing JBoss5 because it was defining remote/local proxies
against the wrong classloader.
If its as you say that deploymentB is using a class from deploymentA and that
is explicit then it is possible to know the dependency and redeploy.
Any other semantic will break the modularity and/or cause memory leaks. -
16. Re: Generated Classes not found if they do not match any of
I think we have crossed wires :-)
"adrian@jboss.org" wrote:
"kabir.khan@jboss.com" wrote:
"adrian@jboss.org" wrote:
You haven't answered why it is defined against the user classloader?
See my point about classloader leaks.
I guess I'm not understanding what you are doing. But what you described above
doesn't match what you originally said.
Above you have deploymentB using the proxy class from its classloader
but earlier you want it to see deploymentA's class?
I'm not really sure what you mean here, but I'll try to answer what I think you're asking :-)
We don't create a proxy class every time we go to the proxy factory. There is a cache of generated classes which is used behind the scenes. When we want to CREATE a proxy class we ask the factory for it. The first time it generates it using the deployment's classloader and caches it. Subsequent attempts to CREATE the same proxy for the same classloader return the cached copy for that classloader. In another classloader if we want to CREATE a proxy for the same class we end up with a new generated proxy cached for that classloader.
When we want to USE a proxy from a different classloader, we need to be able to load the proxy class. Proxies have unique names (static counter maintained by proxy factory across all loaders). In the example I gave deploymentA's proxy for ArrayList will be AOPProxy$1, and deploymentB's proxy for ArrayList will be AOPProxy$2 (both in the default package)"adrian@jboss.org" wrote:
If the proxy uses a deployment class then obviously it should be generated
against that deployments classloader. Other deployments then become dependent
upon that deployment and must be redeployed.
I don't see why the class can't use the deployment class's package in that case?
It uses the proxied class's package if the classname does not start with "java" or "sun". For example:
-a proxy for java.util.ArrayList is AOPProxy$1 (currently in default package)
-a proxy for org.jboss.some.Thing is org.jboss.some.AOPProxy$2"adrian@jboss.org" wrote:
But that doesn't sound like your original requirement which was to do "instrumenting
JDK classes" by creating classes in the base package.
Only JDK classes should not be in the package of the proxied class, and we need to find another home for these. Deployment classes are causing no problem.
In short, we always use the classloader of the deployment rather than the classloader of the class we are proxying. This happens whether it is a system class (ArrayList) or something else. There might be interface introductions etc. that need to be able to see classes from the deployment classloader, so that is the easiest way. -
17. Re: Generated Classes not found if they do not match any of
"kabir.khan@jboss.com" wrote:
When we want to USE a proxy from a different classloader, we need to be able to load the proxy class. Proxies have unique names (static counter maintained by proxy factory across all loaders). In the example I gave deploymentA's proxy for ArrayList will be AOPProxy$1, and deploymentB's proxy for ArrayList will be AOPProxy$2 (both in the default package)
So I want to be able to look up "AOPProxy$1" from deploymentB, and "AOPProxy$2" from deploymentA. That is what is not working at the moment. -
18. Re: Generated Classes not found if they do not match any of
"kabir.khan@jboss.com" wrote:
"kabir.khan@jboss.com" wrote:
When we want to USE a proxy from a different classloader, we need to be able to load the proxy class. Proxies have unique names (static counter maintained by proxy factory across all loaders). In the example I gave deploymentA's proxy for ArrayList will be AOPProxy$1, and deploymentB's proxy for ArrayList will be AOPProxy$2 (both in the default package)
So I want to be able to look up "AOPProxy$1" from deploymentB, and "AOPProxy$2" from deploymentA. That is what is not working at the moment.
Yes that is because the package is not exported, as discussed above.
It uses the proxied class's package if the classname does not start with "java" or "sun". For example:
-a proxy for java.util.ArrayList is AOPProxy$1 (currently in default package)
-a proxy for org.jboss.some.Thing is org.jboss.some.AOPProxy$2
Only JDK classes should not be in the package of the proxied class, and we need to find another home for these. Deployment classes are causing no problem.
In short, we always use the classloader of the deployment rather than the classloader of the class we are proxying. This happens whether it is a system class (ArrayList) or something else. There might be interface introductions etc. that need to be able to see classes from the deployment classloader, so that is the easiest way.
My concern was that you had one class for all deployments defined against the
"first" classloader, which could later get redeployed and thus cause that
classloader to leak.
It doesn't sound like you are doing that. The other deployments will only
use those classes from other deployments if they are directed to do so, e.g. by passing
an object that uses that class.
OFF TOPIC:
But it does sound like you are generating instrumented classes for things like
java.util.ArrayList in every deployment which could be minimized if they were shared
in the AOP classloader. I get your point about introductions, but isn't that an edge case? -
19. Re: Generated Classes not found if they do not match any of
"adrian@jboss.org" wrote:
My concern was that you had one class for all deployments defined against the
"first" classloader, which could later get redeployed and thus cause that
classloader to leak.
That was the case before
https://jira.jboss.org/jira/browse/JBAS-5505"adrian@jboss.org" wrote:
But it does sound like you are generating instrumented classes for things like
java.util.ArrayList in every deployment which could be minimized if they were shared
in the AOP classloader. I get your point about introductions, but isn't that an edge case?
Yes, that is an optimization that can be done -
20. Re: Generated Classes not found if they do not match any of
"adrian@jboss.org" wrote:
We also haven't discussed your other point about needing to flush the blacklist.
False alarm :-)
The domain's blacklist is being cleared by BaseClassLoader via the policypublic void clearBlackList(String name) { if (blackList != null) { boolean trace = log.isTraceEnabled(); if (trace) log.trace(this + " removing from blacklist " + name); blackList.remove(name); policy.clearBlackList(name); } }
It works fine, as long as the class created is in one of the existing packages.
I did notice however that if I do//Blacklist classe //Loader A belongs to default domain and exports packageA loaderA.load("packageA.NewClassA"); //Loader B belongs to defaultDomain and exports packageB loaderB.load("packageA.NewClassA"); //Create packageA.NewClassA in loaderA loaderA.load("packageA.NewClassA"); //Works fine loaderB.load("packageA.NewClassA");
loaderB has packageA.NewClassA in its blackList, but this is never checked when loading classes. But it does seem to be used in the following method, so ClassLoaderDomain.clearBlackList() should probably be clear all the domain's classloaders?public class BaseClassLoader extends SecureClassLoader implements BaseClassLoaderMBean, RealClassLoader { URL getResourceLocally(final String name, final boolean trace) { ... // Is this resource blacklisted? if (blackList != null && blackList.containsKey(name)) { if (trace) log.trace(this + " resource is blacklisted " + name); return null; } ... } } -
21. Re: Generated Classes not found if they do not match any of
https://jira.jboss.org/jira/browse/JBAOP-698
AOP not generates proxies that previously would go in the default package in "org.jboss.aop.generatedproxies".
Proxies for classes in user packages still go into that package. -
22. Re: Generated Classes not found if they do not match any of
"kabir.khan@jboss.com" wrote:
so ClassLoaderDomain.clearBlackList() should probably be clear all the domain's classloaders?
I'm not sure I understand this. The classloader blacklists only refer to local resources.
After you generate packageA.newClassA against LoaderA, LoaderB still doesn't have
that resource so it can still remember that in its blacklist.
Where I think there might be a problem is if you go through getResourceFromImports()
and related methods because then there is some caching against the ClassLoaderInformation which isn't being cleared by those flush methods.
e.g. something like the following should fix that:[ejort@warjort jboss-cl-2.0]$ svn diff --diff-cmd diff -x "-U 10" Index: classloader/src/main/java/org/jboss/classloader/spi/base/BaseClassLoaderDomain.java =================================================================== --- classloader/src/main/java/org/jboss/classloader/spi/base/BaseClassLoaderDomain.java (revision 83809) +++ classloader/src/main/java/org/jboss/classloader/spi/base/BaseClassLoaderDomain.java (working copy) @@ -84,20 +84,23 @@ /** * Flush the internal caches */ public void flushCaches() { globalClassCache.clear(); globalClassBlackList.clear(); globalResourceCache.clear(); globalResourceBlackList.clear(); + + for (ClassLoaderInformation info : classLoaders) + info.flushCaches(); } public int getClassBlackListSize() { return globalClassBlackList.size(); } public int getClassCacheSize() { return globalClassCache.size(); @@ -1423,13 +1426,18 @@ protected void clearBlackList(String name) { if (globalClassBlackList != null) { globalClassBlackList.remove(name); } if (globalResourceBlackList != null) { globalResourceBlackList.remove(name); } + + // Need to clear the import caches as well + List<ClassLoaderInformation> infos = classLoaders; + for (ClassLoaderInformation info : infos) + info.clearBlackList(name); } } Index: classloader/src/main/java/org/jboss/classloader/spi/base/ClassLoaderInformation.java =================================================================== --- classloader/src/main/java/org/jboss/classloader/spi/base/ClassLoaderInformation.java (revision 83809) +++ classloader/src/main/java/org/jboss/classloader/spi/base/ClassLoaderInformation.java (working copy) @@ -278,16 +278,29 @@ * * @param name the resource name to black list */ public void blackListResource(String name) { Map<String, String> resourceBlackList = this.resourceBlackList; if (resourceBlackList != null) resourceBlackList.put(name, name); } + /** + * Cleans the entry with the given name from the blackList + * + * @param name the name of the resource to clear from the blackList + */ + public void clearBlackList(String name) + { + if (classBlackList != null) + classBlackList.remove(name); + if (resourceBlackList != null) + resourceBlackList.remove(name); + } + @Override public String toString() { return policy.toString(); } } -
23. Re: Generated Classes not found if they do not match any of
"adrian@jboss.org" wrote:
"kabir.khan@jboss.com" wrote:
so ClassLoaderDomain.clearBlackList() should probably be clear all the domain's classloaders?
I'm not sure I understand this. The classloader blacklists only refer to local resources.
After you generate packageA.newClassA against LoaderA, LoaderB still doesn't have
that resource so it can still remember that in its blacklist.
Ah, I see, it is the getResourceLocally() method so that will be local resources as you say. -
24. Re: Generated Classes not found if they do not match any of
"adrian@jboss.org" wrote:
Where I think there might be a problem is if you go through getResourceFromImports()
and related methods because then there is some caching against the ClassLoaderInformation which isn't being cleared by those flush methods.
This has been fixed
https://jira.jboss.org/jira/browse/JBCL-82 -
25. Re: Generated Classes not found if they do not match any of
Thanks :-) I was just about to start looking at it
-
26. Re: Generated Classes not found if they do not match any of
I'm trying to wrap this up now
"adrian@jboss.org" wrote:
I'm sure Ales would want to do something more complicated
but more dynamic and pluggable such as defining apublic interface GlobalCapabilitiesProvider { Capabilities getCapabilities() }
I'm experimenting with such a provider added to the ClassLoading bean with in/uncallbacks. What to do with it once set?
I envision something like:public class ClassLoading { private final Set<GlobalCapabilitiesProvider> globalCapabilitiesProviders = new ConcurrentSet<GlobalCapabilitiesProvider>(); //Incallback public void addGlobalCapabilitiesProvider(GlobalCapabilitiesProvider provider) { if (provider == null) throw new IllegalArgumentException("Null global capabilities provider"); globalCapabilitiesProviders.add(provider); } //Uncallback public void removeGlobalCapabilitiesProvider(GlobalCapabilitiesProvider provider) { if (provider == null) throw new IllegalArgumentException("Null global capabilities provider"); globalCapabilitiesProviders.remove(provider); } ... public void addModule(Module module) { if (module == null) throw new IllegalArgumentException("Null module"); //New line module.setGlobalCapabilitiesProviders(globalCapabilitiesProviders); String domainName = module.getDeterminedDomainName(); boolean parentFirst = module.isJ2seClassLoadingCompliance(); String parentDomainName = module.getDeterminedParentDomainName(); Domain domain = getDomain(domainName, parentDomainName, parentFirst); domain.addModule(module); } } public abstract class Module extends NameAndVersionSupport { ... /** * Set the global capabilities providers * @param provider The provider list */ void setGlobalCapabilitiesProviders(List<GlobalCapabilitiesProvider> providers) { globalCapabilitiesProviders = providers; } /** * Get the global capabilities providers * @return The providers */ protected List<GlobalCapabilitiesProvider> getGlobalCapabilitiesProvider() { return globalCapabilitiesProviders; } }
Then the plan is to use the global capabilities provider in Module when determining the capabilities. -
27. Re: Generated Classes not found if they do not match any of
"kabir.khan@jboss.com" wrote:
Then the plan is to use the global capabilities provider in Module when determining the capabilities.
There's no need to add extra state to the Module. Its shared information.
You should just delegate to the domain which will in turn delegate to the ClassLoading.
That way we can also have "Global capabilities" at the domain level in future if we want.
They can be package private methods.
In Module.java/** * Get the capabilities. * * @return the capabilities. */ public List<Capability> getCapabilities() { // Have we already worked this out? if (capabilities != null) return capabilities; // Are there any configured ones? List<Capability> capabilities = determineCapabilities(); // Use the defaults if (capabilities == null) capabilities = defaultCapabilities(); HERE!!!! // Merge in the global capabilities capabilities = getDomain().mergeGlobalCapabilities(capabilites); // Which does classloading.mergeGlobalCapabilies(capabilities) // Cache it this.capabilities = capabilities; return capabilities; } -
28. Re: Generated Classes not found if they do not match any of
I have modified classloading-vfs/src/test/resources/org/jboss/test/classloading/vfs/metadata/Common.xml to set up the in/uncallbacks (I can move this somewhere else if you're not happy with that):
<bean name="ClassLoading" class="org.jboss.classloading.spi.dependency.ClassLoading"> <incallback method="addModule" state="Configured"/> <uncallback method="removeModule" state="Configured"/> <incallback method="addGlobalCapabilitiesProvider" state="Configured"/> <uncallback method="removeGlobalCapabilitiesProvider" state="Configured"/> </bean>
My test-specific classloading-vfs/src/test/resources/org/jboss/test/classloading/vfs/metadata/test/GeneratedClassesUnitTestCase.xml<?xml version="1.0" encoding="UTF-8"?> <deployment xmlns="urn:jboss:bean-deployer:2.0"> <bean name="MockGlobalCapabilitiesProvider" class="org.jboss.test.classloading.vfs.metadata.support.MockGlobalCapabilitiesProvider"> <property name="capabilities"> <capabilities xmlns="urn:jboss:classloading:1.0"> <!-- line 6 --> <package name="newpackage"/> </capabilities> </property> </bean> </deployment>
fails to parse with the exception:org.jboss.xb.binding.JBossXBRuntimeException: {urn:jboss:classloading:1.0}capabilities not found as a child of {urn:jboss:bean-deployer:2.0}property
If I try to modify Common.xml to add classloading:1.0 there:<bean name="SchemaResolverConfig" class="org.jboss.xb.binding.sunday.unmarshalling.SchemaResolverConfig"> <property name="bindingClasses"> <map keyClass="java.lang.String" valueClass="java.lang.String"> <entry> <key>urn:jboss:classloader:1.0</key> <value>org.jboss.classloading.spi.vfs.metadata.VFSClassLoaderFactory10</value> </entry> <entry> <key>urn:jboss:classloading:1.0</key> <value>org.jboss.classloading.spi.metadata.ClassLoadingMetaData10</value> </entry> </map> </property> </bean>
I getorg.jboss.xb.binding.JBossXBException: Failed to parse source: file:/Users/kabir/sourcecontrol/jboss-cl/trunk/subversion/classloading-vfs/target/tests-classes/org/jboss/test/classloading/vfs/metadata/test/GeneratedClassesUnitTestCase.xml@6,56
The nested exception isorg.jboss.xb.binding.JBossXBRuntimeException: {urn:jboss:classloading:1.0}capabilities not found as a child of {urn:jboss:bean-deployer:2.0}property
I then noticed that 'capabilities' is spelt wrong in CapabilitiesMetaData:@XmlType(name="capabilties", propOrder= {"capabilities"}) public class CapabilitiesMetaData implements Serializable, Cloneable
and tried this in my test, but that gives me"org.jboss.xb.binding.JBossXBRuntimeException: {urn:jboss:classloading:1.0}capabilties not found as a child of {urn:jboss:bean-deployer:2.0}property" with and without classloading:1.0 in Common.xml's SchemaResolverConfig. -
29. Re: Generated Classes not found if they do not match any of
My memory is a bit faint on this, but neither AbstractBeanMetaData
@ManagementProperty(managed=true) // TODO - this ok? @XmlElement(name="property", type=AbstractPropertyMetaData.class) public void setProperties(Set<PropertyMetaData> properties)
or AbstractPropertyMetaData@XmlElements ({ @XmlElement(name="array", type=AbstractArrayMetaData.class), @XmlElement(name="collection", type=AbstractCollectionMetaData.class), @XmlElement(name="inject", type=AbstractInjectionValueMetaData.class), @XmlElement(name="search", type= AbstractSearchValueMetaData.class), @XmlElement(name="list", type=AbstractListMetaData.class), @XmlElement(name="map", type=AbstractMapMetaData.class), @XmlElement(name="set", type=AbstractSetMetaData.class), @XmlElement(name="null", type=AbstractValueMetaData.class), @XmlElement(name="this", type=ThisValueMetaData.class), @XmlElement(name="value", type=StringValueMetaData.class), @XmlElement(name="value-factory", type=AbstractValueFactoryMetaData.class) }) public void setValue(ValueMetaData value)
seem to support @XmlAnyElement. I might be looking in the wrong place. AbstractPropertyMetaData does however have this, but I am unsure what it is for@XmlAnyElement @ManagementProperty(ignored = true) public void setValueObject(Object value)
I could bypass classloading:1.0 for doing this, but if it should/can be made to work I'd like to do so.