I thought over the problem again this weekend. I now think it could be possible to implement an algorithm in JBoss than will lead to the desired behaviour without sacrifing any performance. Here it is:
(1) Construction of Classpath while deploying EJB (jar) file.
(a) When an EJB (jar) file is deployed the ejb-jar.xml descriptor is examined. If there is an <ejb-client-jar/> tag, a ClassLoader pointing to this jar is used as EJB interfaces ClassLoader.
(b) If there isn't an <ejb-client-jar/> tag, (home, remote, local) interface classes of the xml descriptor are used as root for an class file dependency analysis.
(c) Dependency analysis is performed by looking at the class file constant pool (for example with jakarta bcel). (Using reflection is NOT appropriate because it isn't giving us full dependencies.) Dependency analysis will split the classes in the EJB jar file in 2 sets: one closure that is used by the (home, remote, local) interfaces and one used by the implementation only. In order to build two independant ClassLoaders, the EJB jar file is repacked according to the dependency analysis.
Result of phase 1 is the construction of 2 independant ClassLoaders: one for the EJB interfaces (home, remote, local) and one for the EJB implementation.
(2) Finding the references to other (EJB) components
(a) Looking at ejb-jar.xml for <ejb-ref/> and <ejb-local-ref/> tags, the corresponding EJB interfaces ClassLoaders are found.
Result of phase 2 is a set of ClassLoaders to the EJB interfaces (home, remote, local) that are used by this EJB.
(3) Construction of conclusive ClassLoader for the EJB
The 'real' ClassLoader of the EJB consists of (from parent to child):
(a) the EJB interfaces ClassLoader
(b) the EJB interfaces ClassLoaders of the referred components/EJBs
(c) the EJB implementation ClassLoader.
I expect this algorithm to solve the ClassLoader problem in nearly all practical situations without escaping to the Hierachical ClassLoader hack. If, however the (home, remote, local) interfaces are directly dependant on classes that exist in different version within a EJB calling chain, the proposed solution certainly did NOT work. In this case, only a ByValue call would be appropriate (perhaps JBoss should allow declaration of this on component and not on container basis).
As an improvement the algorithm could be extended to seperate treatment of remote and local interfaces.
Is anybody willing to help me to implement this?