The Unified classloader a little too unified?

[erikreut]

"erikreut" wrote:
I have a couple of EAR files deployed on the same server. I'm currently doing development on one of these.

Some of the EARs are sharing a utility jar file (jar is in the EAR and is referred to by the individual modules manifest Class-Path inside each EAR). This means that I have a couple of copies of the same utility jar (with the same name), spread among the EARs.

I experienced a NoSuchMethodException, adding a method to the utility jar inside the EAR I'm doing further development on (the same utility jar in the other EARs do not have this method).

This leads me to think that the Classloader in JBOSS only loads my utility jar once among all the EARs (even though each EAR has its own copy).

Isn't this a problem? I wouldn't expect an EAR file to share namespace with other EARs, as I would expect the Classloader hierachy to be a tree-like structure and a module can only "go up" that structure, not "cross over".

Here's a little overview of the EAR structure:

EAR1:
- web1.war (has manifest Class-Path: utility.jar)
- ejb1.jar
- utility.jar
EAR2:
- web2.war (has manifest Class-Path: utility.jar)
- ejb2.jar
- utility.jar (is different from the other utility.jar)

I'm running with exploded EARs on JBoss 3.2.2 with integrated Tomcat.

[raja05]

"raja05" wrote:
JBoss maintains a flat namespace for all the classes that have been loaded. So if there are 2 utility.jars with the same package structure , there is a classloader issue.
To prevent this, you can scope your app classloader by adding a
<jboss-app> <loader-repository>some.dot.com:loader=webtest.ear</loader-repository></jboss-app>

in ur jboss-app.xml
This way, all the classes in ur ear file have a separate namespace, hence, separate utility.jar's

-Raj

[erikreut]

"erikreut" wrote:
I thought you guys were trying to get close to the J2EE spec with JBoss...

This way of dealing with classloaders will cause a slew of problems with enterprise-deployed servers, where applications (homegrown and 3rdparty) will have to play nicely together.

The problems really arises when you have alot of applications sharing the same jars (e.g. JDOM, Struts, POI etc..) and you don't wanna continously upgrade each and everyone of these just because, say, I want to use a newer version of Struts in a new project.

The default behaviour should be the J2EE classloading scheme, this will help ensure portability between the other vendor's appserver's and give developers (like myself) a uniform way of developing J2EE apps.

I read:

http://members.capmac.org/~orb/blog.cgi/tech/java/jboss_ucl.html

and couldn't agree more on the points made.

[erikreut]

"erikreut" wrote:
Don't get me wrong : )

I like JBoss, but I like standards more.

[frito]

"Frito" wrote:
Tell me if I am wrong, but the specs don't say anything about j2ee applications (.ear files) sharing resources.
Ok, the classloading scheme has some advantages... and disadvantages ;-) Even the decision of JBoss invoking with call by reference as default is very fast but some problems may arise with this behaviour.
But always remember: as developer you don't have to change anything. You have to change deployment descriptors as deployer, but you have to do this with every application server.

Greetings,
Frito

[clajoie]

"clajoie" wrote:
Putting aside the argument on whether the unified classloader is a good thing or not, is it possible to use the "standard" J2EE classloading scheme? If so what settings in which configuration files do I need to change in 3.2.2. The seperate repository loader is not what I want, that only chops the problem up, that is it makes a bunch of tiny unified loaders where the same issues can still occur. What I want is the standard delegating loader for everything but webapps (wars) which instead use a seperate loader for each war.

[frito]

"Frito" wrote:
Is there a "standard" J2EE classloading scheme? If so, how does it look like?
If you mean the standard java classloaders, then just configure every deployment unit with its own loader repository and let the loader delegate to the parent repository first. Don't forget to use the ByValueInvoker when invoking beans loaded in different repositories than the caller, since the jboss standard invoker does calling by reference, which will give you some errors then.
Is this what you mean?

Greetings,
Frito

[julien.dubois]

"julien.dubois" wrote:
Oh my God you are sooooo right!!
I lost DAYS because of this issue. I started having problems with the UCL in July (see http://sourceforge.net/tracker/index.php?func=detail&aid=777412&group_id=22866&atid=376685 still opened and ignored by Scott Stark). And I still have this problem.

Let's take an example where you will see this behavior is plain stupid.
I have my own framework (framework.jar), which has code useful in all my layers (Struts actions and EJB Sessions for example).
So I deploy it with my first app :
EAR 1
- framework.jar
- EJB1.jar (with manifest entry to framework.jar)
- webapp1.war
----|- struts.jar

framework.jar is used by EJB1.jar because it contains useful code for handling DAOs.

Now let's deploy a simple app, which contains just a War :
WAR 2
- webapp2.war
----|- struts.jar
----|- framework.jar

Oups, now my own tag lib fails!!!! Why?? Because my tag lib uses code from Struts. And Struts is not in my framework's classpath!! Yes, webapp2 does not use the framework.jar it came bundled with, but the framework.jar from EAR1. This is most unexpected.

Julien.

[erikreut]

"erikreut" wrote:
Julien, this just shows another strange sideeffect of using this strange way of handling namespaces.

According to the J2EE spec, section J2EE.6.2.4.8 it says that the container should provide a "per thread context classloader", which, in my world translates to i.e. a war being isolated from other modules running in the same container. The spec hints that ears at least should have separate Classloaders (section J2EE.8.2.1 paragraph 3e).

A classloader carries its own namespace, a namespace relevant only for the context it is created in, hence my expectation for a "context classloader" as described by the spes would then be, that it knows only about the resources loaded at that specific level.

The JBoss Unified ClassLoader knows alot more than just the module/application since it assumes that a namespace with the same name loaded elsewhere (in a whole other ear) will always contain the same classes and revision of classes.

This effectivly hampers any attempt to load different versions of the same classes in different applications, giving me an awesome headache thinking about an app server with alot of applications developed over alot of time where each application evolves in turn (utilizing newer 3rd party utilities than the rest of the apps).

What I what like added to JBoss would be a "J2EE ClassLoader" / "Unified Classloader" switch, which would give me the flexibility of having my different versions of the same classes in different apps. And yes, this would probably mean that I would have to run "By value" instead of "By reference" delegating to/between EJBs as well at the cost of speed the added serialization would incur. Why do you think they added Local Interfaces in EJB2.0?

And Frito, please read the spec, and tell me where the spec says delegate "across" instead of "up" the ClassLoader hierachy?

I work with one of those 3-letter company's app server as well as JBoss, and they do not suffer from the same weird behaviour.. Here we have no problem running 70+ EARs having different versions of the same utility jar spread among them.

The loader-repository approach just makes JBoss harder to work with, here I have to specifically state (and remember to do so) for each and every one of my modules that "yes, this should have its own Classloader". The danger here is, that if I forget to do it when initially deploying the ear, the same ear might break if I later deploy another ear having the a different version of the same utility jar, or the same for that matter (Just like in Julien's case).

The "ClassLoader scheme switch" would be a nice addition to JBoss.

Sorry for the long posting, but this is really one of the features that can make J2EE hard to work with in large installations if not functioning properly.

[clajoie]

"clajoie" wrote:
I disagree with the idea of the flag for the JBoss. For a long time now I've taken a particular stance when looking at the classloader mechanism in JBoss. I've thought, "I wouldn't have done it this way, but I'm sure they had their reasons and if it works then great." But after working with more and features of JBoss and deploying more and more complex applications I've finally come to the conlusion that the unified class loader is simply broken. Here's why I believe that and what I'd like to see take it's place.

Why it's broken:
1. While there is no Sun defined standard for classloading processes during J2EE application deployment there is a defacto standard. For years the most widely deployed J2EE application servers (WebLogic, WebSphere, Tomcat, etc) have performed this classloading in a particular manner and application programmers have come to expect it. JBoss has choosen to deviate from this, which admittedly is not an incorrect choice in and of itself, but this deviation has caused numerous problems, which bring me to the next point.
2. Real world users of the J2EE application server expect, and rightfully so I think, that they can deploy multiple application on the same server that may contain different versions of the same libraries without running into problems. This just isn't possible on JBoss. While a hack is provided (the ability to specify a seperate repository per ear) this doesn't solve the problem. For the issue still exists within that segmented repository. So while the scope of the problem has been narrowed it has not been resolved.
3. I've seen many posts on these forums indicating that the unified class loader has many advantages over more traditional mechanisms and that this is what JBoss users want. However, I've never seen any information saying what these benifits are nore have I seen a single user post state that the unified class loader is a good thing. I have, however, seen a large number of posts that say it is a bad thing and causing a lot of problems.
4. JBoss itself, in version 3.2.2, and above, can't even start now without getting classloader warnings when using the server configuration profiles that ship with it.

What I'd like to see:
Bascially I'd like to see JBoss behave like every other major J2EE container out there. So what does that really mean?

Well first of all it means that every webapp, whether deployed as a war or as an unpacked webapp, has it's own classloader which does NOT use a parent-delegation classloader mechanism (i.e. the standard J2SE classloading mechanism) but instead loads its classes in its own loader. Classes in this loader should be able to see up the classloading chain, that is it should be able to see things on the classpath for example, but other deployed compenents should NOT be able to see into the webapp's class loader.

EJB jars would be deployed using a parent-delegation class loading mechanism.

EARs, like webapps, would have their own classloader such that they could see up the chain but would not allow other deployed components, like other deployed ears, to see into its particular class loader. It would handle webapp and the deployment of ejb-jars as decribed above.

I realize that the above model does not take into account all the possible nuances of classloading but I believe it does accurately describe, at a high-level, the way people expect the classloader to work.

[julien.dubois]

"julien.dubois" wrote:
I agree 100% with clajoie.
Point #2 is really important. This is exactly my problem today, and to me it is a major issue if you want to use JBoss in production. I mean, even with the jboss-app.xml hack it's still not working correctly.
I have a cluster of JBoss servers, and it's supposed to serve 6 apps (for beginning) to 3500+ users. This is no game. Bill, Scott, if you read this thread : real users want a "standard" (a la Weblogic) classloader.

[julien.dubois]

"julien.dubois" wrote:
Norman Richards saw this thread, and this made him write 2 more weblog entries about (against?) the JBoss UCL.
Here they are :

http://members.capmac.org/~orb/blog.cgi/tech/java/jboss_ucl2.html

http://members.capmac.org/~orb/blog.cgi/tech/java/jboss_ucl3.html

The last one covers exactly my own problems - I really really agree with him. Yes, I'm part of the crowd using PrefixDeploymentSorter. And my ears are called 1_icf.ear, 2_hyperactif.ear, 3_desimlockage.ear, 4_gala.ear, 5_valoview.ear...

Julien Dubois.

[julien.dubois]

"julien.dubois" wrote:
Even Rickard Öberg does not like the UCL....
http://www.jroller.com/page/rickard/20031213

[juha]

I really like UCL.

So there.

[clajoie]

"clajoie" wrote:
So there? Okay that seems a bit child'ish.

But yes, I'd imagine some people do like it because it allows people to write amazingly sloppy J2EE apps. In fact the web-console application that comes with JBoss is a GREAT example this. It expects to be able to look in to the jmx-console war to get at things.

That entire issue aside, for those who do want a more standard J2EE classloading mechanism it appears that 3.2.3 provides that. I've been testing it for a little while now and it seems to work okay. Here's the scope.

1. In the jboss-service.xml for jbossweb change Java2ClassLoadingCompliance AND UseJBossWebLoader to false. You can find that file here:
$JBOSS_HOME/server/<profile>/deploy/jbossweb-tomcat41.sar/META-INF/jboss-service.xml

2. Remove the web-console webapp that comes with JBoss (cause it will complain to no end since it can't look into the jmx-console webapp anymore). You can find it here:
$JBOSS_HOME/server/<profile>/deploy/management/web-console.war

3. Change any webapps that use local EJB interfaces to use remote ones. I'll talk about this in a bit.

4. Make sure you have EJB client jars (for any EJBs your webapps use) and any libs your webapp uses packed with the war now.

That should do it. I've had pretty good success with this config. However there are a few lingering issues.

First, this only seems to fix the classloading problem for webapps, not EJBs. So if you have session beans that you want to have differing versions of the same library this doesn't fix that. I'll see what I can do about that AFTER I get our apps out into production.

Second, your webapps, it would seem, must use remote interfaces to your EJBs in this configuration. I haven't tracked down why yet, but I'm guessing it's because of the pass by reference mechanism used to implement the local interfaces in EJBs. I suspect that these things can't be seen by the now isolated webapps. However I didn't notice that supposed order of magnitude performance hit that supposedly comes with switching to remote interfaces in my webapps. I suppose it's possible that JBoss is doing some sort of magic and actually using the local interfaces under the cover for me.

Third, you do NOT need to use RMI's PortableRemoteObject.narrow() IF you are using JRMP as your transport (which is the out of the box config). Narrows are needed because of issues with IIOP and what it can and can not tell you about the serialized objects you work with. JRMP apparently doesn't have this limitation which makes sense since it has a more narrow focus then the insanely verbose IIOP.

If anyone would like to know more about our configuration here please let me know. Hope this helps some people who've been beating thief head against this problem.