We started to port a big enough application from commercial app server to JBoss. We was very excited about new JBoss 3.0 release. We forced many problems but after 2 months we have still one problem and our deadline is not so far. Our problem seats in JBoss implemetation that we discovered an described to Mr Dain Sundstrom. We are not able to get into JBoss source code to mend it.
The question is: Has anybody fired big enough EJB 2.0 application on JBoss 3.0 ?
PS. I enclode our correspondention:
> We don't use microtransacion.We are aware performance problem without
> transaction block.
> We investigate the problem and this is what we found :
> We have the follwoing relations (see number of entieties)
> (50 000 entites) Doc - DocType (6) -one typeDoc have many doc
> (300 000) Role - User (250) -one user have many
> and now what happens with the follwing code :
> User user = userHome.findByPrimaryKey(userId); //on-load cache finds many
> DocType type = typeHome.findByPrimaryKey(typdokId); //on-load cache finds
> all 4 doctypes
> //now call roleHome.create so ejbCreate is called
> setId(counter) //etc
> setUser(user); //!!!!!!!!!!! now JBoss materialize relation
> WHY ?????. It finds all roles for users loaded in cache.
> //becouse there is about 2000 roles for each user and there is 250 users
> cache then it materilizes almost whole 300 000 roles. It must take very
> //is there any way to avoid it.
> setDoc(docHome.findByPrimaryKey(docId)); //similary JBoss materializes
> docs for DocType which are already in cache. In this case all docs.
> it seems that problem arise becouse relations between number of entities
> between two sides of relation and becouse JBoss wants to load to much data
> from relation. But why this relation is loaded. I don't want to be. I
> call user.getRoles() (I know it could take long time) so I want to no
> materilize relation. In fact setDoc(0 or setUser() in postcreate should
> simply set foreing
> key id.
> I have read Optimized Loading in Your book but I cant find nothing about
> this behaviour. I'm not sure if I could avoid this behaviour with some
> loading settings. We tried to turn of read-ahead for this relation
> none nothing helps
> Maybe unidirectional realtion would not load this data (i dont know), but
> can't use unidirectaional relations becouse both sides are used in
> ----- Original Message -----
> From: "Dain Sundstrom" <firstname.lastname@example.org>
> To: "Marek Mosiewicz" <email@example.com>
> Sent: Wednesday, July 03, 2002 10:14 PM
> Subject: Re: Could we help in read a-head implementation ?
> > I don't know of any current issues. I would love any help you can give
> > on this code. I have not stress tested this code, only unit tested for
> > correctness, so any performance analyses would be greatly appreciated.
> > Have you read the Optimized Loading chapter of my documentation? If the
> > loading system is misconfigured, you can get very bad performance
> > O(n^2). The is caused by using readahead with micro transactions (tx
> > for each method invocation), which causes the system to load a lot of
> > extra data and then immediately throw it out when the tx ends.
> > -dain
> > Marek Mosiewicz wrote:
> > > Hello Dain.
> > >
> > > We port some serious EJB 2.0 application from WebLogic to JBoss 3.0.
> > > met bug [ 569077 ] read-ahead issues in JBoss 3.0.0
> > > It is little better in JBoss 3.0.1 RC1 but it still can hang for few
> > > minutes in easy operations(our PK is Integer so hashkey is
> > > implemented).Are there still any problems with read achead which You
> > > aware ? Could We help in any way ? We could analyze perofrmance, play
> > > with diferent variants or something similar.
> > >
> > > Thanks for Your great work on JBoss :-) It's really good piece of code
> > >
> > >
> > > Marek Mosiewicz
Hmmm... Marek, it seems like you have four options.
1) Sit and wait until someone fixes the problem for you for free.
2) Hire JBossGroup to fix the problem for you.
3) Fix the problem yourself by modifying the JBoss source code.
4) Don't rely on container-managed relationships, and manage the relationships yourself until you are happy with the CMR performance.
You seem to feel that #3 is not a viable option. If my job depended upon executing this project successfully, I would not consider option #1 to be viable either. So I would consider the costs of options #2 and options #4. If I only have a few entity relationships to manage and a limited budget, option #4 makes more sense. If #4 seems extremely costly, then #2 is your best option.
All your analisys are OK. Thanks ! But following your tip #2 that means that free JBoss is only the trap for catching consulting services since it does not work. And buying JBoss consulting hours can be equal to buying commercial license. We are enough smart to solve advanced problems but this one is too difficult for us. I asked whether someone has run EJB 2.0 application working on huge tables. The fact that JBoss unnecesarily caches the whole tables cannot be seen in simple test envronment with few tables having few records. So I ask : Has anybody ...
To sum up : there is no sense for big free projects because neither I am able to mend the 1MB source code nor anybody will solve complicated problems for free.
BTW: I have already hacked Tomcat and even decompiled and hacked commercial Java products so don't think I am a kind of lame.
I will try to prepare the deploying application together with the table contents that shows the problem. I will post it and when not solved there will be time to stop marketing JBoss 3.0 EJB 2.0 implemtation that does not supports n* 10k record tables.
I only have one well worn, but true quote for you.
"There is no such thing as a free lunch."
Bug 569077 "read-ahead issues" has been closed on SF lately. Also two patches have been applied to Branch_3_0 and HEAD.
If you didn't do that already I would reommend that you get Branch_3_0 from CVS (module jboss-all), compile and see if the patches fix your problem.
If they do not, please put together a test case and submit a new bug report at SF.
If you feel this is a issue that JBoss permanently should keep track of please have a look at the testsuite. If you can come up with a new test case for the testsuite to test this issue you'll probably have a 95% chance that you won't see such problems in future anymore.
Hello Tobias ! Thanks for Yopur answer. I hope you will guide mi somehow. For the first tell mi what is SF so I could follow your tips.
Hello Tobias !
I hope you will guide me somehow. For the first tell me what is SF so I could follow your tips.
JBoss is hosted on sourceforge (SF), at
What is stopping you from looking at the source code to jboss? (In your first post you appear to indicate that you cannot access the source code). If you are having problems I highly recommend running from a cvs checkout that you compile yourself. (see the "cvs" page on sourceforge. Checkout the jboss-all module). It may take a day or two to find where everything is, but I don't think it will take you long to find the cause of the behavior you are seeing. The code you need to look at is in the server/src/main/org/jboss/ejb/plugins/cmp directory. (This is under "jboss" in the cvs viewer, it is mapped to "server" on checkout).
I'm sorry for our team member. We really have encourted serious problem with Jboss, but I know that this is impossible to release bug-free product (especially so huge and compliacted as JBoss is) - In fact Weblogic which we are moving from also had very serious bugs (you could not believe but WL JDBC Driver when used in BMP EJB could corrupt occasionally data when columns had null values - this is confirmed and fixed in WL 6.1 SP 2) So I don't want to complain for JBoss which is for me one of the best OpenSource projects.
We checked 569077 read-ahead issues and this is not that problem (from HEAD).
I would try to describe it best I can:
We have relation between to tables one have 5 entities and second have 500 000. So for one entity in first table there is 100 000 in second. The problem is that JBoss tries to read this 100 000 at once (at least thier ids if cache is turned off) when it materilizes relation, what worst it does it in setting relation field in ejbPostCreate and it takes LONG time.
That problem would not arise if where better proportion between entities in relation (eg 500K in one table and 50K in second)
If we would like to avoid it we must way to read only part of relation and differetate which part of relation is read. If somebody would have some ideas I would like to help.
Another option would be to use one of the JBoss partners 3rd party cmp plugins, if you really feel you cannot get jboss cmp to work.