Huge memory consumption with finder method

[msquance]

Hi,

I've found an issue with memory consumption when running a find method on an entity bean (using BMP)where the result is a large number of objects (e.g. 5000).

I have a simple client that is running a "findAll" method for an entity bean. My ejbFindAll method completes within a reasonable amount of time and does not consume an unusual amount of memory. However, before the findAll completes, the container is doing something that allocates a huge amount of memory (about 80 Megs when I'm dealing with 3600 objects).

I've run the server with the GC verbose flag and a bit of other trace and I see the following:

- my ejbFindAll ends
- after this, I see the memory steadily creep up from about 4000K to 36000K

[GC 4356K->3897K(4872K), 0.0045291 secs]
[GC 4408K->3921K(4872K), 0.0029208 secs]
[GC 4433K->3956K(4872K), 0.0030560 secs]
[GC 4461K->3991K(4872K), 0.0022601 secs]
[GC 4503K->4008K(4872K), 0.0024855 secs]
[GC 4518K->4018K(4872K), 0.0021673 secs]
[GC 4530K->4059K(4872K), 0.0027973 secs]
...

[GC 37414K->35453K(58716K), 0.0043559 secs]
[GC 37495K->35544K(58716K), 0.0040648 secs]
[GC 37592K->35626K(58716K), 0.0044386 secs]
[GC 37673K->35720K(58716K), 0.0041357 secs]
[GC 37765K->35780K(58716K), 0.0046285 secs]
[GC 37828K->35870K(58716K), 0.0043427 secs]
[GC 37918K->36018K(58716K), 0.0069305 secs]
INFO 2001-09-26 14:58:52.469 [OracleDS] Pool OracleDS [0/1/4] returned object
org.jboss.pool.jdbc.xa.wrapper.XAConnectionImpl@4ecfdd to the pool.

- after the trace for the OracleDS being returned, the memory jumps up very quickly
[GC 37682K->36431K(58716K), 0.0117493 secs]
[GC 38027K->37482K(58716K), 0.0110483 secs]
[GC 39609K->39566K(58716K), 0.0037432 secs]
[GC 43817K->43734K(58716K), 0.0035577 secs]
[GC 52233K->52070K(58716K), 0.0048461 secs]
[Full GC 52070K->44902K(77144K), 0.2997856 secs]
[GC 61896K->61862K(77144K), 0.0082016 secs]
[Full GC 61862K->53382K(91276K), 0.3706804 secs]
[GC 87231K->87174K(91276K), 0.0289799 secs]
[Full GC 87174K->68390K(116292K), 0.8860492 secs]

After several seconds or so of idle time, I see the GC free the bulk of the memory.
[Full GC 97663K->2295K(116292K), 0.3726851 secs]

I have found this with both JBoss 2.2.1 and 2.4.0, on both a Windows and UNIX platform. I've tried different bean cache sizes and hasn't made a difference.

What is the container doing here? This results in very poor find performance and ultimately out of memory problems.

Is it a configuration issue that I'm missing?

Thanks for any information,
Mike.

[fionnuala]

Hi
We too found similar issues with our application which contains EJB calls. Our application does EJB finds and queries on various tables in a Hypersonic database. When we ran it through a thread/memory analyser we noticed the amount of memory that JBOSS was
using continually increased (even during idle times). Eventually after several hours the application fell over with an "out of memory" exception. The rest of the application is behaving normally. The platform is Windows NT 256MB of memory, Java 1.3.1 and Jboss 2.4.1 (we also ran it with Jboss 2.2.1). We just installed JBOSS as it is and haven't changed any of the
configuration files.
Any information appreciated
Fionnuala

[jardia]

Hi.
Relates to the original posting. I've seen this discussed in a lot of books and found the stuff from sun quite useful...

"Typically, a client uses the results of a query for read-only purposes, such as displaying the result
list. Often, the client views only the first few matching records, and then may discard the remaining
records and attempt a new query. The search activity often does not involve an immediate transaction on the
matching objects. The practice of getting a list of values represented in entity beans by calling an
ejbFind method which returns a collection of remote objects, then calling each entity bean to get the
value is very network expensive and considered a bad practice." - extract from...read on

http://developer.java.sun.com/developer/restricted/patterns/ValueListHandler.html

related stuff:
http://developer.java.sun.com/developer/restricted/patterns/ValueObject.html

Cheers
Oliver Henlich
www.pogo-tech.com

[fionnuala]

Hi,

Thanks for your help. As you suggested the articles indicate that using the finder method and not using all the objects is bad practice. Unfortunately this situation does not apply to us as we always use all the results returned from the finder method.

our code goes something like this...

Object ref = context.lookup("blah/GL");
GLHome glHome = (GLHome) PortableRemoteObject.narrow(ref, blah.GLHome.class);

GL gl = glHome.create();
Collection c = glfindBySomething(possibletableentry);

Iterator i = c.iterator()
while (i.next()) {

/*
* Do some stuff here with the information
*/

}

and thats it..... are we missing something?

The thread/memory anaylser indicates that there is alot of memory being allocated in these methods but (my understanding is) the code is generated by JBOSS itself. Has something inside the ejb code still got a reference to the ejb object returned from the finder method and this is why the Garbage Collector cannot clear up the memory?

I was hoping if you (or anyone else) had come across anything else that relates to this sort of problem..... surely this is a extremely serious problem and if we are not doing anything wrong then someone else nust have come across the same problem and has a workaround........ again any help greatly appreciated

Thanks
Fionnuala