I have a concern about how JBossDO will work when dealing with very large datasets. In the JDOAbstractCommand.executeQuery() method there is the following few lines of code (from line #186):
while(rs.next()) { for(int i = 1; i <= columnsInRow; ++i) results.add(i, rs.getObject(i)); }
Won't this try and load all of the information from the database into memory before returning? How will this work if the JDO Query returns a large number of rows (1 000 000+)? I suspect it will cause a heavy drain on the amount of resources used by the system.
About a month ago I posted patches 777134 & 777142 as an alternative approach for this. The patch implements a Collection that uses an iterator to return the data from the database, therefore the data is not loaded into memory until it is needed and is not held in memory unless the client holds a reference to it. The only drawback is that the Collection holds a reference to the open ResultSet therefore it can not be serialised.
