2 Replies Latest reply on Dec 10, 2010 12:14 PM by meetoblivion

    Correct approach for creating a large number of records

    meetoblivion

      I'm on JBoss AS 6.


      I have a very large process that runs.  It consists of taking an entity and persisting it as well as a number of its children - roughly 12 child rows for each parent.  I can get it to successfully process the first 40 or so records, but then eventually I get this:

       


      2010-12-10 10:10:11,257 WARN  [org.hibernate.util.JDBCExceptionReporter] (http-0.0.0.0-8080-2) SQL Error: 0, SQLState: null
      2010-12-10 10:10:11,257 ERROR [org.hibernate.util.JDBCExceptionReporter] (http-0.0.0.0-8080-2) Connection is not associated with a managed connection.org.jboss.resource.adapter.jdbc.jdk6.WrappedConnectionJDK6@bc9ef2
      2010-12-10 10:10:11,258 ERROR [org.jboss.ejb3.tx2.impl.CMTTxInterceptor] (http-0.0.0.0-8080-2) javax.ejb.EJBTransactionRolledbackException: could not insert:

       


      In my ds.xml, I have setup

       


      <max-pool-size>20000</max-pool-size>
            <idle-timeout-minutes>1</idle-timeout-minutes>
            <blocking-timeout-millis>60000</blocking-timeout-millis>

       

      Is this a situation where I want to specify the JDBC batch size as well?  Overally, I'm a bit confused about why it wants to create so many connections - looks like one per record it wants to insert.  As a way to try to break it out further and further, I've even gotten to the point of directly using SessionFactory and StatelessSession in hibernate to avoid transactions, but no luck so far.

        • 1. Re: Correct approach for creating a large number of records
          jaikiran

          Could you please post some relevant code and the entire exception stacktrace?

          • 2. Re: Correct approach for creating a large number of records
            meetoblivion

            well among my issues now is that i'm realizing conceptually, what i'm doing won't work long term.  Basically, I have this method that will insert a collection of objects into the database

             

            public void persistAsAGroup(List<Evaluation> evals) {
                    StatelessSession session = sf.openStatelessSession();
                    for(Evaluation evaluation : evals) {
                        logger.info("inserting an eval");
                        session.insert(evaluation);
                    }
                    session.close();
                }

             

            Part of my issue with this ended up being a discrepancy between the size of the objects in here and hibernate.jdbc.batch_size.  When they got to the same size, everything worked fine.  I wonder though, should the same code above work correctly if I'm using EntityManager instead of StatelessSession? Why does it generate a new database connection on each insert?

             

            BTW - it may end up that my question is more of a hibernate question rather than a JBoss EJB issue - since it's now looking more like it was started there.