6 Replies Latest reply on Nov 7, 2007 6:47 PM by Pete Muir

    Batch Index using Hibernate Search and Seam

    mat Newbie

      We need to index a large number of records using Hibernate Search;
      Following the documentations and examples from SEAM, we have implemented the followings. This is to eliminate Out Of Memory Exception and Timeout Exception when indexing a large number of records.
      The method works fine (well, haven?t tested it with a large number yet) when it is called from a stateful seam component (as the backing bean for use actions).
      However, since this would be a long running process we need to add the @Asynchronous. Adding this annotation stops the method after the EntityManager call and log.info("EntityManager initilized."); is never called !
      The persistence.xml has been modified for the specified batchSize as per documentation.

      Your help and feedback is highly appreciated;

      Environment:

      JBoss-4.2.1
      SEAM-2.0.CR1
      RichFaces-3.1.2

      @Name("indexManager")
      public class IndexManagerBean {
      
       @Logger
       static Log log;
      
       public int batchSize = 50;
      
      
       //@Asynchronous
       public void buildIndex(Class entityClass) {
       log.info("asynchronously rebuilding Lucene index for entity: " + entityClass);
       EntityManager em = (EntityManager) Component.getInstance("entityManager");
       log.info("EntityManager initilized.");
       FullTextSession fullTextSession = (FullTextSession)em.getDelegate();
       fullTextSession.setFlushMode(FlushMode.MANUAL);
       fullTextSession.setCacheMode(CacheMode.IGNORE);
       org.hibernate.Transaction transaction = fullTextSession.beginTransaction();
       //Scrollable results will avoid loading too many objects in memory
       ScrollableResults results = fullTextSession.createCriteria( entityClass ).scroll( ScrollMode.FORWARD_ONLY);
       int index = 0;
       while( results.next() ) {
       index++;
       fullTextSession.index( results.get(0) ); //index each element
       if (index % batchSize == 0) fullTextSession.clear(); //clear every batchSize since the queue is processed
       }
       transaction.commit();
      
       try {
       if (results != null) results.close();
       } catch (Exception ex) {
       ex.printStackTrace();
       }
       }
      
      }
      
      


      The entityManager is configured in the components.xml as:

       <persistence:filter name="accessCompanyFilter">
       <persistence:name>accessCompanyFilter</persistence:name>
       <persistence:parameters>
       <key>currentAccessCompany</key>
       <value>#{currentAccessCompany}</value>
       </persistence:parameters>
       </persistence:filter>
      
       <persistence:managed-persistence-context name="entityManager"
       auto-create="true"
       entity-manager-factory="#{TestEntityManagerFactory}"
       persistence-unit-jndi-name="java:/ TestEntityManagerFactory">
       <persistence:filters><value>#{accessCompanyFilter}</value></persistence:filters>
       </persistence:managed-persistence-context>
      
      
       <persistence:managed-persistence-context name="gloabalEntityManager"
       auto-create="true"
       entity-manager-factory="#{TestEntityManagerFactory}"
       persistence-unit-jndi-name="java:/TestEntityManagerFactory">
       </persistence:managed-persistence-context>
      
      



        • 1. Re: Batch Index using Hibernate Search and Seam
          Pete Muir Master

          Put a try catch Exception around all of the code in the method and see what exception is thrown.

          • 2. Re: Batch Index using Hibernate Search and Seam
            mat Newbie

            Hi Pete,

            When calling

            EntityManager em = (EntityManager) Component.getInstance("entityManager");
            


            I get the following Exception:

            07:55:26,593 ERROR [STDERR] org.hibernate.HibernateException: Filter [accessCompanyFilter] parameter [currentAccessCompany] value not set
             at org.hibernate.impl.FilterImpl.validate(FilterImpl.java:145)
             at org.jboss.seam.persistence.HibernatePersistenceProvider.enableFilter(HibernatePersistenceProvider.java:151)
             at org.jboss.seam.persistence.ManagedPersistenceContext.initEntityManager(ManagedPersistenceContext.java:88)
             at org.jboss.seam.persistence.ManagedPersistenceContext.getEntityManager(ManagedPersistenceContext.java:108)
             at sun.reflect.GeneratedMethodAccessor288.invoke(Unknown Source)
             at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
             at java.lang.reflect.Method.invoke(Method.java:597)
             at org.jboss.seam.util.Reflections.invoke(Reflections.java:21)
             at org.jboss.seam.util.Reflections.invokeAndWrap(Reflections.java:125)
             at org.jboss.seam.Component.callComponentMethod(Component.java:2083)
             at org.jboss.seam.Component.unwrap(Component.java:2109)
             at org.jboss.seam.Component.getInstance(Component.java:1888)
             at org.jboss.seam.Component.getInstance(Component.java:1841)
             at org.jboss.seam.Component.getInstance(Component.java:1835)
             at com.cheetah.seam.modules.admin.IndexManagerBean.buildIndex(IndexManagerBean.java:248)
             at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
             at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
             at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
             at java.lang.reflect.Method.invoke(Method.java:597)
             at org.jboss.seam.util.Reflections.invoke(Reflections.java:21)
             at org.jboss.seam.intercept.RootInvocationContext.proceed(RootInvocationContext.java:31)
             at org.jboss.seam.intercept.SeamInvocationContext.proceed(SeamInvocationContext.java:56)
             at org.jboss.seam.transaction.RollbackInterceptor.aroundInvoke(RollbackInterceptor.java:31)
             at org.jboss.seam.intercept.SeamInvocationContext.proceed(SeamInvocationContext.java:68)
             at org.jboss.seam.core.MethodContextInterceptor.aroundInvoke(MethodContextInterceptor.java:42)
             at org.jboss.seam.intercept.SeamInvocationContext.proceed(SeamInvocationContext.java:68)
             at org.jboss.seam.async.AsynchronousInterceptor.aroundInvoke(AsynchronousInterceptor.java:42)
             at org.jboss.seam.intercept.SeamInvocationContext.proceed(SeamInvocationContext.java:68)
             at org.jboss.seam.intercept.RootInterceptor.invoke(RootInterceptor.java:106)
             at org.jboss.seam.intercept.JavaBeanInterceptor.interceptInvocation(JavaBeanInterceptor.java:155)
             at org.jboss.seam.intercept.JavaBeanInterceptor.invoke(JavaBeanInterceptor.java:91)
             at com.cheetah.seam.modules.admin.IndexManagerBean_$$_javassist_7.buildIndex(IndexManagerBean_$$_javassist_7.jav
            a)
             at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
             at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
             at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
             at java.lang.reflect.Method.invoke(Method.java:597)
             at org.jboss.seam.util.Reflections.invoke(Reflections.java:21)
             at org.jboss.seam.util.Reflections.invokeAndWrap(Reflections.java:125)
             at org.jboss.seam.async.AsynchronousInvocation.call(AsynchronousInvocation.java:52)
             at org.jboss.seam.async.Asynchronous.executeInContexts(Asynchronous.java:76)
             at org.jboss.seam.async.Asynchronous.execute(Asynchronous.java:45)
             at org.jboss.seam.async.ThreadPoolDispatcher$RunnableAsynchronous.run(ThreadPoolDispatcher.java:114)
             at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441)
             at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
             at java.util.concurrent.FutureTask.run(FutureTask.java:138)
             at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.j
            ava:98)
             at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:207
            )
             at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:885)
             at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907)
             at java.lang.Thread.run(Thread.java:619)
            
            
            
            
            The "currentAccessCompany" is set to int values corresponding to companyId in the AuthenticatorAction bean:
            @Name("authenticator")
            public class AuthenticatorActionBean
            {
             ...
             Contexts.getSessionContext().set("currentAccessCompany", companyId);
             ...
            
            }
            


            For some reason the "value not set" error occurs due to the @Asynchronous;


            Thank you and appreciate your feedback;


            • 3. Re: Batch Index using Hibernate Search and Seam
              Pete Muir Master

              What does #{accessCompanyFilter} refer to?

              • 4. Re: Batch Index using Hibernate Search and Seam
                mat Newbie

                An int value set in the AuthenticatorActionBean seam component;
                the value refers to the company id of the users when they login as we need to restrict the entities;
                I have set it as:

                Contexts.getSessionContext().set("currentAccessCompany", companyId);
                


                The entities are filtered similarly to the wiki example but instead of "currentAccessLevel", we have "currentAccessCompany";

                The indexing method as per the wiki example also gives the same exception when using the entityManager with persistence:filter.
                The "gloabalEntityManager" defined in the components.xml works ok.

                regards,


                • 5. Re: Batch Index using Hibernate Search and Seam
                  mat Newbie

                  I submitted too soon;
                  Entities filtered with the following annotations:

                  @org.hibernate.annotations.FilterDef(
                   name = "accessCompanyFilter",
                   parameters = {@org.hibernate.annotations.ParamDef(name = "currentAccessCompany", type="integer")}
                  )
                  @org.hibernate.annotations.Filter(
                   name = "accessCompanyFilter",
                   condition = "READ_ACCESS_COMPANY = :currentAccessCompany"
                  )
                  public class Order implements Serializable {
                  
                  ...
                  
                   @Column(name = "READ_ACCESS_COMPANY", nullable = false)
                   @org.hibernate.search.annotations.Field(
                   index = org.hibernate.search.annotations.Index.UN_TOKENIZED,
                   store = org.hibernate.search.annotations.Store.YES
                   )
                   @org.hibernate.search.annotations.FieldBridge(impl = PaddedIntegerBridge.class)
                   public int getReadAccessCompany() {
                   return readAccessCompany;
                   }
                  
                   public void setReadAccessCompany(int readAccessCompany) {
                   this.readAccessCompany = readAccessCompany;
                   }
                  
                  
                  }
                  



                  • 6. Re: Batch Index using Hibernate Search and Seam
                    Pete Muir Master

                    Asynchronous methods are executed in a separate thread from the caller so don't have access to the same contexts (except for application)