5 Replies Latest reply on Aug 31, 2011 6:09 AM by Galder Zamarreño

    Help - preload cache seems hanging

    JaeHee Roh Newbie



      I am new to Infinispan. I am evaluating Infinispan by applying existing product if we can gain performance improvement.

      Currently, I am using glassfish application server and use toplink essential as JPA.

      I don't want to use infinispan as Hibernate L2 cache.

      To reduce development time, I use existing entity class by removing all JPA annotation as POJO to be stored in cache.

      For search (or query) cache, I am using Hibernate Search and Apache Lucene.

      I will test with replication mode and distributed mode.


      For now, I am doing replication mode first.


      Here are my concerns because I am not sure if I am going to right direction or not:

      1) What to put into Cache

      I have a subscriber and each subscriber has a list of users.

      I created 2 caches; one for subscriber and the other for user.



      public class Subscriber implements Serializable {


           private String id;



           private Collection<User> users;





      public class User implements Serializable {


          private String id;



          private Subscriber subscriber;



      I create subscriberCache to put id as key and subscriber object as value which contains list of users.

      I create userCache to put id as key and user object as value.


      Question: I don't know if this is good way or not. Each cache has copy of other objects in other cache.

      Only benefit is that if I need to search subscriber by user name, then I can just search subscriber cache.

      I don't think that cache has notion about join.

      Altenative, I can store list of user ids instead of list of user objects in Subscriber class and subscriber id instead of Subscriber object in User class.

      Then, I need to search user cache to get subscriber id by user name and then search subscriber cache by subscriber id.

      This is ok for replicated cache but I am concerning about distributed cache.


      2) InfinispanDirectory

      I am confusing about infinispan directory.

      I have 2 caches already and I want to save indexes into persistent storage.

      When I use "infinispan" as hibernate.search.default.directory_provider and setting preload=true in cacheloader, when I call DefaultCacheManager.getCache(subscriber), application hangs.


      I don't find good document or example how to use infinispan as directory provider.

      In infinispan document, https://docs.jboss.org/author/display/ISPN/Infinispan+as+a+Directory+for+Lucene, it just shows to create constructor.

      new InfinispanDirectory(cache, "indexName")

      Is this directory a file system or do I need to create cache for index and use cache store/loader?



        • 1. Re: Help - preload cache seems hanging
          Sanne Grinovero Master

          Hi, you're having some interesting use cases.

          To answer you first question, I wonder if you tried Hibernate OGM ? It's still in early development, not really production ready, but it supports mapping relations correctly on top of Infinispan which is quite tricky to do.




          To answer your second question, yes when using the Query module with an infinispan backend it will start a second cachemanager, and you should make sure it's configured differently as this one should not join the primary cluster; I didn't try this myself but I believe you might have a bootup deadlock. You don't need to follow the "Infinispan as a Directory for Lucene" guide, that's meant for low level direct access to the index; you should refer to the Hibernate Search documentation to configure the Directory.

          • 2. Re: Help - preload cache seems hanging
            JaeHee Roh Newbie



            I haven't tried Hibernate OGM. I will try it if I have time.

            If I don't use Hibernate OGM, should I use Collection<String> instead of Collection<User> in Subscriber class and have subscriberId in User class?

            Infinispan does not provide API of joining of 2 caches, if I need to get subscriber by user name, then my search has to be

            1) Get subscriber id by user name in User Cache

            2) Get subscriber by subscriber id in Subscriber Cache.

            I don't know if this is good enough for performance in the case of distributed cache. (ok for replication because it is just one get from other map)


            For InfinispanDirectoryProvider, I update my configuration.xml file to force to use same DefaultCacheManager by setting

            hibernate.search.infinispan.cachemanager_jndiname and I defined 3 named caches I copied from default-hibnernatesaerch-infinispan.xml file.

            Then, I got the following error.

            In the FileListOperations class, FileListCacheKey is built by indexName which is provider name so it should be ok.

            I dont' know if hanging problem is related to this lucene directory problem or sharing same cache manager.

            I am really lost in lucene directory.

            I will try to use default InfinispanDirectory with different cache manager and if I have problem, I will post it again.




            javax.ejb.EJBException: nested exception is: java.rmi.ServerException: RemoteException occurred in server thread; nested exception is:

            java.rmi.RemoteException: null; nested exception is:

            java.lang.IllegalArgumentException: Indexing only works with entries keyed on Strings, primitives and classes that have the @Transformable annotation - you passed in a class org.infinispan.lucene.FileListCacheKey

            at org.infinispan.query.backend.KeyTransformationHandler.keyToString(KeyTransformationHandler.java:149)

            at org.infinispan.query.backend.QueryInterceptor.addToIndexes(QueryInterceptor.java:197)

            at org.infinispan.query.backend.QueryInterceptor.visitPutKeyValueCommand(QueryInterceptor.java:103)


            at org.infinispan.CacheImpl.putIfAbsent(CacheImpl.java:524)

            at org.infinispan.CacheSupport.putIfAbsent(CacheSupport.java:74)

            at org.infinispan.lucene.FileListOperations.getFileList(FileListOperations.java:58)

            at org.infinispan.lucene.InfinispanDirectory.list(InfinispanDirectory.java:183)

            at org.infinispan.lucene.InfinispanDirectory.listAll(InfinispanDirectory.java:338)

            at org.apache.lucene.index.SegmentInfos$FindSegmentsFile.run(SegmentInfos.java:637)

            at org.apache.lucene.index.SegmentInfos$FindSegmentsFile.run(SegmentInfos.java:589)

            at org.apache.lucene.index.SegmentInfos.read(SegmentInfos.java:355)

            at org.apache.lucene.index.IndexReader.indexExists(IndexReader.java:808)

            at org.hibernate.search.store.DirectoryProviderHelper.initializeIndexIfNeeded(DirectoryProviderHelper.java:157)

            at org.hibernate.search.infinispan.InfinispanDirectoryProvider.start(InfinispanDirectoryProvider.java:88)

            at org.hibernate.search.store.DirectoryProviderFactory.startDirectoryProviders(DirectoryProviderFactory.java:144)

            at org.hibernate.search.spi.SearchFactoryBuilder.initDocumentBuilders(SearchFactoryBuilder.java:403)

            at org.hibernate.search.spi.SearchFactoryBuilder.buildIncrementalSearchFactory(SearchFactoryBuilder.java:196)

            at org.hibernate.search.spi.SearchFactoryBuilder.buildSearchFactory(SearchFactoryBuilder.java:147)

            at org.hibernate.search.impl.MutableSearchFactory.addClasses(MutableSearchFactory.java:247)

            at org.infinispan.query.backend.QueryInterceptor.enableClassesIncrementally(QueryInterceptor.java:253)

            at org.infinispan.query.backend.QueryInterceptor.enableClassesIncrementally(QueryInterceptor.java:260)

            at org.infinispan.query.backend.QueryInterceptor.enableClasses(QueryInterceptor.java:231)

            at org.infinispan.query.SearchManagerImpl.buildQueryBuilderForClass(SearchManagerImpl.java:80)

            • 3. Re: Help - preload cache seems hanging
              JaeHee Roh Newbie



              I tried with different cache manager using different configuration xml.

              I don't get FileListCacheKey related exception but I have a problem with search on cache created by other cache manager.

              If I want to use InfinispanDirectoryProvider, do I have to use different cache manager and each cache manager has different configuration?

              I want to have one configuration file defining subscirber and user cache and lucene metadata, data, locking cache.

              This means that one cachemanager per JVM and managing all caches + index caches.


              I have a hard time to understand how indexes are applied to subscriber/user caches.

              I even use "ram" as lucene directory provider and when I put "preload=true" in subscriber/user caches, it hangs.

              During startup, I create cache manager with attached configuration file.

              I create Subscriber and User object and put it to each cache.

              cacheManager = new DefaultCacheManager("test-cache.xml"); -> manager status is "INSTANTIATED"

              register this cacheManager to JNDI for other components to use cache manager.

              subscriberCache = cacheManager.getCache("subscriber"); -> cache status is "INIATAILIZED"

              subscriber = new Subscriber(...);

              subscriberCache.put("sub1", subscriber);


              Once I create subscriber, I see DB has subscriber data.

              I restart cluster with preload=true on subscriber/user cache.

              cacheManager.getCache("subscriber") hangs.

              I dont' have any exception so I don't know why it hangs.


              I dont' know if index is already created when I create subscriber object, but I don't know how to check index.

              I suspect hanging problem may relate to index but I am not sure.

              BTW, I am using infinispan 5.0.0.FINAL.


              I attached infinispan.log file.

              Basically, "Preloading transient state from cache loader org.infinispan.loaders.jdbc.stringbased.JdbcStringBasedCacheStore@..." is the last line.

              This thread hangs forever.

              Infinispan.log does not show much.

              Is it bug?


              If anyone makes preload works, please let me know what I did wrong.



              • 4. Re: Help - preload cache seems hanging
                Galder Zamarreño Master

                What about you get a thread dump when preloading hangs so that we can see what's going on with that thread and why is it hanging?

                • 5. Re: Help - preload cache seems hanging
                  Galder Zamarreño Master

                  In fact, get several ones, every 30 secs or so once you notice the hang