11 Replies Latest reply on Jun 16, 2011 4:59 AM by sannegrinovero

    Issues with Infinispan query

    developerguy

      Let me start by saying that this product is wonderful. From my initial evaluation, I could go to production with this pretty soon. However I am having issues with the query part. I am using version 4.2.1.CR1. I have attached all the files to this discussion.

       

      So the problem is that I need to have search along with my cache. When I turn indexing on and insert large amounts of data, it takes a very long time even if I use RAMDirectory. Moreover, the search doesnt seem to work with CacheLoaders. I need to setup a cacheloader and make it preload. But it appears that preload happens before my QueryHelper can start, so it is not able to index. So I have these two pressing issues and any help with this would be great. Please see all attached files.

       

      Another side note is that the query returns duplicate results, but that can be fixed by using a set. However, it seems to be unnecessary.

        • 1. Issues with Infinispan query
          sannegrinovero

          Hi developer, sorry for the late response I didn't notice this. I'm going to inspect you example to see what's happening with the CacheLoaders.  Could you clarify what you mean by "long time" when inserting large amounts of data (how many objects, how big are the indexed properties). ?  Also it should not return duplicate results, I'm going to check your tests.

          • 2. Issues with Infinispan query
            craigching

            Hi Sanne,

             

            I just noticed the duplicate results from query as well and came to the boards to see if anyone else was seeing it.  Let me know if you need help in reproducing the problem.

             

            Cheers

            • 3. Re: Issues with Infinispan query
              sannegrinovero

              To Developer Guy:

              I've been reviewing your tests, the problem is that they would pass in a real scenario. Your tests are failing to me because they expect one results but actually nothing is found. As you use a RAMDirectory, the index is empty when you start it up so no matches will be found, but your data is not gone as you used a permanent Cache Loader for them. if you want the index to be saved as well, you should use the FSDirectory (default) at the minimum, or use the InfinispanDirectory to have it stored in the same CacheLoader.

               

              To Craig Ching:

              I couldn't reproduce the duplicate results issue with Developer Guy's tests. I tried double-checking myself, but I can't reproduce it. Do you have a test for it?

              • 4. Issues with Infinispan query
                craigching

                I don't have a test off-hand, but I'll see what I can do.

                • 5. Issues with Infinispan query
                  craigching

                  Hi Sanne,

                   

                  I figured out what I was doing to get multiple results for the same cached object.  If I call Cache.put() with the same key multiple times, e.g.:

                   

                  cache.put("1", obj);

                  cache.put("1", obj);

                  cache.put("1", obj);

                  cache.put("1", obj);

                   

                  I get 4 results when I query.  This doesn't *seem* like the right behavior, what do you think?  If I instead remove the key before putting it again, then you only get one result for that key.  IOW, replacing an object will result in that object getting indexed multiple times.

                  • 6. Re: Issues with Infinispan query
                    sannegrinovero

                    Hi Craig, it's not that simple, I just checked this on latest version from master and it seems to properly detect duplicates.

                     

                    Could you check that version and create a test case?

                     

                    thanks,

                    Sanne

                    • 7. Issues with Infinispan query
                      craigching

                      Ahem ...

                       

                      I should've known better, of course it wasn't going to be that easy, with all the possible configuration options available, I should have taken the time to write a proper test

                       

                      Sorry about that!

                       

                      I am pretty sure I should be able to write a test that reproduces this and I can test it with the various versions and let you know.  FYI, I'm currently using 4.2.0.FINAL.

                      • 8. Re: Issues with Infinispan query
                        jroberg

                        Hi,

                         

                        I'm facing duplicate (multiple) results from queries as well.

                        We are using Infinispan 5.0.0.CR3 and our cache has a JdbcStringBasedCacheStore loader with preload="true".

                         

                        When reaching the QueryInterceptor (see StackTrace below) entry.isCreated() returns true.

                         

                        ReadCommittedEntry(a44130){

                                  key=0eebcb5b-d581-49ec-8e49-565a8f134b4c,

                                  value={objectID=0eebcb5b-d581-49ec-8e49-565a8f134b4c, street=Street 66},

                                  oldValue=null,

                                  isCreated=true,

                                  isChanged=true,

                                  isRemoved=false,

                                  isValid=true

                        }

                         

                        so the QueryInterceptor adds the entry to the index again instead of just updating the index.

                         

                              public Object visitPutKeyValueCommand(InvocationContext ctx, PutKeyValueCommand command) throws Throwable {

                         

                              // This method will get the put() calls on the cache and then send them into Lucene once it's successful.

                              // do the actual put first.

                              Object toReturn = invokeNextInterceptor(ctx, command);

                         

                              if (shouldModifyIndexes(ctx)) {

                                 // First making a check to see if the key is already in the cache or not. If it isn't we can add the key no problem,

                                 // otherwise we need to be updating the indexes as opposed to simply adding to the indexes.

                                 Object key = command.getKey();

                                 Object value = extractValue(command.getValue());

                                 updateKnownTypesIfNeeded( value );

                                 CacheEntry entry = ctx.lookupEntry(key);

                         

                                 // New entry so we will add it to the indexes.

                                 if(entry.isCreated()) {

                                    log.debug("Entry is created");

                        >>>>        addToIndexes(value, extractValue(key));

                                 }

                                 else{

                                    // This means that the entry is just modified so we need to update the indexes and not add to them.

                                    log.debug("Entry is changed");

                                    updateIndexes(value, extractValue(key));

                                 }

                                }

                              return toReturn;

                           }

                         

                         

                        Thread [main] (Suspended)

                                  LocalQueryInterceptor(QueryInterceptor).visitPutKeyValueCommand(InvocationContext, PutKeyValueCommand) line: 98

                                  PutKeyValueCommand.acceptVisitor(InvocationContext, Visitor) line: 77

                                  LockingInterceptor(CommandInterceptor).invokeNextInterceptor(InvocationContext, VisitableCommand) line: 119

                                  LockingInterceptor.visitPutKeyValueCommand(InvocationContext, PutKeyValueCommand) line: 294

                                  PutKeyValueCommand.acceptVisitor(InvocationContext, Visitor) line: 77

                                  CacheStoreInterceptor(CommandInterceptor).invokeNextInterceptor(InvocationContext, VisitableCommand) line: 119

                                  CacheStoreInterceptor.visitPutKeyValueCommand(InvocationContext, PutKeyValueCommand) line: 204

                                  PutKeyValueCommand.acceptVisitor(InvocationContext, Visitor) line: 77

                                  CacheLoaderInterceptor(CommandInterceptor).invokeNextInterceptor(InvocationContext, VisitableCommand) line: 119

                                  CacheLoaderInterceptor.visitPutKeyValueCommand(InvocationContext, PutKeyValueCommand) line: 82

                                  PutKeyValueCommand.acceptVisitor(InvocationContext, Visitor) line: 77

                                  NotificationInterceptor(CommandInterceptor).invokeNextInterceptor(InvocationContext, VisitableCommand) line: 119

                                  NotificationInterceptor(CommandInterceptor).handleDefault(InvocationContext, VisitableCommand) line: 133

                                  NotificationInterceptor(AbstractVisitor).visitPutKeyValueCommand(InvocationContext, PutKeyValueCommand) line: 60

                                  PutKeyValueCommand.acceptVisitor(InvocationContext, Visitor) line: 77

                                  TxInterceptor(CommandInterceptor).invokeNextInterceptor(InvocationContext, VisitableCommand) line: 119

                                  TxInterceptor.enlistWriteAndInvokeNext(InvocationContext, WriteCommand) line: 204

                                  TxInterceptor.visitPutKeyValueCommand(InvocationContext, PutKeyValueCommand) line: 152

                                  PutKeyValueCommand.acceptVisitor(InvocationContext, Visitor) line: 77

                                  InvocationContextInterceptor(CommandInterceptor).invokeNextInterceptor(InvocationContext, VisitableCommand) line: 119

                                  InvocationContextInterceptor.handleAll(InvocationContext, VisitableCommand) line: 96

                                  InvocationContextInterceptor.handleDefault(InvocationContext, VisitableCommand) line: 63

                                  InvocationContextInterceptor(AbstractVisitor).visitPutKeyValueCommand(InvocationContext, PutKeyValueCommand) line: 60

                                  PutKeyValueCommand.acceptVisitor(InvocationContext, Visitor) line: 77

                                  InterceptorChain.invoke(InvocationContext, VisitableCommand) line: 274

                                  CacheDelegate<K,V>.put(K, V, long, TimeUnit, long, TimeUnit) line: 505

                                  CacheLoaderManagerImpl.preload() line: 160

                                  NativeMethodAccessorImpl.invoke0(Method, Object, Object[]) line: not available [native method]

                                  NativeMethodAccessorImpl.invoke(Object, Object[]) line: not available

                                  DelegatingMethodAccessorImpl.invoke(Object, Object[]) line: not available

                                  Method.invoke(Object, Object...) line: not available

                                  ReflectionUtil.invokeAccessibly(Object, Method, Object[]) line: 172

                                  AbstractComponentRegistry$PrioritizedMethod.invoke() line: 908

                                  ComponentRegistry(AbstractComponentRegistry).internalStart() line: 706

                                  ComponentRegistry(AbstractComponentRegistry).start() line: 608

                                  ComponentRegistry.start() line: 179

                                  CacheDelegate<K,V>.start() line: 362

                                  DefaultCacheManager.createCache(String) line: 548

                                  DefaultCacheManager.getCache(String) line: 445

                                  Main.main(String[]) line: 45

                        • 9. Re: Issues with Infinispan query
                          sannegrinovero

                          I'm impressed that such a simple bug made it so far without being spotted; well done!

                           

                          Could you please create a JIRA issue and assign it to me? thanks

                           

                          https://issues.jboss.org/browse/ISPN

                          • 10. Re: Issues with Infinispan query
                            jroberg

                            I just created an JIRA issue, but don't know how to change the assignee.

                             

                            https://issues.jboss.org/browse/ISPN-1179

                            • 11. Re: Issues with Infinispan query
                              sannegrinovero

                              Thank you, it's assigned to me now, looking into it.