9 Replies Latest reply on Dec 20, 2011 6:55 AM by Pedro Igor

    Custom CacheLoader for a HotRodServer's cache

    Pedro Igor Master

      Hi,

       

         I wrote a custom CacheLoader to do a on-demand load of entries from a database. To create this cacheloader I followed the instructions in https://docs.jboss.org/author/display/ISPN/Interacting+With+Hot+Rod+Server+From+Within+Same+JVM.

       

         Basically, this cacheloader does a select on a database to look for a key and creates a entry for it in the cache.  As this cache is running inside a HotServer, I created a InternalCacheEntry like this:

       

            CacheValue cacheValue = new CacheValue(this.marshaller.objectToByteBuffer(instanceRetrievedFromDatabase), 1);

       

            InternalCacheEntry entry = InternalEntryFactory.create(key, cacheValue);

       

         When a Hot Rod client tries to get a entry is always returned null and the weird thing is that when the existence of the entry is checked using the containsKey method is returned true.

       

         I'm using infinispan 5.1.0.BETA3.

       

         Any idea for this problem ?

       

      Thanks.  

        • 1. Re: Custom CacheLoader for a HotRodServer's cache
          Pedro Igor Master

          I created a testecase to startup a Hot Rod Server for debbuging. After some testes I realized that changing the marshaller to GenericJBossMarshaller my code worked.

           

          Why the default marshaller is VersionAwareMarshaller and to create the CacheValue i have to use the GenericJBossMarshaller ?

          • 2. Re: Custom CacheLoader for a HotRodServer's cache
            Galder Zamarreño Master

            Basically, VersionAwareMashaller is used internally by Infinispan to marshall things to the cache loader or other nodes in the cluster. The main difference with GenericJBossMarshaller is the fact that VersionAwareMarshaller uses the Externalizer arquitecture to provide highly performant serialization of known types. The GenericJBossMarshaller does not have this yet, and this is the marshaller used by Hot Rod clients to marshall/unmarshall things to/from Hot Rod server.

             

            The documentation link above is very specific on its of the GenericJBossMarshaller.

             

            I was wondering if you'd like to contribute your custom cache loader and provide some tests for it? It'd be interesting to see what you've done here.

            1 of 1 people found this helpful
            • 3. Re: Custom CacheLoader for a HotRodServer's cache
              Pedro Igor Master

              Thanks for the answer, but why the VersionAwareMarshaller is not used on the client as well If it is better than the GenericJBossMarshaller and is used internally by the HotRod Server ?

               

              About the contribution, of course I can do it. I will make some refactoring on the code to externalize some configurations and remove my project's specific dependencies.

               

              I will attach the code as soon as I finish on monday (19/12).

               

              Regards.

              • 4. Re: Custom CacheLoader for a HotRodServer's cache
                Galder Zamarreño Master

                Pedro Igor wrote:

                 

                Thanks for the answer, but why the VersionAwareMarshaller is not used on the client as well If it is better than the GenericJBossMarshaller and is used internally by the HotRod Server ?

                Cos we haven't got around to doing it. It's not simply a matter of changing the marshaller, but you also need to set up other components such as the externalizer table with the user defined externalizers on the client so that it can marshall/unmarshall user defined classes. Feel free to open a JIRA for this. Maybe you can even code that functionality? . We can help you along with any doubts you might have.

                 

                Looking forward to seeing your contribution!

                • 5. Re: Custom CacheLoader for a HotRodServer's cache
                  Pedro Igor Master

                  Galder,

                   

                      Take a look at the project that is attached.

                   

                      if you have any doubts please let me know.

                   

                  Regards,

                  • 6. Re: Custom CacheLoader for a HotRodServer's cache
                    Galder Zamarreño Master

                    Hmmm, nothing's attached. What about you start a document in http://community.jboss.org/en/infinispan?view=documents and we evolve over time there? You can attach code, write up something about the use case...etc. What do you think?

                    • 7. Re: Custom CacheLoader for a HotRodServer's cache
                      Pedro Igor Master

                      Now is attached.

                       

                      I think it's a great idea, but could you take a look at the implementation first ? I would like to see your feedback before doing this.

                       

                      Inside this project there is a test case called TestHotRodClient. You will se that I used a property called infinispan.client.hotrod.marshaller to change the default marshaller implementation. With this configuration is possible to use VersionAwareMarshaller on both sides without the need to use the GenericJBossMarshaller on the server side.

                      • 8. Re: Custom CacheLoader for a HotRodServer's cache
                        Galder Zamarreño Master

                        Pedro, I had a look and had a question. Couldn't you just have plugged the cache that's used by Hot Rod with our JDBCCacheStore? Wouldn't you have obtained the same thing?

                         

                        Interesting to see that the client can be configured with VersionAwareMarshaller and that it does not give you problems.

                        • 9. Re: Custom CacheLoader for a HotRodServer's cache
                          Pedro Igor Master

                          Galder, let me explain my scenario:

                           

                                   - The customer already have a table with a lot of entries. This table is maintained by an IDM (Identity Manager) to CRUD users identities. This identities will be used to authenticate users (using Picketlink IDP);

                           

                                   - To avoid create a batch service to populate the cache, I created this custom cache loader. That way, the cache will be loaded on-demand and even if user's identity is not present in-memory the cache will load it from the database;

                           

                                   - This project uses 5 nodes in cluster (DIST_ASYNC). Each of them is using this custom implementation to maintain a single database with all users identities;

                           

                                   - This project also uses 28 local caches running embbebed inside the IDP (geografically distributed). They act as clients for the cluster using the RemoteCacheStore (near-cache).

                           

                          As you can see, the JDBCCacheStore shipped with Infinispan has it's own data structure that do not suits my needs. That is the reason, reuse a existing data model.

                           

                          Actually, in this project I'm using a CacheStore and not a CacheLoader. The customer needs also to update the database when the entries in the cache are updated.

                           

                          What do you think ?