4 Replies Latest reply on Mar 3, 2010 3:57 AM by mircea.markus

    Affinity accross caches - static or dynamic

      Hi, first of all great thanks for your fantastic work. We were just about to launch internal development effort based on JGroups to reach somewhat similar goals (having been evaluating a lot of cache products both open source and commercial without finding a perfect solution for our needs).

       

      Infinispan seems to be the best thing since... you name it.

       

      Background: we develop real-time tariffing and business logic solutions for very large (up to 100M subscriber) mobile operators. Current technology is based on C++ in real-time parts. Now looking at the create the next generation core, entirely on Java. The usage pattern is as follows:

      - relatively simple data model

      - system wide information: infrequent changes

      - partitionable transactions: from 1.000 up to 100.000 transactions per second to be processed, with response time requirement in tens of milliseconds

      - local server clusters serving the requests locally

      - geographical distribution of system wide information to several sites accross several time zones over unreliable and/or congested links

       

      Key architectural requirements:

      - small, simple, self-contained, POJO

      - extreme high performance and scalability requirements, with the need to define the ACID properties in a very finegrained level (particularly to optimize between performance and durability)

       

      Infinispan seems to offer the perfect level of functionality and abstraction, as we want:

      - the cache to take care of distribution, cluster management, state transfers, and persistence

      - automatic partitioning with "unlimited" scalability => DIST model

      - configurable ACID properties

      - avoid configuration clutter of multiple platforms

      - avoid version management challenges between a number of components (or to have somebody to take care of that for us...)

      - programmatically and dynamically manage the entity relations at each node (such as "secondary key" indices etc) => flat storage model works best

      - avoid lock-in to a particular implementation => Map-style interface is great as it allows easy migration between cache implementations should the need arise

       

      So far, so good.

       

      I have a couple of questions / requests for advice (posting others separately in a separe thread(s) as they are unrelated).

       

      We would benefit from programmable affinity either through:

       

      1) statically (e.g. annotating a certain field as the key for consistent hashing)

      - in our case this cannot be based on the hashCode() as the affinity requirement and object identity are not directly related

       

      2) dynamically (e.g. providing a overridable/annotatable consistentHashCode() function)

       

      This would help to assign entities (in multiple Caches), which are needed together in the same nodes (to avoid remote lookups)

       

      Is there a way to achieve this with the current implementation? How? Is it in the roadmap?

       

      We will be testing infinispan more over a few weeks, will post back results.

        • 1. Re: Affinity accross caches - static or dynamic
          mircea.markus

          Thank you for the nice words Tero!

           

           

          1) statically (e.g. annotating a certain field as the key for consistent hashing)

          - in our case this cannot be based on the hashCode() as the affinity requirement and object identity are not directly related

          ISPN allows you to specify the ConsistentHash implementation. What would need to be done here is extend default implementation to look for such annotations first, and if present take them from the field rather than the #hashCode

           

          2) 
          dynamically (e.g. providing a overridable/annotatable 
          consistentHashCode() function)

          You are allowd to specify the consisten hash function through configuration. This would override the default consistent hash function. Is that what you need?

           

          Is there a way to achieve this with the current implementation? How? 
          Is it in the roadmap?

          It's not, but I think it is something that's quite useful. Mind creating a JIRA for this? Also if you feel like implementing it, would be extraordinary! Once implemented you can use is stright away, witouth having to wait for a new ISPN release.

           

          We will be testing infinispan more over a few weeks, will post back 
          results.

          Thanks! We've developed an benchmarking tool to help you evaluate ISPN's perfromance on your environment. You can find it here.

           

          HTH

          • 2. Re: Affinity accross caches - static or dynamic
            manik

            Hi Tero

            Further to Mircea's comments, I suggest you read this blog article I wrote a few months ago:

             

                 http://infinispan.blogspot.com/2009/08/distribution-instead-of-buddy.html

             

            It was primarily targeted at folks using JBoss Cache's Buddy Replication and looking for similar behaviour in Infinispan (and as such most of it would not apply to you), but one interesting aspect of Buddy Replication is affinity of entries and co-locating entries with such affinity together.  Now this is implicit in the design of Buddy Replication, but in the article I do discuss a few strategies for achieving similar effects in Infinispan with Distribution.

             

            Cheers

            Manik

            • 3. Re: Affinity accross caches - static or dynamic

              Thanks for the clarification. I added Jira https://jira.jboss.org/jira/browse/ISPN-359 based on my original request to consider.

               

              However, even the configurable ConsistentHash seems actually to do the work, assuming 1) this is only used for locating the nodes in the cluster, 2) there is no dependency in implementation between ConsistentHash and hashCode()

               

              I was thinking something like:

               

              public interface Affiniable

              {

                   public int affinityHash();

              }

               

              public AffiniableConsistentHash extends DefaultConsistentHash

              {

                   ...

               

                   @override

                 protected int hash(Object o) { // needs to be changed to protected in DefaultConsistentHash

                    int h;

                    if (o instanceOf Affiniable) {

                        h = ((Affinable)o).affinityHash();

                    } else {
                        // Borrowed from Sun's JDK, a bit spreader to help normalize distribution.
                         // Uses a variant of single-word Wang/Jenkins hash.
                         h = o.hashCode();
                    }

                    h += (h << 15) ^ 0xffffcd7d;
                    h ^= (h >>> 10);
                    h += (h << 3);
                    h ^= (h >>> 6);
                    h += (h << 2) + (h << 14);
                    return h ^ (h >>> 16);
                 }

                   ...

              }

              • 4. Re: Affinity accross caches - static or dynamic
                mircea.markus
                thanks for this! What about using method annotations for this? Also, I'm thinking that for keys that cannot be annotated (e.g. 3rd party classes) it would be nice to be able to specify this in the configuration. Anyway these are rather implementation details.