Affinity accross caches - static or dynamic
teroheinonen Feb 26, 2010 6:53 AMHi, first of all great thanks for your fantastic work. We were just about to launch internal development effort based on JGroups to reach somewhat similar goals (having been evaluating a lot of cache products both open source and commercial without finding a perfect solution for our needs).
Infinispan seems to be the best thing since... you name it.
Background: we develop real-time tariffing and business logic solutions for very large (up to 100M subscriber) mobile operators. Current technology is based on C++ in real-time parts. Now looking at the create the next generation core, entirely on Java. The usage pattern is as follows:
- relatively simple data model
- system wide information: infrequent changes
- partitionable transactions: from 1.000 up to 100.000 transactions per second to be processed, with response time requirement in tens of milliseconds
- local server clusters serving the requests locally
- geographical distribution of system wide information to several sites accross several time zones over unreliable and/or congested links
Key architectural requirements:
- small, simple, self-contained, POJO
- extreme high performance and scalability requirements, with the need to define the ACID properties in a very finegrained level (particularly to optimize between performance and durability)
Infinispan seems to offer the perfect level of functionality and abstraction, as we want:
- the cache to take care of distribution, cluster management, state transfers, and persistence
- automatic partitioning with "unlimited" scalability => DIST model
- configurable ACID properties
- avoid configuration clutter of multiple platforms
- avoid version management challenges between a number of components (or to have somebody to take care of that for us...)
- programmatically and dynamically manage the entity relations at each node (such as "secondary key" indices etc) => flat storage model works best
- avoid lock-in to a particular implementation => Map-style interface is great as it allows easy migration between cache implementations should the need arise
So far, so good.
I have a couple of questions / requests for advice (posting others separately in a separe thread(s) as they are unrelated).
We would benefit from programmable affinity either through:
1) statically (e.g. annotating a certain field as the key for consistent hashing)
- in our case this cannot be based on the hashCode() as the affinity requirement and object identity are not directly related
2) dynamically (e.g. providing a overridable/annotatable consistentHashCode() function)
This would help to assign entities (in multiple Caches), which are needed together in the same nodes (to avoid remote lookups)
Is there a way to achieve this with the current implementation? How? Is it in the roadmap?
We will be testing infinispan more over a few weeks, will post back results.