Large scale infinispan deployments?

[eboily]

Hi,

Last February, Manik said here:

Where we stand right now, we are very close to releasing GA (general availability) to the community. The reason why we've held back is we finally got our hands on a very large grid to test on. In the past we've only been testing on medium sized grid of 20 or 30 nodes and we finally got a chance to test on 1000 nodes.

Are the results of these large scale deployments (1000 nodes) test available?

I am trying to get information on some large scale infinispan deployment to see to what level it has been tested in term of number of nodes, number of elements, geographical distribution, etc.

Are there any reference deployments? I did some googling but did not find much information.

Thanks for your answers.

- Edouard

[manik]

Hi there - a lot of in-house testing is currently ongoing.  Here are some results, pushed up to 48 nodes - http://infinispan.blogspot.com/search/label/benchmarks - but this is on Infinispan 4.0.  4.1 has got significantly faster too.  Due to resourcing constraints, we've only just had the time to pick up on the benchmarking from before.  Expect to see more results posted. 

In terms of usage, I know of a few large-scale apps designing their systems around 400 - 600 node Infinispan clusters, if that helps you.  We will be encouraging them to write up white papers on the subject as their systems go live.

[girishadat]

Do Infinispan has some way to manage the huge memory for JVM heap for large scale deployments? This is an area where users of JBossCache or Infinispan can get stuck.

Like for terracotta : http://www.theserverside.com/news/thread.tss?thread_id=60893

Also, http://www.theserverside.com/feature/Tackling-the-Garbage-Collection-Problem-of-the-Java-Virtual-Machine

The distributed mode is a reply for this. Still, do we have some GC recommendations?

[owentran]

I'm actually working on swapping out 2x32 Gb memcached servers with Infinispan and planning to use CMS (concurrent mark sweep). It works like a charm for my Java tomcat servers running 12 Gb heaps and have only seen the sporadic Full GC and when I do, it's about 4-8 seconds and not 1-2 minutes as described in the article. Keep in mind, I'm not running a financial trading system or mission critical application.

[galder.zamarreno]

CMS works well with apps that are not very CPU heavy as it uses parts of CPU to improve GC. However, if you're app is very CPU centric, you can instead use Infinispan as a server, for example a Hot Rod server, and tune each JVM accordingly.

[dlmarion]

Manik,

  For the 1000 node test, what modifications did you have to make to the jgroups.xml file? Did you test replication, distribution, or both on the 1000 nodes?

Dave Marion