Anyone using Hotrod or Rest server in production?

[rm72]

Hi,

We're currently investigating Infinispan 5.2 for as an in-memory datagrid using the Hotrod or Rest server. We are primarily leaning towards Hotrod.  

The initial plan is to have about 5 machines with each machine having 8GB ram and 80GB disc. We would store about 20GB overall and the network traffic is bursty - I don't have the exact numbers but we have about 2000 concurrent users for this project.

I'm interested in any experiences you have running Hotrod or the Rest server in a production environment. I could not find much googling or searching the forum and JBoss can't or won't say which of their customers is using Hotrod in JBoss Data Grid.

Thanks

Riad

[rvansa]

Hi Riad,

I think that you rather under-resource the grid. If you want to store 20GB of data in distributed cache, with numOwners=2 (one backup), it would mean that 40GB of raw data. Moreover, storing entries carries some data overhead, roughly speaking about 200 bytes per entry. If you use, e.g. 1kB entries, that's 20% up, 48GB. And JVM does not operate well if the heap is full, full GCs occurr too often and that means several seconds of total freeze on one of the nodes.

You need some memory for the OS below the JVM, so even if you put 6GB JVM heap size you'd have only 30GB of memory, in the case of one node crash it is 24GB.

My recommendation would be to reserve resources for at most 50% raw data load in the case of one node crash, therefore, with 8 GB machines ~ 6 GB JVM's it's 20 * 2 (numOwners) / (6 (JVM size) * 0.5 (load percentage)) + 1 (for crashing node) ~= 15 nodes in the cluster

The exact values depend on the HW, network and detailed traffic pattern, but you can expect that 450 MB/s data throughput to the whole grid can be handled pretty smoothly.

[rm72]

Hi Radim,

Thanks for the insights. I hadn't fully scoped out the requirements for Infinispan so this definitely helps.

Are you using Infinispan in a production environment?

Thanks

Riad

[rvansa]

No, regrettably not in production environment, I've just done a handful of tests.