cache.size() and cache.keyset() in Remote Client-Server Distributed cache

[raghav_11]

I am trying to set up Remote Client-Server mode cache(distributed cache) using Hot Rod API.

I am using 2 remote Hot Rod Servers and added configuration for ip-address:port in client. I have made all  necessary XML Configuration in server side.

Sample Code for client is:

RemoteCacheManager cacheManager = new RemoteCacheManager();

RemoteCache<Integer, String> cache = cacheManager.getCache("myCache");

for(int i = 0 ;i< 10 ;i++)

cache.put(i, "hi"+i);

Sample XML Configuration(clustered.xml) in server Node1:

<cache-container name="clustered" default-cache="default" statistics="true">

                <transport executor="infinispan-transport" lock-timeout="60000"/>

                <distributed-cache name="default" mode="SYNC" segments="20" owners="2" remote-timeout="30000" start="EAGER">

                  <locking isolation="READ_COMMITTED" acquire-timeout="30000" concurrency-level="1000" striping="false"/>

                  <transaction mode="NONE"/>

                </distributed-cache>

                <distributed-cache name="myCache" mode="SYNC" segments="20" owners="2" remote-timeout="30000" start="EAGER">

                 <!-- <locking isolation="READ_COMMITTED" acquire-timeout="3000" concurrency-level="1000" striping="false"/>

                  <transaction mode="NONE"/>   -->

                       <eviction max-entries="5" strategy="LRU" />  

                            <expiration lifespan="-1" max-idle="-1" />

                        <file-store name="myFileStore"

                                      passivation="false"

                                      purge="false"

                                      shared="true"

                                      relative-to="jboss.server.log.dir"

                                      path="file2"

                                      max-entries="10000"

                                      fetch-state="true"

                                      preload="true"

                                      singleton="false"/>

                </distributed-cache>

</cache-container>

When i use cache.size() its returning value lesser than 10,if i set eviction maxentries="5" in XML file at server side.(Note: Its working fine if i remove the eviction tag).

and i tried with cache.keySet().size() and it gives exact value 10. But if one node goes down, all the distributed keys in that particular  node get missed and cache.keySet().size() returns lesser value.

Can some please help me in configuring Hot Rod Remote client server for distributed cache.

[rvansa]

http://infinispan.org/docs/7.0.x/faqs/faqs.html#_cache_s_number_of_entries_never_reaches_configured_maxentries_why_is_that

Beyond that, size() returns number of entries in the target node - and target is in this case random node (from those two). On the other hand, keySet() returns keys from all nodes.

Also, in configuration: you can't have shared file store, that should be private for each server.

[raghav_11]

Thank you.  I got it. Its specified in document too,shared attribute is not recommended for file store.

So in my case, cache.keyset.size() returns all the keys in all nodes in the cluster(in my case it is 10).

But i am setting up a distributed cache with 2 nodes and say 5 keys are present in each node. By adding  owners = 2  attribute in distributed cache, am having two copies of each entry. Hence 10keys will be available in both nodes.

Is there any way to get full size(10) of distributed cache without using keySet().size() since it is an expensive operation when compared to cache.size() when using large number of entries.

Thanks

Raghav.

[rvansa]

Not an easy one - you could connect to each node through JMX and retrieve the local size, and them sum them up.

The reason is that there's no real use-case for getting the size beyond monitoring, and monitoring is done through JMX. Any such number would be just a guess anyway, as the number is changing all the time in live (concurrently accessed) cache. The RemoteCache.size() is implemented just for the sake of Map interface.

[raghav_11]

Okay. I have few other doubts.

1) I would like to know the number of segments recommended for a distributed cache if there are two nodes and if it will affect the performance.( say for 100000 entries). Whats the default number of segments

2)At times when i start the hot grid server with huge load to put and remove, it crashes and aborts saying

# A fatal error has been detected by the Java Runtime Environment:
#
#  SIGSEGV (0xb) at pc=0x00007f95726fec05, pid=28677, tid=140275445909248
#
# JRE version: 6.0_38-b05
# Java VM: Java HotSpot(TM) 64-Bit Server VM (20.13-b02 mixed mode linux-amd64
compressed oops)
# Problematic frame:
# J  org.infinispan.commons.util.concurrent.jdk8backported.LongAdder.add(J)V
#
# An error report file with more information is saved as:
# /root/JBoss/jboss-datagrid-6.2.1-server/bin/hs_err_pid28677.log

Can you tel me the reason why am getting this exception very occasionally??(mostly during startup) . Most of the time server works fine even in huge load.

Thanks for your support.

Raghav.

[rvansa]

1) Default (I believe 40) should work well. Segments affect distribution of keys, but with 2 nodes, both servers hold copy of the same data. However, for half of the segments one node will be the primary owner and for half the other one - and with HotRod, request is targetted at the primary owner, therefore, it may affect load balancing a bit. But it doesn't really matter as long as the number is reasonable ( > 10 && < 1000, ideally divisible by 2)

2) Looks like JVM bug. Upgrade JVM (preferably to latest Java 7 one, OpenJDK would be a good choice).