Infinispan randomly miss entries

[ks.zealot]

I use infinispan, it was deployed as jboss service at jboss eap 4. Infinispan working over jgroups in 2-node cluster. Cache was  configured  as jndi resource with infinity lifespan.

<replicated-cache name="Profiles" mode="SYNC" start="EAGER" batching="true">

                    <locking isolation="REPEATABLE_READ"/>

                </replicated-cache>

And one of my Cache randomly drop entries.

19:13:43,467 DEBUG [org.infinispan.jmx.ResourceDMBean] (RMI TCP Connection(25573)-172.16.100.35) Attribute numberOfEntries has r=true,w=false,is=false and value 14

19:14:13,584 DEBUG [org.infinispan.jmx.ResourceDMBean] (RMI TCP Connection(25575)-172.16.100.35) Attribute numberOfEntries has r=true,w=false,is=false and value 14

19:14:43,150 DEBUG [org.infinispan.jmx.ResourceDMBean] (RMI TCP Connection(25575)-172.16.100.35) Attribute numberOfEntries has r=true,w=false,is=false and value 14

19:15:13,268 DEBUG [org.infinispan.jmx.ResourceDMBean] (RMI TCP Connection(25575)-172.16.100.35) Attribute numberOfEntries has r=true,w=false,is=false and value 14

19:15:43,391 DEBUG [org.infinispan.jmx.ResourceDMBean] (RMI TCP Connection(25576)-172.16.100.35) Attribute numberOfEntries has r=true,w=false,is=false and value 13

19:16:13,523 DEBUG [org.infinispan.jmx.ResourceDMBean] (RMI TCP Connection(25576)-172.16.100.35) Attribute numberOfEntries has r=true,w=false,is=false and value 13

19:16:43,392 DEBUG [org.infinispan.jmx.ResourceDMBean] (RMI TCP Connection(25579)-172.16.100.35) Attribute numberOfEntries has r=true,w=false,is=false and value 13

19:17:13,549 DEBUG [org.infinispan.jmx.ResourceDMBean] (RMI TCP Connection(25579)-172.16.100.35) Attribute numberOfEntries has r=true,w=false,is=false and value 13

19:17:43,829 DEBUG [org.infinispan.jmx.ResourceDMBean] (RMI TCP Connection(25581)-172.16.100.35) Attribute numberOfEntries has r=true,w=false,is=false and value 13

19:18:13,888 DEBUG [org.infinispan.jmx.ResourceDMBean] (RMI TCP Connection(25581)-172.16.100.35) Attribute numberOfEntries has r=true,w=false,is=false and value 13

19:18:43,827 DEBUG [org.infinispan.jmx.ResourceDMBean] (RMI TCP Connection(25584)-172.16.100.35) Attribute numberOfEntries has r=true,w=false,is=false and value 13

19:19:13,020 DEBUG [org.infinispan.jmx.ResourceDMBean] (RMI TCP Connection(25584)-172.16.100.35) Attribute numberOfEntries has r=true,w=false,is=false and value 13

19:19:43,989 DEBUG [org.infinispan.jmx.ResourceDMBean] (RMI TCP Connection(25586)-172.16.100.35) Attribute numberOfEntries has r=true,w=false,is=false and value 13

How could I fix that?

[wdfink]

How do you fill and access the cache ?

Do you see missing entries if you use the cache?

Maybe setting log level DEBUG or TRACE for org.infinispan will show more details.

[ks.zealot]

Filling cache happens after appilcation start.

if ( Cache == null) {

            Cache = CacheHolder.getCache(CF_NAME);

            List  profiles = list();

            OperatorManager om = OperatorManager.getInstance();

            for (Operator o : om.list()) {

                List l = new ArrayList();

                for (Profiles p : profiles) {

                    if (p.getOperatorId().equals(o.getId())) {

                        l.add(p);

                    }

                }

                profileCache.put(o.getId(), l);

            }

        }

Getting entries from cache by id

public List<Profiles> cachedList(String operatorId) {      

            List list = profileCache.get(operatorId);

            return list;

    }

I couldnot see   entries if they are dropped.

I turn on TRACE level log, but log are too huge. grep by "clear" or "remove" did not return anything

[wdfink]

Not sure what you are doing.

Is it possible to extract a small reproducer and attach it?

[ks.zealot]

No, cache filled from remote database.

Could cache had   fail    on too huge commit?

[wdfink]

Sorry, I did not understand what you are doing or what might be wrong.

In general the cache did not fail with huge commits, but if this is the case you will see a error message.

So an example might be helpful to show what you do.

Also you can enable the org.infinispan logging to TRACE, this might show you after adding the entries whether there is some action which remove it.

[ks.zealot]

I already enabled TRACE level. Every time when I could not retrieve entries from cache, infinispan made huge commit (Log string ~ 700kb).