12 Replies Latest reply on Mar 20, 2012 9:42 PM by Peter Wong

    Problem with Cassandra Cache Store

    Federico Wachs Newbie

      Hi all


      I've been trying effortlessly to try and setup a Cassandra Cache Store.


      This is my XML configuration which I use to start Infinispan:


      <?xml version="1.0" encoding="UTF-8"?>


        ~ JBoss, Home of Professional Open Source

        ~ Copyright 2009 Red Hat Inc. and/or its affiliates and other

        ~ contributors as indicated by the @author tags. All rights reserved.

        ~ See the copyright.txt in the distribution for a full listing of

        ~ individual contributors.


        ~ This is free software; you can redistribute it and/or modify it

        ~ under the terms of the GNU Lesser General Public License as

        ~ published by the Free Software Foundation; either version 2.1 of

        ~ the License, or (at your option) any later version.


        ~ This software is distributed in the hope that it will be useful,

        ~ but WITHOUT ANY WARRANTY; without even the implied warranty of


        ~ Lesser General Public License for more details.


        ~ You should have received a copy of the GNU Lesser General Public

        ~ License along with this software; if not, write to the Free

        ~ Software Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA

        ~ 02110-1301 USA, or see the FSF site: http://www.fsf.org.




            xsi:schemaLocation="urn:infinispan:config:5.1 http://www.infinispan.org/schemas/infinispan-config-5.1.xsd"




            <transport clusterName="demoCluster" distributedSyncTimeout="50000" />

            <globalJmxStatistics enabled="true"/>





            <jmxStatistics enabled="true"/>

            <clustering mode="distribution">

               <l1 enabled="true" lifespan="60000"/>

               <hash numOwners="2" rehashRpcTimeout="120000"/>





                <namedCache name="cassandraCache">

              <loaders passivation="false" shared="true" preload="false">

                              <loader class="org.infinispan.loaders.cassandra.CassandraCacheStore" fetchPersistentState="true" ignoreModifications="false"



                                      <property name="host" value="localhost" />

                                <property name="keySpace" value="Infinispan" />

                                <property name="entryColumnFamily" value="InfinispanEntries" />

                                <property name="expirationColumnFamily" value="InfinispanExpiration" />

                                <property name="sharedKeyspace" value="false" />

                                <property name="readConsistencyLevel" value="ONE" />

                                <property name="writeConsistencyLevel" value="ONE" />

                                <property name="configurationPropertiesFile" value="/Users/admin/Desktop/cassandrapool.properties" />         

                                <property name="keyMapper" value="org.infinispan.loaders.keymappers.DefaultTwoWayKey2StringMapper" />

                                <property name="autoCreateKeyspace" value="true" />








      This is my cassandrapool.properties:


      socketTimeout = 5000

      initialSize = 10

      maxActive = 100

      maxIdle = 20

      minIdle = 10

      maxWait = 30000

      testOnBorrow = false

      testOnReturn = false

      testWhileIdle = false

      timeBetweenEvictionRunsMillis = 5000

      removeAbandoned = false

      removeAbandonedTimeout = 60

      logAbandoned = false


      I'm using Cassandra 1.0.7 .


      I've checked the log files, no errors there. But I checked Infinispan's cassandras module source code and It's not logging what that class does or should be doing.

      It's not even creating the KeySpace on my cassandra db...

      So basically i'm totally lost since the log files are not clear but I do know that nothing's being done in cassandra because the keyspace is not being created.


      This is how I run the server:


      ./bin/startServer.sh -r memcached -c etc/config-samples/custom-cache-config.xml -p 11211


      And I have also moved the jar files from the modules folder to the lib folder in order to infinispan to start.


      Anybody has any idea what could possible be going on?


      Any kind of help would be greatly appreciated!




        • 1. Re: Problem with Cassandra Cache Store
          Federico Wachs Newbie

          So I performed a last test and ran:


          ./bin/startServer.sh -r hotrod -c etc/config-samples/custom-cache-config.xml -p 11211


          And everything worked perfectly. Now, I was wondering if there's a way to have Casandra CacheStore working with a Infinispan Memcached server?


          Thanks in advanced,



          • 2. Re: Problem with Cassandra Cache Store
            Tristan Tarrant Master

            Hi Federico,

            I will be able to look at this on Monday.

            • 3. Re: Problem with Cassandra Cache Store
              Federico Wachs Newbie

              I'm sorry but I just can't use a tool that is hardly supported by its developers and is very bad documented; it's just not serious.

              Thanks for your reply anyway!




              • 4. Re: Problem with Cassandra Cache Store
                Tristan Tarrant Master

                Sorry Federico, I forgot that I had to reply.


                The problem is that the memcached server cannot handle different caches. The cache name has to be called "memcachedCache". Because you don't have a cache with that name defined in the config it just creates a default (in-memory) cache. Just replace <namedCache name="cassandraCache"> with <namedCache name="memcachedCache"> in your file above. I apologize again for not getting back to you sooner. I will make sure the docs are clearer on this fact.

                • 5. Re: Problem with Cassandra Cache Store
                  Peter Wong Newbie

                  Hi all,


                  i am a newbie to cassandra cache store and i am working on configuring JBoss7.1 final (standalone-full-ha.xml)+ infinispan 5.1 (embedded in jboss 7.1 subsystem) + cassandra 1.07. I would like to ask is it possible to use cassandra cache store in the latest JBoss 7.1 infinispan. As the cache container configuration in the infinspan subsystem  under  JBoss 7.1 doesnt support the <loader class>


                  <subsystem xmlns="urn:jboss:domain:infinispan:1.1" default-cache-container="cluster">

                  <cache-container name="CassandraCache" default-cache="dist" jndi-name="java:jboss/infinispan/CassandraCache">

                  <transport lock-timeout="60000"/>

                  <replicated-cache name="repl" mode="ASYNC" batching="true">

                  <eviction strategy="LRU"/>



                  <replicated-cache name="remote-connector-client-mappings" mode="SYNC" batching="true"/>

                  <distributed-cache name="dist" mode="ASYNC" batching="true">

                  <eviction strategy="LRU"/>






                  Any one know how to configure the cassandra cache store in the Infinispan under JBoss 7.1 .


                  Thanks for your help.

                  • 6. Re: Problem with Cassandra Cache Store
                    Tristan Tarrant Master

                    Hi Pete, I wouldn't recommend using the JBoss's Infinispan subsystem to configure a cache with a cache store which is not part of the AS distribution. It's much easier to deploy an app with the appropriate dependencies and build it programmatically or using the declarative Infinispan XML configuration.


                    However, if you're feeling adventurous, I have attached a set of modules which you need to unpack over an AS 7.1.1 installation which will help you get started.


                    You then need to add the following to your cache declaration:


                    <store class="org.infinispan.loader.cassandra.CassandraCacheStore">

                    <property name="...">...</property>



                    • 7. Re: Problem with Cassandra Cache Store
                      Peter Wong Newbie

                      Hi Tristan,


                      Thanks for your help.


                      Sorry to bother you again. I have unpack the module folder over an AS 7.1 and i made some change on the module.xml under jboss/as/clustering/infinispan/main which your xml has declared  a SNAPSHOT version.


                      <resource-root path="jboss-as-clustering-infinispan-7.1.1.Final.jar"/>



                      i have declared a cache like this :

                      <cache-container name="CassandraCache" aliases="CassandraCache" default-cache="cache-store" jndi-name="java:jboss/infinispan/CassandraCache">

                                      <transport lock-timeout="60000"/>

                                      <local-cache name="cache-store">

                                          <store class="org.infinispan.loader.cassandra.CassandraCacheStore">



                                      <replicated-cache name="repl" mode="ASYNC" batching="true">

                                          <eviction strategy="LRU"/>



                                      <replicated-cache name="remote-connector-client-mappings" mode="SYNC" batching="true"/>

                                      <distributed-cache name="dist" mode="ASYNC" batching="true">

                                          <eviction strategy="LRU"/>




                      i got the error message from cache configuration , i have declared a cache like this :


                      14:19:24,344 ERROR [org.jboss.as.controller.management-operation] (ServerService Thread Pool -- 39) JBAS014612: Operation ("add") failed - address: ([

                          ("subsystem" => "infinispan"),

                          ("cache-container" => "CassandraCache"),

                          ("local-cache" => "cache-store")

                      ]): java.lang.IllegalArgumentException: org.infinispan.loader.cassandra.CassandraCacheStore is not a valid cache store

                          at org.jboss.as.clustering.infinispan.subsystem.CacheAdd.buildCacheStore(CacheAdd.java:545)

                          at org.jboss.as.clustering.infinispan.subsystem.CacheAdd.processModelNode(CacheAdd.java:426)

                          at org.jboss.as.clustering.infinispan.subsystem.CacheAdd.performRuntime(CacheAdd.java:165)

                          at org.jboss.as.controller.AbstractAddStepHandler$1.execute(AbstractAddStepHandler.java:50) [jboss-as-controller-7.1.1.Final.jar:7.1.1.Final]

                          at org.jboss.as.controller.AbstractOperationContext.executeStep(AbstractOperationContext.java:385) [jboss-as-controller-7.1.1.Final.jar:7.1.1.Final]

                          at org.jboss.as.controller.AbstractOperationContext.doCompleteStep(AbstractOperationContext.java:272) [jboss-as-controller-7.1.1.Final.jar:7.1.1.Final]

                          at org.jboss.as.controller.AbstractOperationContext.completeStep(AbstractOperationContext.java:200) [jboss-as-controller-7.1.1.Final.jar:7.1.1.Final]

                          at org.jboss.as.controller.ParallelBootOperationStepHandler$ParallelBootTask.run(ParallelBootOperationStepHandler.java:311) [jboss-as-controller-7.1.1.Final.jar:7.1.1.Final]

                          at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) [rt.jar:1.6.0_29]

                          at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) [rt.jar:1.6.0_29]

                          at java.lang.Thread.run(Thread.java:662) [rt.jar:1.6.0_29]

                          at org.jboss.threads.JBossThread.run(JBossThread.java:122) [jboss-threads-2.0.0.GA.jar:2.0.0.GA]

                      Caused by: java.lang.ClassNotFoundException: org.infinispan.loader.cassandra.CassandraCacheStore from [Module "org.infinispan:main" from local module loader @ee22f7 (roots: C:\Java\jboss-as-7.1.1.Final\modules)]

                          at org.jboss.modules.ModuleClassLoader.findClass(ModuleClassLoader.java:190) [jboss-modules.jar:1.1.1.GA]

                          at org.jboss.modules.ConcurrentClassLoader.performLoadClassUnchecked(ConcurrentClassLoader.java:468) [jboss-modules.jar:1.1.1.GA]

                          at org.jboss.modules.ConcurrentClassLoader.performLoadClassChecked(ConcurrentClassLoader.java:456) [jboss-modules.jar:1.1.1.GA]

                          at org.jboss.modules.ConcurrentClassLoader.performLoadClass(ConcurrentClassLoader.java:398) [jboss-modules.jar:1.1.1.GA]

                          at org.jboss.modules.ConcurrentClassLoader.loadClass(ConcurrentClassLoader.java:120) [jboss-modules.jar:1.1.1.GA]

                          at org.jboss.as.clustering.infinispan.subsystem.CacheAdd.buildCacheStore(CacheAdd.java:542)

                          ... 11 more



                      Am i going wrong direction? as i have to use the distributed infinispan cache for Cassandra in stead of using cache initializer when failover, It will definitely much more convenient to use a  JBoss's Infinispan subsystem rather than deploying an app with standalone infinispan.

                      • 8. Re: Problem with Cassandra Cache Store
                        Tristan Tarrant Master

                        I forgot to add the updated modules/org/infinispan/main/module.xml to the archive I uploaded. Just add the cachestore module to the list of dependencies in that file


                        <module name="org.infinispan.cachestore.cassandra" />


                        java.lang.ClassNotFoundException: org.infinispan.loader.cassandra.CassandraCacheStore from [Module "org.infinispan:main"...

                        • 9. Re: Problem with Cassandra Cache Store
                          Peter Wong Newbie

                          Hi Tristan,


                          I tried to add <module name="org.infinispan.cachestore.cassandra" /> to the list of dependencies. But it still doesnt work .


                          Moreover, i got the same error when tried to use the jdbc cachestore for testing purpose . I reed some comment about the cache store config https://bugzilla.redhat.com/show_bug.cgi?id=745890.


                          Any idea?


                          Thanks you for kindly reply



                          • 10. Re: Problem with Cassandra Cache Store
                            Galder Zamarreño Master

                            Hmmm, are you sure you got the same error? Please show the first exception that you get, plus the contents of modules/org/infinispan/main/module.xml and modules/org/infinispan/cachestore/cassandra/main/module.xml


                            Please also do an ls/dir of modules/org/infinispan/main and modules/org/infinispan/cachestore/cassandra/main directories.

                            • 11. Re: Problem with Cassandra Cache Store
                              Tristan Tarrant Master

                              Hi Peter,


                              I found the mistake:


                              it's not org.infinispan.loader.cassandra.CassandraCacheStore but org.infinispan.loaders.cassandra.CassandraCacheStore


                              Simple thing to get lost in the noise



                              • 12. Re: Problem with Cassandra Cache Store
                                Peter Wong Newbie

                                Hi Tristan and Galder,


                                Thanks for your generous help. I will keep trying.