Probably bug in configuration parser (<persistence/> element)

[aolenev]

Hi,

I'm using Infinispan 6.0.2.Final and trying to write my own CacheStore implementation and configure it to work in async mode (write-behind). According to documentation page I have to add element

<persistence>
    <store blah-blah>
        <async enabled="true"/>
        <properties/>
    </store>
</persistence>

I wrote my CacheStore implementation (implements AdvancedLoadWriteStore) but when I add any child element of <store/> I get an exception

"Unexpected element '{urn:infinispan:config:6.0}properties'"

I tried to find the root cause and found this code in org.infinispan.configuration.parsing.Parser60 (lines 668-685):

if (store != null) {
    if (store instanceof SingleFileStore) {
        SingleFileStoreConfigurationBuilder sfs = builder.persistence().addSingleFileStore();
        if (fetchPersistentState != null)
           sfs.fetchPersistentState(fetchPersistentState);
        if (ignoreModifications != null)
           sfs.ignoreModifications(ignoreModifications);
        if (purgeOnStartup != null)
           sfs.purgeOnStartup(purgeOnStartup);
        if (preload != null)
           sfs.preload(preload);
        if (shared != null)
           sfs.shared(shared);
        parseStoreChildren(reader, sfs);
    } else if (store instanceof ClusterLoader) {
        ClusterLoaderConfigurationBuilder cscb = builder.persistence().addClusterLoader();
         parseLoaderChildren(reader, cscb);
    }
}

It means that if my store is not a subclass of SingleFileStore then I can't use <async/>, <properties/> and <singleton/> elements.

Is it a bug? If not, how can I write my CacheStore implementation configured for write-behind?

Thank you in advance.

BTW, I can't find the way to enable code highlighting in this online discussion editor.

Syntax highlighting is fixed

[vbchin2]

Addressing concerns in reverse

Choose 'Use advanced editor' to your top-right hand corner of your comment window for Syntax highlighting and other stuff. If already in that mode, click >> in the editor menu for syntax highlighting.

Coming to your original question, if you are good are following XSD schema documents, I would strongly recommend following the file mentioned below when putting together the <persistence> element.

http://www.jboss.org/schema/infinispan/infinispan-config-6.0.xsd

Now, the error is very clearly stated. Based on how you have put together the XML, it is saying that it wasn't expecting <properties> element.

If you think you are doing everything as suggested, then I recommend pasting the original and full XML file content as is without any blah-blah's , this helps us understand where you might be going wrong.

[aolenev]

First of all thank you for your hint about editor:)

Yes, I saw this XSD, and as I can see element named "persistence" contains element named "store" which is of type "tns:customStore" which in its turn of abstract type "tns:store"

And store definition is the following:

<xs:complexType name="store" abstract="true">
     <xs:sequence>
          <xs:element name="async" type="tns:async" minOccurs="0" maxOccurs="1">
               <xs:annotation>
                    <xs:documentation>Configuration for the async cache loader. If enabled, this provides you with asynchronous writes to the cache store, giving you 'write-behind' caching.</xs:documentation>
               </xs:annotation>
          </xs:element>
          <xs:element name="singleton" type="tns:singletonStore" minOccurs="0" maxOccurs="1">
               <xs:annotation>
                    <xs:documentation>SingletonStore is a delegating cache store used for situations when only one instance in a cluster should interact with the underlying store. The coordinator of the cluster will be responsible for the underlying CacheStore. SingletonStore is a simply facade to a real CacheStore implementation. It always delegates reads to the real CacheStore.</xs:documentation>
               </xs:annotation>
          </xs:element>
          <xs:element name="properties" type="tns:properties" minOccurs="0" maxOccurs="1">
               <xs:annotation>
                    <xs:documentation>Properties passed to the cache store or writer</xs:documentation>
               </xs:annotation>
          </xs:element>
     </xs:sequence>
...
</xs:complexType>

So from my POV it means that <properties /> is allowed element. Am I missed something?

[galder.zamarreno]

I wonder if the issue is that "properties" is empty? If not providing any properties, can you try removing <properties/> altogether?

[aolenev]

No, <properties /> element is not empty. If my example of xml configuration is confusing a little, here is snippet from my real configuration file:

<namedCache name="string-cache">
    <clustering mode="distribution">
        <sync />
        <stateTransfer fetchInMemoryState="true" />
        <hash numOwners="2" />
    </clustering>
    <persistence>
        <store class="org.sproot_grid.infinispan.cachestore.StringCacheStore"
        fetchPersistentState="false" ignoreModifications="false" purgeOnStartup="false">
            <properties>
                <property name="connectionUrl" value="jdbc:mysql://localhost:3306/test" />
                <property name="driverClass" value="com.mysql.jdbc.Driver" />
            </properties>
        </store>
    </persistence>
</namedCache>

But anyway parser throws exception before parsing internals of <properties />.

[vbchin2]

Alexey,

That snippet is still not the full XML file. It would be good if you pasted the entire Infinispan.xml file and a proper stack trace for us to help further.

[aolenev]

Vijay, thank you for your help, here is the config:

<infinispan xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    xsi:schemaLocation="urn:infinispan:config:6.0 http://www.infinispan.org/schemas/infinispan-config-6.0.xsd"
    xmlns="urn:infinispan:config:6.0">

    <global>
        <transport
            transportClass="org.infinispan.remoting.transport.jgroups.JGroupsTransport"
            clusterName="Sproot">
            <properties>
                <property name="configurationFile" value="jgroups-udp.xml" />
            </properties>
        </transport>
    </global>

    <namedCache name="string-cache">
        <clustering mode="distribution">
            <sync />
            <stateTransfer fetchInMemoryState="true" />
            <hash numOwners="2" />
        </clustering>
        <persistence>
            <store class="org.sproot_grid.infinispan.cachestore.StringCacheStore"
            fetchPersistentState="false" ignoreModifications="false" purgeOnStartup="false">
                <properties>
                    <property name="connectionUrl" value="jdbc:mysql://localhost:3306/test" />
                    <property name="driverClass" value="com.mysql.jdbc.Driver" />
                </properties>
            </store>
        </persistence>
    </namedCache>

</infinispan>

Here is the stack trace:

org.infinispan.commons.CacheConfigurationException: javax.xml.stream.XMLStreamException: ParseError at [row,col]:[104,29]
Message: Unexpected element '{urn:infinispan:config:6.0}properties'
    at org.infinispan.configuration.parsing.ParserRegistry.parse(ParserRegistry.java:102)
    at org.infinispan.manager.DefaultCacheManager.<init>(DefaultCacheManager.java:253)
    at org.infinispan.manager.DefaultCacheManager.<init>(DefaultCacheManager.java:226)
    at org.infinispan.manager.DefaultCacheManager.<init>(DefaultCacheManager.java:213)
    at org.sproot_grid.cache.ClusterFacade.init(ClusterFacade.java:48)
    at org.sproot_grid.cache.ClusterFacade.getFacade(ClusterFacade.java:41)
    at org.sproot_grid.cache.ClusterFacadeTest.initCluster(ClusterFacadeTest.java:21)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:601)
    at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:44)
    at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15)
    at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:41)
    at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:27)
    at org.junit.runners.ParentRunner.run(ParentRunner.java:236)
    at org.eclipse.jdt.internal.junit4.runner.JUnit4TestReference.run(JUnit4TestReference.java:50)
    at org.eclipse.jdt.internal.junit.runner.TestExecution.run(TestExecution.java:38)
    at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:467)
    at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:683)
    at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.run(RemoteTestRunner.java:390)
    at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.main(RemoteTestRunner.java:197)
Caused by: javax.xml.stream.XMLStreamException: ParseError at [row,col]:[104,29]
Message: Unexpected element '{urn:infinispan:config:6.0}properties'
    at org.infinispan.configuration.parsing.ParserRegistry.parseElement(ParserRegistry.java:139)
    at org.infinispan.configuration.parsing.XMLExtendedStreamReaderImpl.handleAny(XMLExtendedStreamReaderImpl.java:37)
    at org.infinispan.configuration.parsing.Parser60.parsePersistence(Parser60.java:558)
    at org.infinispan.configuration.parsing.Parser60.parseCache(Parser60.java:150)
    at org.infinispan.configuration.parsing.Parser60.parseNamedCache(Parser60.java:109)
    at org.infinispan.configuration.parsing.Parser60.readElement(Parser60.java:76)
    at org.infinispan.configuration.parsing.ParserRegistry.parseElement(ParserRegistry.java:141)
    at org.infinispan.configuration.parsing.ParserRegistry.parse(ParserRegistry.java:123)
    at org.infinispan.configuration.parsing.ParserRegistry.parse(ParserRegistry.java:110)
    at org.infinispan.configuration.parsing.ParserRegistry.parse(ParserRegistry.java:97)
    ... 21 more

[vbchin2]

Did you trim the infinispan.xml file ? The parse error points to line #104 at column #29 and your file isn't that big to begin with. 

[aolenev]

Yes, I removed declarations of other caches, but these caches don't have <persistence /> element. Line 104 is the line with <properties /> element.

<infinispan xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    xsi:schemaLocation="urn:infinispan:config:6.0 http://www.infinispan.org/schemas/infinispan-config-6.0.xsd"
    xmlns="urn:infinispan:config:6.0">

    <global>
        <transport
            transportClass="org.infinispan.remoting.transport.jgroups.JGroupsTransport"
            clusterName="Sproot">
            <properties>
                <property name="configurationFile" value="jgroups-udp.xml" />
            </properties>
        </transport>
    </global>

    <namedCache name="service-cache">
        <clustering mode="distribution">
            <sync />
            <stateTransfer fetchInMemoryState="true" />
            <hash numOwners="2" />
        </clustering>
    </namedCache>

    <namedCache name="map-branch-cache">
        <clustering mode="distribution">
            <sync />
            <stateTransfer fetchInMemoryState="true" />
            <hash numOwners="2" />
        </clustering>
    </namedCache>

    <namedCache name="map-string-cache">
        <clustering mode="distribution">
            <sync />
            <stateTransfer fetchInMemoryState="true" />
            <hash numOwners="2" />
        </clustering>
    </namedCache>

    <namedCache name="account-cache">
        <clustering mode="distribution">
            <sync />
            <stateTransfer fetchInMemoryState="true" />
            <hash numOwners="2" />
        </clustering>
    </namedCache>

    <namedCache name="list-branch-cache">
        <clustering mode="distribution">
            <sync />
            <stateTransfer fetchInMemoryState="true" />
            <hash numOwners="2" />
        </clustering>
    </namedCache>

    <namedCache name="list-cache">
        <clustering mode="distribution">
            <sync />
            <stateTransfer fetchInMemoryState="true" />
            <hash numOwners="2" />
        </clustering>
    </namedCache>

    <namedCache name="tree-cache">
        <clustering mode="distribution">
            <sync />
            <stateTransfer fetchInMemoryState="true" />
            <hash numOwners="2" />
        </clustering>
    </namedCache>

    <namedCache name="user-cache">
        <indexing enabled="true" indexLocalOnly="true">
            <properties>
                <property name="default.directory_provider" value="ram" />
            </properties>
        </indexing>

        <clustering mode="distribution">
            <sync />
            <stateTransfer fetchInMemoryState="true" />
            <hash numOwners="2" />
        </clustering>
        <eviction maxEntries="1000" />
        <expiration lifespan="2000" maxIdle="5000" wakeUpInterval="5000" />
    </namedCache>

    <namedCache name="map-cache">
        <clustering mode="distribution">
            <sync />
            <stateTransfer fetchInMemoryState="true" />
            <hash numOwners="2" />
        </clustering>
    </namedCache>

    <namedCache name="string-cache">
        <clustering mode="distribution">
            <sync />
            <stateTransfer fetchInMemoryState="true" />
            <hash numOwners="2" />
        </clustering>
        <persistence>
            <store class="org.sproot_grid.infinispan.cachestore.StringCacheStore"
            fetchPersistentState="false" ignoreModifications="false" purgeOnStartup="false">
                <properties>
                    <property name="connectionUrl" value="jdbc:mysql://localhost:3306/test" />
                    <property name="driverClass" value="com.mysql.jdbc.Driver" />
                </properties>
            </store>
        </persistence>
    </namedCache>

</infinispan>

[aolenev]

Moreover, even write-through without <properties /> doesn't work because it doesn't store <persistence /> configuration in case I initialize cluster using constructor DefaultCacheManager(String configurationFile). Here is stack trace I took in debugger. The top method (Parser60.parseStore) is responsible for parsing <store /> element. As I see from code it parses element, but it doesn't store configuration if my CacheStore is not an instance of SingleFileStore. So for it looks like xml configuration of read-through/write-through/write-behind is broken in Infinispan 6.x

Thread [main] (Suspended (breakpoint at line 640 in Parser60))    
    Parser60.parseStore(XMLExtendedStreamReader, ConfigurationBuilderHolder) line: 640    
    Parser60.parsePersistence(XMLExtendedStreamReader, ConfigurationBuilderHolder) line: 555    
    Parser60.parseCache(XMLExtendedStreamReader, ConfigurationBuilderHolder) line: 150    
    Parser60.parseNamedCache(XMLExtendedStreamReader, ConfigurationBuilderHolder) line: 109    
    Parser60.readElement(XMLExtendedStreamReader, ConfigurationBuilderHolder) line: 76    
    ParserRegistry.parseElement(XMLExtendedStreamReader, ConfigurationBuilderHolder) line: 141    
    ParserRegistry.parse(XMLExtendedStreamReader, ConfigurationBuilderHolder) line: 123    
    ParserRegistry.parse(InputStream, ConfigurationBuilderHolder) line: 110    
    ParserRegistry.parse(InputStream) line: 97    
    DefaultCacheManager.<init>(InputStream, boolean) line: 253    
    DefaultCacheManager.<init>(String, boolean) line: 226    
    DefaultCacheManager.<init>(String) line: 213    
    ClusterFacade.init() line: 48    
    ClusterFacade.getFacade() line: 41    
    ClusterFacadeTest.initCluster() line: 21    
    NativeMethodAccessorImpl.invoke0(Method, Object, Object[]) line: not available [native method]    
    NativeMethodAccessorImpl.invoke(Object, Object[]) line: 57    
    DelegatingMethodAccessorImpl.invoke(Object, Object[]) line: 43    
    Method.invoke(Object, Object...) line: 601    
    FrameworkMethod$1.runReflectiveCall() line: 44    
    FrameworkMethod$1(ReflectiveCallable).run() line: 15    
    FrameworkMethod.invokeExplosively(Object, Object...) line: 41    
    RunBefores.evaluate() line: 27    
    BlockJUnit4ClassRunner(ParentRunner<T>).run(RunNotifier) line: 236    
    JUnit4TestMethodReference(JUnit4TestReference).run(TestExecution) line: 50    
    TestExecution.run(ITestReference[]) line: 38    
    RemoteTestRunner.runTests(String[], String, TestExecution) line: 467    
    RemoteTestRunner.runTests(TestExecution) line: 683    
    RemoteTestRunner.run() line: 390    
    RemoteTestRunner.main(String[]) line: 197

[aolenev]

I raised two bugs about it:

ISPN-4348

ISPN-4349

[aolenev]

Ok guys, I'm giving up. Could you tell me how to configure my own custom store?

[nadirx]

Alexey, this is fixed in the current 7.0 series.

[aolenev]

Thank you, Tristan! Do you have any information about date of 7.0.0.Final release?