javax.jcr.RepositoryException: org.modeshape.jcr.index.local.LocalIndexException while defining an index

[bes82]

Hi there,

this is my config, I get the following StackTrace:

{code}

2014-08-25 15:19:45,710 INFO  [MSC service thread 1-7] stdout: javax.jcr.RepositoryException: org.modeshape.jcr.index.local.LocalIndexException: The local provider does not support multi-column indexes

2014-08-25 15:19:45,710 INFO  [MSC service thread 1-7] stdout: at org.modeshape.jcr.JcrRepositoriesContainer$UrlRepositoryLookup.repository(JcrRepositoriesContainer.java:223)

2014-08-25 15:19:45,710 INFO  [MSC service thread 1-7] stdout: at org.modeshape.jcr.JcrRepositoriesContainer.getRepository(JcrRepositoriesContainer.java:90)

2014-08-25 15:19:45,711 INFO  [MSC service thread 1-7] stdout: at org.modeshape.jcr.JcrRepositoryFactory.getRepository(JcrRepositoryFactory.java:123)

2014-08-25 15:19:45,711 INFO  [MSC service thread 1-7] stdout: at org.erratic.cr.repository.repositoryProvider.RepositoryProvider.contextInitialized(RepositoryProvider.java:55)

2014-08-25 15:19:45,711 INFO  [MSC service thread 1-7] stdout: at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

2014-08-25 15:19:45,711 INFO  [MSC service thread 1-7] stdout: at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)

2014-08-25 15:19:45,711 INFO  [MSC service thread 1-7] stdout: at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

2014-08-25 15:19:45,711 INFO  [MSC service thread 1-7] stdout: at java.lang.reflect.Method.invoke(Method.java:606)

2014-08-25 15:19:45,711 INFO  [MSC service thread 1-7] stdout: at org.jboss.weld.injection.producer.DefaultLifecycleCallbackInvoker.invokeMethods(DefaultLifecycleCallbackInvoker.java:89)

2014-08-25 15:19:45,712 INFO  [MSC service thread 1-7] stdout: at org.jboss.weld.injection.producer.DefaultLifecycleCallbackInvoker.postConstruct(DefaultLifecycleCallbackInvoker.java:72)

2014-08-25 15:19:45,712 INFO  [MSC service thread 1-7] stdout: at org.jboss.weld.injection.producer.BasicInjectionTarget.postConstruct(BasicInjectionTarget.java:95)

2014-08-25 15:19:45,712 INFO  [MSC service thread 1-7] stdout: at org.jboss.weld.injection.producer.BeanInjectionTarget.postConstruct(BeanInjectionTarget.java:63)

2014-08-25 15:19:45,712 INFO  [MSC service thread 1-7] stdout: at org.jboss.weld.bean.ManagedBean.create(ManagedBean.java:162)

2014-08-25 15:19:45,712 INFO  [MSC service thread 1-7] stdout: at org.jboss.weld.context.AbstractContext.get(AbstractContext.java:96)

2014-08-25 15:19:45,712 INFO  [MSC service thread 1-7] stdout: at org.jboss.weld.bean.proxy.ContextBeanInstance.getInstance(ContextBeanInstance.java:98)

2014-08-25 15:19:45,712 INFO  [MSC service thread 1-7] stdout: at org.jboss.weld.bean.proxy.ProxyMethodHandler.invoke(ProxyMethodHandler.java:78)

2014-08-25 15:19:45,713 INFO  [MSC service thread 1-7] stdout: at org.erratic.cr.repository.repositoryProvider.RepositoryProvider$Proxy$_$$_WeldClientProxy.getRepo(Unknown Source)

2014-08-25 15:19:45,713 INFO  [MSC service thread 1-7] stdout: at org.erratic.cr.transactional.access.SessionProducer.getSession(SessionProducer.java:71)

2014-08-25 15:19:45,713 INFO  [MSC service thread 1-7] stdout: at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

2014-08-25 15:19:45,713 INFO  [MSC service thread 1-7] stdout: at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)

2014-08-25 15:19:45,713 INFO  [MSC service thread 1-7] stdout: at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

2014-08-25 15:19:45,713 INFO  [MSC service thread 1-7] stdout: at java.lang.reflect.Method.invoke(Method.java:606)

2014-08-25 15:19:45,713 INFO  [MSC service thread 1-7] stdout: at org.jboss.weld.injection.MethodInjectionPoint.invokeOnInstanceWithSpecialValue(MethodInjectionPoint.java:93)

2014-08-25 15:19:45,713 INFO  [MSC service thread 1-7] stdout: at org.jboss.weld.injection.MethodInjectionPoint.invokeOnInstance(MethodInjectionPoint.java:86)

2014-08-25 15:19:45,714 INFO  [MSC service thread 1-7] stdout: at org.jboss.weld.injection.producer.ProducerMethodProducer.produce(ProducerMethodProducer.java:96)

2014-08-25 15:19:45,714 INFO  [MSC service thread 1-7] stdout: at org.jboss.weld.injection.producer.AbstractMemberProducer.produce(AbstractMemberProducer.java:151)

2014-08-25 15:19:45,714 INFO  [MSC service thread 1-7] stdout: at org.jboss.weld.bean.AbstractProducerBean.create(AbstractProducerBean.java:183)

2014-08-25 15:19:45,714 INFO  [MSC service thread 1-7] stdout: at org.jboss.weld.context.unbound.DependentContextImpl.get(DependentContextImpl.java:69)

2014-08-25 15:19:45,714 INFO  [MSC service thread 1-7] stdout: at org.jboss.weld.manager.BeanManagerImpl.getReference(BeanManagerImpl.java:733)

2014-08-25 15:19:45,714 INFO  [MSC service thread 1-7] stdout: at org.jboss.weld.manager.BeanManagerImpl.getReference(BeanManagerImpl.java:789)

2014-08-25 15:19:45,714 INFO  [MSC service thread 1-7] stdout: at org.jboss.weld.injection.FieldInjectionPoint.inject(FieldInjectionPoint.java:92)

2014-08-25 15:19:45,715 INFO  [MSC service thread 1-7] stdout: at org.jboss.weld.util.Beans.injectBoundFields(Beans.java:358)

2014-08-25 15:19:45,715 INFO  [MSC service thread 1-7] stdout: at org.jboss.weld.util.Beans.injectFieldsAndInitializers(Beans.java:369)

2014-08-25 15:19:45,715 INFO  [MSC service thread 1-7] stdout: at org.jboss.weld.injection.producer.DefaultInjector.inject(DefaultInjector.java:72)

2014-08-25 15:19:45,715 INFO  [MSC service thread 1-7] stdout: at org.jboss.weld.injection.producer.ResourceInjector.inject(ResourceInjector.java:60)

2014-08-25 15:19:45,715 INFO  [MSC service thread 1-7] stdout: at org.jboss.weld.injection.producer.DefaultInjector$1.proceed(DefaultInjector.java:66)

2014-08-25 15:19:45,715 INFO  [MSC service thread 1-7] stdout: at org.jboss.weld.injection.InjectionContextImpl.run(InjectionContextImpl.java:48)

2014-08-25 15:19:45,716 INFO  [MSC service thread 1-7] stdout: at org.jboss.weld.injection.producer.DefaultInjector.inject(DefaultInjector.java:64)

2014-08-25 15:19:45,716 INFO  [MSC service thread 1-7] stdout: at org.jboss.weld.injection.producer.BasicInjectionTarget.inject(BasicInjectionTarget.java:90)

2014-08-25 15:19:45,716 INFO  [MSC service thread 1-7] stdout: at org.jboss.weld.bean.ManagedBean.create(ManagedBean.java:150)

2014-08-25 15:19:45,716 INFO  [MSC service thread 1-7] stdout: at org.jboss.weld.context.unbound.DependentContextImpl.get(DependentContextImpl.java:69)

2014-08-25 15:19:45,716 INFO  [MSC service thread 1-7] stdout: at org.jboss.weld.manager.BeanManagerImpl.getReference(BeanManagerImpl.java:733)

2014-08-25 15:19:45,716 INFO  [MSC service thread 1-7] stdout: at org.jboss.weld.manager.BeanManagerImpl.getReference(BeanManagerImpl.java:789)

2014-08-25 15:19:45,716 INFO  [MSC service thread 1-7] stdout: at org.jboss.weld.injection.FieldInjectionPoint.inject(FieldInjectionPoint.java:92)

2014-08-25 15:19:45,716 INFO  [MSC service thread 1-7] stdout: at org.jboss.weld.util.Beans.injectBoundFields(Beans.java:358)

2014-08-25 15:19:45,717 INFO  [MSC service thread 1-7] stdout: at org.jboss.weld.util.Beans.injectFieldsAndInitializers(Beans.java:369)

2014-08-25 15:19:45,717 INFO  [MSC service thread 1-7] stdout: at org.jboss.weld.injection.producer.DefaultInjector.inject(DefaultInjector.java:72)

2014-08-25 15:19:45,717 INFO  [MSC service thread 1-7] stdout: at org.jboss.weld.injection.producer.ResourceInjector.inject(ResourceInjector.java:60)

2014-08-25 15:19:45,717 INFO  [MSC service thread 1-7] stdout: at org.jboss.weld.injection.producer.DefaultInjector$1.proceed(DefaultInjector.java:66)

2014-08-25 15:19:45,717 INFO  [MSC service thread 1-7] stdout: at org.jboss.weld.injection.InjectionContextImpl.run(InjectionContextImpl.java:48)

2014-08-25 15:19:45,717 INFO  [MSC service thread 1-7] stdout: at org.jboss.weld.injection.producer.DefaultInjector.inject(DefaultInjector.java:64)

2014-08-25 15:19:45,717 INFO  [MSC service thread 1-7] stdout: at org.jboss.weld.injection.producer.BasicInjectionTarget.inject(BasicInjectionTarget.java:90)

2014-08-25 15:19:45,718 INFO  [MSC service thread 1-7] stdout: at org.jboss.weld.bean.ManagedBean.create(ManagedBean.java:150)

2014-08-25 15:19:45,718 INFO  [MSC service thread 1-7] stdout: at org.jboss.weld.context.unbound.DependentContextImpl.get(DependentContextImpl.java:69)

2014-08-25 15:19:45,718 INFO  [MSC service thread 1-7] stdout: at org.jboss.weld.event.ObserverMethodImpl.getReceiver(ObserverMethodImpl.java:302)

2014-08-25 15:19:45,718 INFO  [MSC service thread 1-7] stdout: at org.jboss.weld.event.ObserverMethodImpl.getReceiverIfExists(ObserverMethodImpl.java:287)

2014-08-25 15:19:45,718 INFO  [MSC service thread 1-7] stdout: at org.jboss.weld.event.ObserverMethodImpl.sendEvent(ObserverMethodImpl.java:251)

2014-08-25 15:19:45,718 INFO  [MSC service thread 1-7] stdout: at org.jboss.weld.event.ObserverMethodImpl.notify(ObserverMethodImpl.java:232)

2014-08-25 15:19:45,719 INFO  [MSC service thread 1-7] stdout: at org.jboss.weld.event.ObserverNotifier.notifyObserver(ObserverNotifier.java:169)

2014-08-25 15:19:45,719 INFO  [MSC service thread 1-7] stdout: at org.jboss.weld.event.ObserverNotifier.notifyObserver(ObserverNotifier.java:165)

2014-08-25 15:19:45,719 INFO  [MSC service thread 1-7] stdout: at org.jboss.weld.event.TransactionalObserverNotifier.notifyObserver(TransactionalObserverNotifier.java:46)

2014-08-25 15:19:45,719 INFO  [MSC service thread 1-7] stdout: at org.jboss.weld.event.ObserverNotifier.notifyObservers(ObserverNotifier.java:119)

2014-08-25 15:19:45,719 INFO  [MSC service thread 1-7] stdout: at org.jboss.weld.event.ObserverNotifier.fireEvent(ObserverNotifier.java:112)

2014-08-25 15:19:45,719 INFO  [MSC service thread 1-7] stdout: at org.jboss.weld.event.EventImpl.fire(EventImpl.java:83)

2014-08-25 15:19:45,719 INFO  [MSC service thread 1-7] stdout: at org.erratic.servlets.listener.StartupListener.contextInitialized(StartupListener.java:36)

2014-08-25 15:19:45,720 INFO  [MSC service thread 1-7] stdout: at io.undertow.servlet.core.ApplicationListeners.contextInitialized(ApplicationListeners.java:173)

2014-08-25 15:19:45,720 INFO  [MSC service thread 1-7] stdout: at io.undertow.servlet.core.DeploymentManagerImpl.deploy(DeploymentManagerImpl.java:187)

2014-08-25 15:19:45,720 INFO  [MSC service thread 1-7] stdout: at org.wildfly.extension.undertow.deployment.UndertowDeploymentService.startContext(UndertowDeploymentService.java:86)

2014-08-25 15:19:45,720 INFO  [MSC service thread 1-7] stdout: at org.wildfly.extension.undertow.deployment.UndertowDeploymentService.start(UndertowDeploymentService.java:71)

2014-08-25 15:19:45,720 INFO  [MSC service thread 1-7] stdout: at org.jboss.msc.service.ServiceControllerImpl$StartTask.startService(ServiceControllerImpl.java:1948)

2014-08-25 15:19:45,720 INFO  [MSC service thread 1-7] stdout: at org.jboss.msc.service.ServiceControllerImpl$StartTask.run(ServiceControllerImpl.java:1881)

2014-08-25 15:19:45,720 INFO  [MSC service thread 1-7] stdout: at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)

2014-08-25 15:19:45,720 INFO  [MSC service thread 1-7] stdout: at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)

2014-08-25 15:19:45,721 INFO  [MSC service thread 1-7] stdout: at java.lang.Thread.run(Thread.java:744)

2014-08-25 15:19:45,721 INFO  [MSC service thread 1-7] stdout: Caused by: org.modeshape.jcr.index.local.LocalIndexException: The local provider does not support multi-column indexes

2014-08-25 15:19:45,721 INFO  [MSC service thread 1-7] stdout: at org.modeshape.jcr.index.local.ManagedLocalIndexBuilder.create(ManagedLocalIndexBuilder.java:65)

2014-08-25 15:19:45,721 INFO  [MSC service thread 1-7] stdout: at org.modeshape.jcr.index.local.LocalIndexProvider.validateProposedIndex(LocalIndexProvider.java:123)

2014-08-25 15:19:45,721 INFO  [MSC service thread 1-7] stdout: at org.modeshape.jcr.RepositoryIndexManager.registerIndexes(RepositoryIndexManager.java:405)

2014-08-25 15:19:45,721 INFO  [MSC service thread 1-7] stdout: at org.modeshape.jcr.JcrRepository$RunningState.loadIndexDefinitions(JcrRepository.java:1572)

2014-08-25 15:19:45,721 INFO  [MSC service thread 1-7] stdout: at org.modeshape.jcr.JcrRepository$RunningState.<init>(JcrRepository.java:1213)

2014-08-25 15:19:45,722 INFO  [MSC service thread 1-7] stdout: at org.modeshape.jcr.JcrRepository$RunningState.<init>(JcrRepository.java:980)

2014-08-25 15:19:45,722 INFO  [MSC service thread 1-7] stdout: at org.modeshape.jcr.JcrRepository.doStart(JcrRepository.java:389)

2014-08-25 15:19:45,722 INFO  [MSC service thread 1-7] stdout: at org.modeshape.jcr.JcrRepository.start(JcrRepository.java:313)

2014-08-25 15:19:45,722 INFO  [MSC service thread 1-7] stdout: at org.modeshape.jcr.JcrRepositoriesContainer$UrlRepositoryLookup.repository(JcrRepositoriesContainer.java:212)

2014-08-25 15:19:45,722 INFO  [MSC service thread 1-7] stdout: ... 73 more

{code}

Note, we changed "value" to "VALUE" because of MODE-2279

{code}

{

    "name" : "modeshapeRepository",

    "jndiName": "jcr/modeshapeRepository",

    "monitoring" : {

        "enabled" : true

    },

    "indexProviders" : {

        "local" : {

            "classname" : "org.modeshape.jcr.index.local.LocalIndexProvider",

            "directory" : "target/local_index_test_repository"

        }

    },

    "indexes" : {

        "indexWithSingleColumn" : {

            "kind" : "VALUE",

            "provider" : "local",

            "nodeType" : "mix:title",

            "columns" : "jcr:title(STRING)"

        }

    },

    "storage" : {

        "cacheName" : "contentRepository",

        "cacheConfiguration" : "META-INF/infinispan-sql-config.xml",

        "binaryStorage" : {

            "type"  : "database",

            "dataSourceJndiName" : "java:/jdbc/modeshapeRepositoryDb"

        }

    },

    "workspaces" : {

        "default" : "default",

        "allowCreation" : true

    },

    "security" : {

        "anonymous" : {

            "roles" : ["readonly","readwrite","admin"],

            "useOnFailedLogin" : false

        }

    }

}

{code}

[rhauch]

I'm looking into this under MODE-2160, which is still open so that I can add more tests and verify more of the functionality.

[rhauch]

Bjoern, I've merged two more pull-requests (one from MODE-2279 and another for MODE-2160) that should get you farther. As I mentioned on MODE-2160, I've completed some end-to-end testing with LONG and STRING indexes. Give it a whirl; I'll keep doing more testing tomorrow.

[bes82]

Thanks Randall we will test it later.

Currently there is another (more or less) related problem arising and I could user some info about the internals.

We use a standard infinispan config from your examples, one with sql backend and one with file storage.

Now we have about 300k Nodes for testing and of course no indexing atm.

When we fire a SQL2 Query and ask for instance for a nodetype or property, all nodes have to be loaded because there is no index.

So ok, this is slow. BUT this is like 100 times slower with sql backend. From what I can see from iptraf and sql logs is that Modeshape/Infinispan does query for every node in the repo upon firing the first JCR query (which is understandable because all nodes have to be searched) After that the infinispan cache is populated and seems to hold all 300k nodes.

(Related question: are ther some config examples that show how I may configure the size of the cache, the purging policy and stuff like that?)

BUT: this is like 100 times faster with file storage. Now I understand that there is a lot of protocol overhead with SQL TCP connections, but why does Modeshape fire 300k single queries for each node? shouldn't there be some more intelligent "query translation mechanism"?

I mean if I do the same manually I would write select * from ISP_STRING_TABLE and then process the result. that would take lake 5 or 15 seconds or so and not 5 minutes.

I have to admit, I have no idea about infinispan and what its capabilites are, I just assume that there is something like "get me the first 10k of entries" or "get me all keys starting with a,b,c,..."

[rhauch]

BUT: this is like 100 times faster with file storage. Now I understand that there is a lot of protocol overhead with SQL TCP connections, but why does Modeshape fire 300k single queries for each node? shouldn't there be some more intelligent "query translation mechanism"?

I mean if I do the same manually I would write select * from ISP_STRING_TABLE and then process the result. that would take lake 5 or 15 seconds or so and not 5 minutes.

Infinispan does not offer any kind of query functionality. Or rather, it does and it's new, but for various reasons ModeShape can't advantage of it because of how we're storing information in Infinispan and because the structure/schema of what we store varies tremendously. Besides, Infinispan relies upon Hibernate Search and Lucene, like ModeShape did in 3.x. We believe that our approach with 4.x and explicit indexes will be superior. With explicit indexes, your goal should be to define indexes such that each of your queries be able to use an index, and that index should dramatically reduce the number of nodes that need to be examined and evaluated against the other constraints.

As for file storage vs JDBC/RDBMS storage, yes the local file storage is generally much faster and the JDBC/RDBMS storage is much slower. This is largely because file storage is much more agreeable to Infinispan's API and the way it accesses persisted entries. Infinispan 6 added a new LevelDB cache store and a new file system cache store. Hopefully you're already using the faster filesystem cache store, but if you haven't already looked at LevelDB please do, because I understand it to be the fastest cache store. See the Infinispan 6 documentation on LevelDB for more information.

I have to admit, I have no idea about infinispan and what its capabilites are, I just assume that there is something like "get me the first 10k of entries" or "get me all keys starting with a,b,c,..."

Actually, Infinispan's API is not like this at all; the API is basically an ConcurrentMap where some of the entries are available in-memory (because they've been recently accessed) and the remaining entries are persisted to the cache store.

ModeShape and Infinispan both will take advantage of every bit of memory you can give them. Memory is cheap, and it's a very easy way to get extra performance.