Encountering NodeNotFoundInParentException when accessing the same Folder concurrently

[jkreutzfeld]

Hi everyone,

i'm currently encountering a NodeNotFoundInParentException when accessing the same folder node concurrently in multiple threads.

Our application supports uploading files to ModeShape and when multiple threads upload to the same folder, the exception occurs. It seems that the first uploaded file is already listed as a child of the folder, but when the child tries to get the parent's reference to itself, an old version of the parent is referenced which does not contain the new file yet.

Here is the stacktrace:

org.modeshape.jcr.cache.NodeNotFoundInParentException: Cannot locate child node: a69d00a317f1e7286cbfa4-9f57-4f8f-837c-b970f87ef522 within parent: a69d00a317f1e76415439c-c152-41b9-a3a0-f6735b46329c
  at org.modeshape.jcr.cache.document.LazyCachedNode.parentReferenceToSelf(LazyCachedNode.java:247)
  at org.modeshape.jcr.cache.document.LazyCachedNode.getSegment(LazyCachedNode.java:284)
  at org.modeshape.jcr.cache.document.LazyCachedNode.getPath(LazyCachedNode.java:293)
  at org.modeshape.jcr.JcrSession.node(JcrSession.java:559)
  at org.modeshape.jcr.JcrSession.node(JcrSession.java:504)
  at org.modeshape.jcr.JcrSession.node(JcrSession.java:482)
  at org.modeshape.jcr.AbstractJcrNode.getBaseVersion(AbstractJcrNode.java:3215)
  at org.modeshape.jcr.JcrVersionManager.getBaseVersion(JcrVersionManager.java:282)
  at org.apache.chemistry.opencmis.jcr.impl.DefaultDocumentTypeHandler.getJcrNode(DefaultDocumentTypeHandler.java:96)
  at org.apache.chemistry.opencmis.jcr.impl.DefaultDocumentTypeHandler.getJcrNode(DefaultDocumentTypeHandler.java:57)
  at org.apache.chemistry.opencmis.jcr.type.JcrTypeHandlerManager.create(JcrTypeHandlerManager.java:123)
  at org.apache.chemistry.opencmis.jcr.JcrNode.create(JcrNode.java:946)
  at org.apache.chemistry.opencmis.jcr.JcrFolder$1.next(JcrFolder.java:454)
  at org.apache.chemistry.opencmis.jcr.JcrFolder$1.next(JcrFolder.java:448)
  at org.apache.chemistry.opencmis.jcr.util.FilterIterator.hasNext(FilterIterator.java:51)
  at org.apache.chemistry.opencmis.jcr.JcrRepository.getChildren(JcrRepository.java:588)
  at org.apache.chemistry.opencmis.jcr.JcrService.getChildren(JcrService.java:125)
  at org.apache.chemistry.opencmis.server.support.wrapper.ConformanceCmisServiceWrapper.getChildren(ConformanceCmisServiceWrapper.java:631)
  at org.apache.chemistry.opencmis.client.bindings.spi.local.NavigationServiceImpl.getChildren(NavigationServiceImpl.java:80)
  at org.apache.chemistry.opencmis.client.runtime.FolderImpl$2.fetchPage(FolderImpl.java:267)
  at org.apache.chemistry.opencmis.client.runtime.util.AbstractIterator.getCurrentPage(AbstractIterator.java:132)
  at org.apache.chemistry.opencmis.client.runtime.util.CollectionIterator.hasNext(CollectionIterator.java:48)
  at de.doubleslash.drive.base.iface.view.upload.UploadRunnable.checkName(UploadRunnable.java:112)
  at de.doubleslash.drive.base.iface.view.upload.UploadRunnable.run(UploadRunnable.java:73)
  at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
  at java.util.concurrent.FutureTask.run(FutureTask.java:262)
  at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
  at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
  at java.lang.Thread.run(Thread.java:744)

We are using ModeShape 4.0 and the Chemistry JCR Bridge for CMIS access. The problem sounds alot like the one posted at [MODE-2121] LazyCachedNode exception under heavy concurrent load - JBoss Issue Tracker but that should have been resolved a long time ago.

Any idea?

[hchiorean]

If your cache configuration has eviction enabled, you may be running into [MODE-2280] Child node not found under high concurrency when eviction is enabled and SingleFile store is used - JBoss Is… - which is caused by a bug in Infinispan. If you're not using eviction, are you running in Wildfly or standalone ? If you're running in Wildfly, are you using READ_COMMITTED isolation for the Infinispan cache ?

It also seems strange to me from the stack trace that versioning is being used. Are you trying to version the folder/files ?

[jkreutzfeld]

We have not set up eviction in our configuration. Our application is running inside Apache Karaf and ModeShape is deployed as an OSGi bundle. The files in our application need to be versioned, do you think that might be the problem? I'd understand if there were versioning / locking issues, but i cannot relate to this one.

[hchiorean]

I don't necessarily think versioning is the issue, just that using versioning "complicates" the setup. If you can recreate this in a simple-enough test case that we can run locally using 4.1, feel free to open a JIRA. Thanks.

EDIT: also make sure you're using PESSIMISTIC locking for the ISPN cache.

[jkreutzfeld]

I will try to compose an example. If it helps, i tried ModeShape 4.1.0 but it does not make a difference. Locking is already configured to pessimistic.

EDIT: I created a ticket: [MODE-2418] NodeNotFoundInParentException when concurrently accessing a node - JBoss Issue Tracker