[rhauch]

I just merged the fix for MODE-1172 into the 'master' branch, and I closed the issue.

[sjahan]

Well i'm glad i can help you, since your support is a true help to work, it's kind of fair

Good to know that the bug is already solved, thank you very much for your reactivity and your speed!

We'll upgrade as soon as the Beta3 will be released!

Thank you for everything and have a very good weekend,

SJ.

[sjahan]

Hi,

It's been some weeks i've been working on another topic high priority, and today, i'm getting back on the versioning stuff.

Since the last time, i've installed Modeshape 2.5, and i got the lucene error, but you helped me to solve it and i can now run modeshape correctly.

However, i encounter a new problem happening at the checkin():

Caused by: javax.jcr.ItemNotFoundException: Unable to find "/{http://www.jcp.org/jcr/1.0}system/{http://www.jcp.org/jcr/1.0}versionStorage/{}ac/{}a8/{}7b/{}e3/{}3d31-44c5-bae4-e4e947c43a64/{}2011-06-15T16?08?35.222+02?00"; lowest existing path is "/{http://www.jcp.org/jcr/1.0}system/{http://www.jcp.org/jcr/1.0}versionStorage/{}ac/{}a8/{}7b/{}e3/{}3d31-44c5-bae4-e4e947c43a64"

    at org.modeshape.jcr.SessionCache.findJcrNode(SessionCache.java:617)

    at org.modeshape.jcr.JcrVersionManager.checkin(JcrVersionManager.java:367)

    at org.modeshape.jcr.AbstractJcrNode.checkin(AbstractJcrNode.java:1827)

    at org.oea.jcr.management.impl.JCRManagerImpl.updateDOMHierarchyRecursively(JCRManagerImpl.java:976)

    ... 38 more

Caused by: org.modeshape.graph.property.PathNotFoundException: Unable to find "/{http://www.jcp.org/jcr/1.0}system/{http://www.jcp.org/jcr/1.0}versionStorage/{}ac/{}a8/{}7b/{}e3/{}3d31-44c5-bae4-e4e947c43a64/{}2011-06-15T16?08?35.222+02?00"; lowest existing path is "/{http://www.jcp.org/jcr/1.0}system/{http://www.jcp.org/jcr/1.0}versionStorage/{}ac/{}a8/{}7b/{}e3/{}3d31-44c5-bae4-e4e947c43a64"

    at org.modeshape.graph.session.GraphSession$Node.getChild(GraphSession.java:2822)

    at org.modeshape.graph.session.GraphSession$Node.getChild(GraphSession.java:2790)

    at org.modeshape.graph.session.GraphSession.findNodeWith(GraphSession.java:327)

    at org.modeshape.jcr.SessionCache.findJcrNode(SessionCache.java:615)

    ... 41 more

I don't know why this crashes. The db is freshly created and i just create the node and then try to update it. Basically, my node is the following:

<Document>

     <Field/>

</Document>

The document has to be versioned and the field has to be included in the version, so my function basically checks out the document, then updates the field, then checks in the document and i get this exception.

As i said, the DB is freshly created, so no strange stuff in the system workspace that could be left and get troublesome, the environment is clean.

Hoping you know what it is about...

Thank you for your support,

SJ.

[rhauch]

Hmm... I have a few suggestions for helping to isolate the problem.

1) Does this happen on the in-memory connector?

2) What's does the code that uses JCR API look like?

3) Can you reproduce this with a simple test? You could clone the https://github.com/ModeShape/modeshape-examples repo and hack the 'modeshape-embedded-example' to replicate the problem.

[sjahan]

Hi Randall,

1) We are using the mysql connector only for the moment.

2) Here is our update method. I never changed the main routine since it always worked. I just add the versionConfig variable which tells the system either it must check out/in according to the type of the node (Document or Field).

private void updateDOMHierarchyRecursively(Element inputDOM, Node node, int depth, boolean newVersion, Set<String> nodesKept, boolean allowAddNodes, boolean allowAddProperties, boolean allowDeleteNodes, boolean allowDeleteProperties) throws ElementNotFoundException, InvalidXmlException, JCRException {

        //Check if current node (and its hierarchy) should be updated or not and how.

        ArrayList<String> versionConfig = this.versioningConfig.get(inputDOM.getName());

        if(versionConfig==null){versionConfig = this.versioningConfig.get("default");}

        if (!nodesKept.contains(inputDOM.getName())) {

            try {

                //Compare UUIDs

                String domUUID = inputDOM.getAttributeValue("uuid", GlobalContext.JCR_NAMESPACE);

                String nodeUUID = node.getUUID();

                //Verify that UUIDs match.

                if ((domUUID == null) || (nodeUUID.equals(domUUID))) {

                    if ((domUUID != null) || (allowAddNodes)) {

                        if(newVersion && versionConfig.get(0).equals("mix:versionable")){

                            node.checkout();

                        }

                        // Update of node children if depth > 0

                        if (depth > 0) {

                            //Construct a HashMap of existing nodes (JCR Nodes).

                            Map<String, Node> existingChildren = new HashMap<String, Node>();

                            NodeIterator it = node.getNodes();

                            while (it.hasNext()) {

                                Node existingChild = (Node) it.next();

                                existingChildren.put(existingChild.getUUID(), existingChild);

                            }

                            //Create Set and Iterator for existing children.

                            Set<String> existingChildrenKeys = existingChildren.keySet();

                            Iterator<String> existingChildrenKeysIt = existingChildrenKeys.iterator();

                            //Construct a HashMap of nodes to update (DOM Elements).

                            Map<String, Element> childrenToUpdate = new HashMap<String, Element>();

                            //Construct a List of nodes to add (DOM Elements).

                            List<Element> childrenToAdd = new ArrayList<Element>();

                            List<Element> list = inputDOM.getChildren();

                            int i = 0;

                            while (i < list.size()) {

                                Element inputChild = list.get(i);

                                String uuid = inputChild.getAttributeValue("uuid", GlobalContext.JCR_NAMESPACE);

                                if (uuid != null) {

                                    //this is a child to update.

                                    childrenToUpdate.put(inputChild.getAttributeValue("uuid", GlobalContext.JCR_NAMESPACE), inputChild);

                                } else {

                                    //This is a child to add.

                                    childrenToAdd.add(inputChild);

                                }

                                i++;

                            }

                            //Iterate over existing children (treat updates & deletes).

                            while (existingChildrenKeysIt.hasNext()) {

                                String uuid = (String) existingChildrenKeysIt.next();

                                Node existingChild = existingChildren.get(uuid);

                                if (!childrenToUpdate.containsKey(uuid)) {

                                    //Delete child.

                                    if (allowDeleteNodes) {

                                        existingChild.remove();

                                    }

                                    else throw new InvalidXmlException("Existing node "+existingChild.getName()+" cannot be deleted (not allowed).");

                                } else {

                                    //Update child.

                                    updateDOMHierarchyRecursively(childrenToUpdate.get(uuid), existingChild, depth-1, newVersion, nodesKept, allowAddNodes, allowAddProperties, allowDeleteNodes, allowDeleteProperties);

                                }

                            }

                            //Iterate over children to add. Not used right now

                            if ((allowAddNodes) && (childrenToAdd.size() > 0)) {

                                for (int j = 0; j < childrenToAdd.size(); j++) {

                                    //Add child.

                                    Element newChild = childrenToAdd.get(j);

                                    Node childNode = node.addNode(newChild.getName());

                                    //childNode.addMixin("mix:versionable");

                                    //childNode.addMixin("mix:referenceable");

                                    updateDOMHierarchyRecursively(newChild, childNode, depth-1, newVersion, nodesKept, allowAddNodes, allowAddProperties, allowDeleteNodes, allowDeleteProperties);

                                }

                            } else {

                                //Adding nodes is not allowed.

                                if (childrenToAdd.size() > 0) {

                                    Element childToAdd = childrenToAdd.get(0);

                                    throw new InvalidXmlException("New node "+childToAdd.getName()+" cannot be added (not allowed).");

                                }

                            }

                        }

                        //Construct a HashMap of existing properties.

                        Map<String, Property> existingProperties = new HashMap<String, Property>();

                        PropertyIterator it = node.getProperties();

                        while (it.hasNext()) {

                            Property property = it.nextProperty();

                            String propertyName = property.getName();

                            //Get only custom properties.

                            if (!propertyName.startsWith(GlobalContext.JCR_NS) && !propertyName.startsWith(GlobalContext.EA_NS)) {

                                existingProperties.put(propertyName, property);

                            }

                        }

                        //Create Set and Iterator for existing properties.

                        Set<String> existingPropertiesKeys = existingProperties.keySet();

                        Iterator<String> existingPropertiesKeysIt = existingPropertiesKeys.iterator();

                        //Construct a HashMap of properties to update.

                        Map<String, String> propertiesToUpdate = new HashMap<String, String>();

                        //Construct a list of properties to add.

                        List<Attribute> propertiesToAdd = new ArrayList<Attribute>();

                        List<Attribute> list = inputDOM.getAttributes();

                        for (int j = 0; j < list.size(); j++) {

                            Attribute attribute = list.get(j);

                            String attributeName = attribute.getQualifiedName();

                            String attributeValue = attribute.getValue();

                            String attributeNamespace = attribute.getNamespacePrefix();

                            //Get only custom properties.

                            if ((attributeNamespace == null) || (!attributeNamespace.equals(GlobalContext.JCR_NS) && !attributeNamespace.equals(GlobalContext.EA_NS))) {

                                if (existingProperties.containsKey(attributeName)) {

                                    //this is a property to update.

                                    propertiesToUpdate.put(attributeName, attributeValue);

                                } else {

                                    //This is a property to add.

                                    propertiesToAdd.add(attribute);

                                }

                            }

                        }

                        //Iterate over existing properties (treat updates & deletes).

                        while (existingPropertiesKeysIt.hasNext()) {

                            String attributeName = (String) existingPropertiesKeysIt.next();

                            Property property = existingProperties.get(attributeName);

                            if (!propertiesToUpdate.containsKey(attributeName)) {

                                //Delete property.

                                if (allowDeleteProperties) property.remove();

                                else throw new InvalidXmlException("Existing property "+attributeName+" cannot be deleted (not allowed).");

                            } else {

                                //Update property.

                                property.setValue(propertiesToUpdate.get(attributeName));

                            }

                        }

                        //Iterate over properties to add.

                        if ((allowAddProperties) && (propertiesToAdd.size() > 0)) {

                            for (int j = 0; j < propertiesToAdd.size(); j++) {

                                //Add property.

                                Attribute newProperty = propertiesToAdd.get(j);

                                node.setProperty(newProperty.getQualifiedName(), newProperty.getValue());

                            }

                        } else {

                            //Adding properties is not allowed.

                            if (propertiesToAdd.size() > 0) {

                                Attribute newProperty = propertiesToAdd.get(0);

                                throw new InvalidXmlException("New property "+newProperty.getQualifiedName()+" cannot be added (not allowed).");

                            }

                        }

                        //Date of last modification.

                        String now = new SimpleDateFormat("yyyy-MM-dd hh:mm:ss").format(new Date());

                        node.setProperty(GlobalContext.EA_NS+":dateModified", now);

                        session.save();

                        if (newVersion && versionConfig.get(0).equals("mix:versionable")) {

                            node.checkin();

                        }

                    } else {

                        //Adding nodes is not allowed.

                        throw new InvalidXmlException("New node "+inputDOM.getName()+" cannot be added (not allowed).");

                    }

                } else {

                    //UUIDs are different...

                    throw new InvalidXmlException("A '"+inputDOM.getName()+"' node has a wrong UUID.");

                }

            } catch (RepositoryException e) {

                throw new JCRException("Unable to update node: "+e.getMessage(), e);

            }

        }

    }

3) I'll see what i can do with the modeshape-embedded-example

Thank you very much for your help,

SJ.

[sjahan]

Maybe this is coming from our side:

I posted the update method, but i just checked the add method i use to inject content and surprise, there was no checkin() in this one method.

Could that explain? I guess the guy who developped it was better than me with Modeshape...

When i tried to add the checkin at the end of the add method, i get this:

Caused by: javax.jcr.InvalidItemStateException: This operation cannot be performed when the session has pending changes

    at org.modeshape.jcr.JcrVersionManager.checkin(JcrVersionManager.java:296)

    at org.modeshape.jcr.AbstractJcrNode.checkin(AbstractJcrNode.java:1827)

    at org.oea.jcr.management.impl.JCRManagerImpl.addDOMHierarchyRecursively(JCRManagerImpl.java:791)

    ... 43 more

I don't know if this has to be checked in at the end of the add to create a new version, or if the fact of adding a versionable node will do the trick. I assume the guy who implement the addDOMHierarchyRecursively method should have thought it was automatic since the node is versionable.

If this is not auto, i have to checkin(), how should i proceed? I may be in the case where i have to add a whole tree and so, i should check in several nodes to create a version for each versionable branch/node, but this doesn't seem to work that way.

Thank you very much,

SJ.

[rhauch]

sjahan wrote:

1) We are using the mysql connector only for the moment.

Yeah, I remembered you were using the JPA connector with MySQL. If you tried the in-memory connector (purely as a test) and it also exhibited the problem, we'd pretty likely eliminate the JPA connector from the problem space.

FWIW, I initially suspect the JPA connector - or rather it's configuration and/or the MySQL DB configuration.

Are you using MySQL without transactions or are you using the ISAM or InnoDB transaction engine? (I asked a few weeks ago, but did see your answer. Forgive me if I missed it.)

And do you know what the database's transaction isolation level is? I believe the default for InnoDB is REPEATABLE_READ, which may be causing the problem here (of course, it would depend on timing). If we look at part of the stack trace:

    at org.modeshape.jcr.SessionCache.findJcrNode(SessionCache.java:617)

    at org.modeshape.jcr.JcrVersionManager.checkin(JcrVersionManager.java:367)

    at org.modeshape.jcr.AbstractJcrNode.checkin(AbstractJcrNode.java:1827)

    at org.oea.jcr.management.impl.JCRManagerImpl.updateDOMHierarchyRecursively(JCRManagerImpl.java:976)

the "JcrVersionManager.checkin(...)" method has already pushed the version into the version storage area, and on line 367 it is trying to read that version node back out. If the DB is configured to use REPEATABLE read, it might actually be reading the state of the version storage area using a transaction that was started before the transaction storing the new version has completed.

So another thing to try would be to set the transaction isolation level for your MySQL database to be SERIALIZED, which is the most strict. This is not something you'd want in production, but again it'd help us isolate the problem.

All of this is still conjecture at this point. It's obvious there's a problem somewhere, but we just need to figure out how to fix it.

[rhauch]

sjahan wrote:

Maybe this is coming from our side:

I posted the update method, but i just checked the add method i use to inject content and surprise, there was no checkin() in this one method.

Well, they need to behave consistently.

When i tried to add the checkin at the end of the add method, i get this:

Caused by: javax.jcr.InvalidItemStateException: This operation cannot be performed when the session has pending changes

    at org.modeshape.jcr.JcrVersionManager.checkin(JcrVersionManager.java:296)

    at org.modeshape.jcr.AbstractJcrNode.checkin(AbstractJcrNode.java:1827)

    at org.oea.jcr.management.impl.JCRManagerImpl.addDOMHierarchyRecursively(JCRManagerImpl.java:791)

    ... 43 more

This is because you're calling 'checkin' before you've saved all of the changes in your session. Check your 'add' method to make sure you're saving the session before calling 'checkin'.

Also, please verify that you're code is checking in only those nodes that have an OPV of VERSION. Earlier you said this is what you were trying to do, but couldn't because of a bug in 2.4.0.Final (and even 2.5.0.Beta2). That bug was fixed in 2.5.0.Final.

[sjahan]

Hi Randall,

Well, i have corrected the add function and i got the same thing when the first checkin occurs:

org.oea.common.exceptions.JCRException: Unable to add node: Unable to find "/{http://www.jcp.org/jcr/1.0}system/{http://www.jcp.org/jcr/1.0}versionStorage/{}d1/{}65/{}c8/{}a5/{}3d00-419b-a021-ee5902d5d7cc/{}2011-06-16T17?33?43.436+02?00"; lowest existing path is "/{http://www.jcp.org/jcr/1.0}system/{http://www.jcp.org/jcr/1.0}versionStorage/{}d1/{}65/{}c8/{}a5/{}3d00-419b-a021-ee5902d5d7cc"

    at org.oea.jcr.management.impl.JCRManagerImpl.addDOMHierarchyRecursively(JCRManagerImpl.java:800)

The code is checking in only if the node has the mixin versionable. If i'm not wrong, we don't care the OPV of the node checked-in, because OPV matters for the children of this node.

Despite of the fact that our add function wasn't clean, this looks like there is however a trouble with the modeshape check in function.

I'll try to tweak a little bit my MySQL DB to check the InnoDB transaction stuff. I know that we use InnoDB but i'm not such an expert with the DB besides using it, so i'll try your tuning and will return you the results ASAP!

Thank you very much and have a good day,

SJ.

[rhauch]

If you're using InnoDB, you should be using the "org.hibernate.dialect.MySQLInnoDBDialect" for your Hibernate dialect. If I recall from an earlier post, you were using the "org.hibernate.dialect.MySQLDialect" instead. I'm not sure what the difference is between the two, but you always should use the most specific dialect for your DBMS.

[sjahan]

Well, i changed the values to the MySQLInnoDBDialect, and i typed SET TRANSACTION ISOLATION LEVEL SERIALIZABLE; in the sql prompt of my db.

No changes yet, i always get the same error as soon as the first check in is called.

[rhauch]

Okay, I'm out of ideas. At this point, the best way forward is to get a test case of some kind so that we can reproduce what you're seeing.

[sjahan]

Hi Randall,

I'll try to build a simple case that reproduce it this morning, from the modeshape-embedded-example, so that i can send it to you at the beginning of the afternoon (which should be the beginning of your day!).

Thank you very much for your help on this matter,

SJ.

[sjahan]

I've sent you some stuff so that you can reproduce our problem, by mail on your redhat address. I found it on the web, so if you received nothing, please notice me as soon as possible so that i can re-send you the test case on a proper mail address (or using the more convenient way to you).

Thank you very much for all your help,

SJ.

[sjahan]

Hi Randall,

Did you receive my mail with the testcase? Did this help you to reproduce "successfully" the problem i encounter?

Hoping we cound find a solution, please don't hesitate to contact me if you need anything else.

Thank you for your support,

SJ.