[sjahan]

I corrected my CND files as you suggested and i have no longer something in the "problems" variable so i assume this should be good about the loading of new nodetypes.

I'll work on my versioning stuff and will keep this thread updated.

This is quite the emergency right now on this matter but i'll take some time before the end of the week to request on your JIRA the enhancement you mentioned!

Thank you very much Randall,

Have a great day,

SJ.

[sjahan]

Well, this is not so simple.

After some debug, it appears that i came back to my other problem (mentionned above).

I guess the nodetypes are ok since there is no more problems in the JCRConfiguration after the loading of the CND files but when i want to make the JCREngine using that: engine = configuration.build(); then i loose the repository in my JCRConfiguration, and obviously there is also no repository in the engine built.

Hoping you know why!

SJ.

[rhauch]

Can you share your configuration?

[sjahan]

About the global setup, our server runs on JBoss AS 5.1, with JDK6 and the data are stored into a MySQL DB trhough Modeshape 2.4.0.

Here is our modeshape configuration file, i got it like this when i came on the project so this may contain error that i'm not aware of.

Please note that we currently only use the Portal source and repository!

<?xml version="1.0" encoding="UTF-8"?>
<configuration xmlns:mode="http://www.modeshape.org/1.0" xmlns:jcr="http://www.jcp.org/jcr/1.0">
    <mode:sources jcr:primaryType="nt:unstructured">
        <!-- Portal source -->
        <mode:source jcr:name="Portal" mode:classname="org.modeshape.connector.store.jpa.JpaSource" mode:dataSourceJndiName="java:OEAPortal" mode:dialect="org.hibernate.dialect.MySQL5Dialect" mode:model="Simple" mode:maximumConnectionsInPool="1" mode:largeValueSizeInBytes="10000" mode:retryLimit="3" mode:compressData="false" mode:showSql="true" mode:autoGenerateSchema="validate" mode:creatingWorkspacesAllowed="true" mode:defaultWorkspaceName="workspace">
            <mode:predefinedWorkspaceNames>system</mode:predefinedWorkspaceNames>
            <mode:predefinedWorkspaceNames>workspace</mode:predefinedWorkspaceNames>
        </mode:source>
        <!-- Repository source -->
        <mode:source jcr:name="Repository" mode:classname="org.modeshape.connector.store.jpa.JpaSource" mode:dataSourceJndiName="java:OEARepository" mode:dialect="org.hibernate.dialect.MySQL5Dialect" mode:model="Simple" mode:maximumConnectionsInPool="1" mode:largeValueSizeInBytes="10000" mode:retryLimit="3" mode:compressData="false" mode:showSql="true" mode:autoGenerateSchema="validate" mode:creatingWorkspacesAllowed="true" mode:defaultWorkspaceName="workspace">  
                <mode:predefinedWorkspaceNames>system</mode:predefinedWorkspaceNames>
                <mode:predefinedWorkspaceNames>workspace</mode:predefinedWorkspaceNames>
        </mode:source>
    </mode:sources>
    <mode:sequencers>
        <!--mode:sequencer jcr:name="Image Sequencer" mode:classname="org.modeshape.sequencer.image.ImageMetadataSequencer">
            <mode:description>Image metadata sequencer</mode:description>        
            <mode:pathExpression>/foo/source => /foo/target</mode:pathExpression>
            <mode:pathExpression>/bar/source => /bar/target</mode:pathExpression>
        </mode:sequencer-->
    </mode:sequencers>
    <mode:mimeTypeDetectors>
        <mode:mimeTypeDetector jcr:name="Detector">
            <mode:description>Standard extension-based MIME type detector</mode:description>
            <mode:classname>org.modeshape.graph.mimetype.ExtensionBasedMimeTypeDetector</mode:classname>
            <mode:classpath/>
            <mode:classpath/>
        </mode:mimeTypeDetector>
    </mode:mimeTypeDetectors>
    <mode:repositories>        
        <mode:repository jcr:name="Portal">
            <!-- Specify the source that should be used for the repository -->
            <mode:source>Portal</mode:source>
            <!-- Define the options for the JCR repository, using camelcase version of JcrRepository.Option names -->
            <mode:options jcr:primaryType="mode:options">
                <systemSourceName jcr:primaryType="mode:option" mode:value="system@Portal"/>
                <jaasLoginConfigName jcr:primaryType="mode:option" mode:value="modeshape-jcr"/>
            </mode:options>
            <!-- Define any custom node types. Importing CND files via JcrConfiguration is equivalent to specifying here. -->
            <mode:nodeTypes jcr:primaryType="mode:nodeTypes"/>
            <!-- Define any namespaces for this repository, other than those already defined by JCR or ModeShape -->
            <namespaces jcr:primaryType="mode:namespaces">
                <ea jcr:primaryType="mode:namespace" mode:uri="http://www.oea.org"/>
            </namespaces>
        </mode:repository>
        <mode:repository jcr:name="Repository">
            <!-- Specify the source that should be used for the repository -->
            <mode:source>Repository</mode:source>
            <!-- Define the options for the JCR repository, using camelcase version of JcrRepository.Option names -->
            <mode:options jcr:primaryType="mode:options">
                <systemSourceName jcr:primaryType="mode:option" mode:value="system@Repository"/>
                <jaasLoginConfigName jcr:primaryType="mode:option" mode:value="modeshape-jcr"/>
            </mode:options>
            <!-- Define any custom node types. Importing CND files via JcrConfiguration is equivalent to specifying here. -->
            <mode:nodeTypes jcr:primaryType="mode:nodeTypes"/>
            <!-- Define any namespaces for this repository, other than those already defined by JCR or ModeShape -->
            <namespaces jcr:primaryType="mode:namespaces">
                <ea jcr:primaryType="mode:namespace" mode:uri="http://www.oea.org"/>
            </namespaces>
        </mode:repository>
    </mode:repositories>
</configuration>

Please tell me if you need anything else!

Thank you very much for your support

SJ.

Formatted configuration file contents to be more readable.

[rhauch]

I wrote a simple test that loaded your configuration file and started the repository. I had to modify a few things, since I don't have access to MySQL at the moment. If I switched the source to the in-memory connector, everything worked just fine.

A couple of questions:

1) What version of MySQL are you using? There are 3 Hibernate dialects for MySQL, and you have to pick the one that matches what you're using, otherwise it won't work. If you're using InnoDB, you have to use the "org.hibernate.dialect.MySQLInnoDBDialect" dialect.

2) You are specifying a "validate" value for the "mode:autoGenerateSchema" attribute on the JPA source. This means that you or your DB admins have already set up the database with the proper schema (using the DDL generator utility from ModeShape) before starting up your application. Is this the case?

3) Are you printing out or logging all of the problems from the configuration before calling 'build()'? See the first code sample in section 7.3 of our Reference Guide for the suggested way to do this.

4) It's strange that you're not even getting a JcrEngine instance back from the JcrConfiguration.build() method, because that method doesn't really do much. (The JcrEngine.getRepository() method is where the real initialization is.)

Best regards.

[sjahan]

Here is some answers to your questions, hoping it could help:

1) We're using MySQL 5.1.41.

2) I don't know what is the DDL generator, but the database was empty, then we use the mode:autoGenerateSchema with the create value and it generates some tables in the DB.

3) I'll try to reach this data as soon as i posted this. --> Update : No problems in the JCRConfiguration. The repository from the configuration is just disappearing after the configuration.build() is processed. I'll attach the sources and try to debug a little bit deeper.

4) Actually I get back a JCREngine after the .build(). What i don't get is repository in it!

Before configuration.built(), i have my "Portal" repository in the configuration object.

Then i do engine = configuration.built(). Engine isn't not null, it's actually an instance of JCREngine, but there is no longer my "Portal" repository, neither in the JCRConfiguration nor in the JCREngine i get.

Thank you very much!

SJ.

Modified: Sébastien Jahan

[sjahan]

After some debug, here is why i have no more repository in the configuration and engine.

In the build(), i jump into

@Override

    public JcrConfiguration save() {

        super.save();

        this.repositoryDefinitions.clear();

        return this;

    }

What i don't understand is why before, this worked :s I'll investigate on my side, if you've any idea about the topic, i'll look after this thread very carefully!

Thank you very much Randall for your support,

SJ.

-------------------------------------------

UPDATE

That doesn't come from the code i mentioned above. This only cleans the Configuration.

Here is my code:

public void initialize() throws ConfigurationException {

        try {

            //Load configuration

            if (configuration == null) {

                configuration = new JcrConfiguration();

                InputStream inputStream = ResourceLoader.loadClasspathResource(getConfigurationFile());

                configuration.loadFrom(inputStream);

                configuration.repository(getName()).registerNamespace(GlobalContext.OEA_NS, GlobalContext.OEA_URI);

                configuration.repository(getName()).addNodeTypes("OEA_Unstructured.cnd");

                configuration.repository(getName()).addNodeTypes("OEA_CopyUnstructured.cnd");

                configuration.repository(getName()).addNodeTypes("OEA_IgnoreUnstructured.cnd");

                configuration.repository(getName()).addNodeTypes("OEA_VersionUnstructured.cnd");

                inputStream.close();

            }

            //Create engine

            engine = configuration.build();

            engine.start();

        } catch (IOException e) {

            throw new ConfigurationException("Unable to initialize \'"+getName()+"' storage.", e);

        } catch (SAXException e) {

            throw new ConfigurationException("Unable to initialize \'"+getName()+"' storage.", e);

        }

    }

When i comment the bold lines, it's ok, my engine actually has the Portal repository, but as soon as i register the namespace and add the nodetypes, no more repository in the JCREngine when i need it

That is quite strange. Keeping investigate,

Thank you for your support.

SJ.

[rhauch]

1) We're using MySQL 5.1.41.

With InnoDB or ISAM transaction engine? If so, be sure to use the right dialect.

2) I don't know what is the DDL generator, but the database was empty, then we use the mode:autoGenerateSchema with the create value and it generates some tables in the DB.

The DDL generator is a ModeShape utility that will produce a DDL file for your particular DBMS. This is typically used in a production environment, where your DB admins manually set up the database schema, and ModeShape is configured with the "mode:autoGenerateSchema" value set to "validate" or, with ModeShape 2.5.0.Beta1 or later, to "disable".

You can use "create", but note that this will drop any existing database tables and indexes upon startup. So this is often used for test cases (where you always want to start with a clean DB), but should not be used if you want your data to be preserved upon subsequent startups. In those cases, you should really be using "validate" and manually set up your database schema.

3) I'll try to reach this data as soon as i posted this. --> Update : No problems in the JCRConfiguration. The repository from the configuration is just disappearing after the configuration.build() is processed. I'll attach the sources and try to debug a little bit deeper.

4) Actually I get back a JCREngine after the .build(). What i don't get is repository in it!

Before configuration.built(), i have my "Portal" repository in the configuration object.

Then i do engine = configuration.built(). Engine isn't not null, it's actually an instance of JCREngine, but there is no longer my "Portal" repository, neither in the JCRConfiguration nor in the JCREngine i get.

I'm not sure I understand exactly what you are seeing. When you call 'configuration.build()', the JcrEngine instance will indeed not have a javax.jcr.Repository instance for "Portal", but it should create and return it when you first call "JcrEngine.getRepository(String)" with the name of your repository (e.g., "Portal"). Your code should look something like this:

JcrConfiguration configuration = new JcrConfiguration();
File file = new File("path/to/my/configuration/file/modeshapeConfig.xml");
configuration.loadFrom(file);

// Verify there were no problems with the configuration ...

if ( !configuration.getProblems().isEmpty() ) {
    for ( Problem problem : configuration.getProblems() ) {
        // Report these problems and decide whether to continue!
    }
} 

// Create and start the engine ...
JcrEngine engine = configuration.build();     // never null
engine.start();
try {
    Repository repository = engine.getRepository("Portal");
} catch ( RepositoryException e ) {
    // handle problem
} catch ( RuntimeException e ) {
    // handle problem
}

Note that although I loaded the configuration file from the file system, it's always possible to load the configuration file from an InputStream, URL, or from a resource file on the classpath.

And the following information might help when you're debugging, though it's really implementation detail that most users don't need to know about. When "configuration.build()" creates the JcrEngine instance, it actually copies the configuration object into an immutable representation, and the engine then holds onto that immutable representation. And at that point, the JcrEngine no longer references your configuration object, and because the engine's immutable configuration representation is immutable, it will never change.

[rhauch]

public void initialize() throws ConfigurationException {

        try {

            //Load configuration

            if (configuration == null) {

                configuration = new JcrConfiguration();

                InputStream inputStream = ResourceLoader.loadClasspathResource(getConfigurationFile());

                configuration.loadFrom(inputStream);

                configuration.repository(getName()).registerNamespace(GlobalContext.OEA_NS, GlobalContext.OEA_URI);

                configuration.repository(getName()).addNodeTypes("OEA_Unstructured.cnd");

                configuration.repository(getName()).addNodeTypes("OEA_CopyUnstructured.cnd");

                configuration.repository(getName()).addNodeTypes("OEA_IgnoreUnstructured.cnd");

                configuration.repository(getName()).addNodeTypes("OEA_VersionUnstructured.cnd");

                inputStream.close();

            }

            //Create engine

            engine = configuration.build();

            engine.start();

        } catch (IOException e) {

            throw new ConfigurationException("Unable to initialize \'"+getName()+"' storage.", e);

        } catch (SAXException e) {

            throw new ConfigurationException("Unable to initialize \'"+getName()+"' storage.", e);

        }

    }

When i comment the bold lines, it's ok, my engine actually has the Portal repository, but as soon as i register the namespace and add the nodetypes, no more repository in the JCREngine when i need it

That is quite strange. Keeping investigate,

Thank you for your support.

SJ.

BTW, you don't need to close the input stream to the configuration file; the "loadFrom(...)" method will always close the input stream (even in the case of an exception).

Do you get any runtime or caught exceptions when adding any of the CND files to your configuration? If so, there are two possibilities.

The first is that it's actually not finding your CND files, and it's breaking out of your try block. Given the way you specified the paths, the files are expected to be in the same directory where the VM was started, which is the same as "new java.io.File(path)". I'd recommend creating a java.io.File object, verifying the file exists, and then adding the node types by passing the File object. (BTW, it's also possible to specify the CND files in the configuration file itself; see this for an example.)

The second possibility is that you have an error in one of your CND files. This should be reported in the problems.

In either case, you should check the problems after building and even after starting (see my prior post on this thread for an example).

Best regards.

[sjahan]

Hi Randall,

Finally, i figured out the solution about my "add noteype issue".

The trouble was oea:unstructured was correctly added, but the three others were subtypes of oea:unstructured. They were processed at the same time, so oea:copyUnstructured was depending on oea:unstructured which wasn't set up yet.

The validation step throws an exception, catched deeply and silently so that no callers were aware of the problem :s

I corrected the trouble by changing my CNDs, every nodetype has now nt:base as supertype and it works fine.

So this thing is over now, i can use my node types to manage correctly the versioning!

I'd just need some tips with OPV stuff because this doesn't work yet and i don't understand why, except that i must have misunderstood something!

So i have a node Document and some fields below:

<Document>

     <Field/>

     <Field/>

     <Field/>

</Document>

The document is mix:versionable with OPV ignore but we don't care since there should not be new version above documents.

The fields are mix:referenceable with OPV copy, so if i understood correctly, when i create a new version of my document (with checkin()), it should embed the fields, right? I guess wrong, actually, when i get the frozenNode of a previous version, it's still totally empty

Hoping this is clear and you could help me to solve that matter,

SJ.

[rhauch]

Finally, i figured out the solution about my "add noteype issue".

The trouble was oea:unstructured was correctly added, but the three others were subtypes of oea:unstructured. They were processed at the same time, so oea:copyUnstructured was depending on oea:unstructured which wasn't set up yet.

The validation step throws an exception, catched deeply and silently so that no callers were aware of the problem :s

I corrected the trouble by changing my CNDs, every nodetype has now nt:base as supertype and it works fine.

So this thing is over now, i can use my node types to manage correctly the versioning!

Do you still have the exception? I think your CND would have resulted in a recursive node type (base type has children of types that extend the base type), so I wonder if it's the same thing as MODE-1050? If so, the good news is that we've already fixed that and it will be in the next release (hopefully within the next few days). If it doesn't look the same, can you log a new defect?

The bad news is that IIRC you were having trouble with certain XPath queries in 2.5.0.Beta1. If you could, please try Beta2 and if it is still a problem please log a defect, so we can hopefully get that corrected in the next few days.

I'm glad you were able to figure it out, though.

I'd just need some tips with OPV stuff because this doesn't work yet and i don't understand why, except that i must have misunderstood something!

So i have a node Document and some fields below:

<Document>

     <Field/>

     <Field/>

     <Field/>

</Document>

The document is mix:versionable with OPV ignore but we don't care since there should not be new version above documents.

The fields are mix:referenceable with OPV copy, so if i understood correctly, when i create a new version of my document (with checkin()), it should embed the fields, right? I guess wrong, actually, when i get the frozenNode of a previous version, it's still totally empty

OPV does work the way you suggest: if document is mix:versionable and has properties defined with OPV of copy, then when that document node is checked in, JCR will copy the properties as you expect.

There are two things to look at.

First, there is a quirk with JCR versioning that many people (including me) find strange. The first frozen node in the version history will be pretty empty, because it reflects the node at the time it was made versionable and the version history is initialized. Then when you call 'checkin()' the first time, a second frozen node is created in version history, and this second frozen node will contain the versionable properties and children. How are you getting to the frozen node? Can you post some code that does this?

Second, your nodes might not actually be using the property definitions you think they are. If I recall, your earlier CND used residual properties (e.g., those with a name of '*'). If your non-residual REFERENCE property definitions are not correctly defined, when you set the REFERENCE property ModeShape might not be able to match the property name, type and value with the non-residual property definition, and so it would fall back to the residual property definition. Your older CND did not define OPV for these residual property definitions, so they wouldn't be included in the version history.

Could you post your new CND?

[sjahan]

Well i don't think this is coming from the CNDs, actually properties have no OPVs, but there is no trouble about that, i only care about child nodes, not the properties.

Here is the CND:

[oea:unstructured]

orderable

- * (UNDEFINED) multiple

- * (UNDEFINED)

+ * (oea:ignoreUnstructured) = oea:ignoreUnstructured sns IGNORE

+ * (oea:versionUnstructured) = oea:versionUnstructured sns VERSION

+ * (oea:copyUnstructured) = oea:copyUnstructured sns COPY

+ * (oea:unstructured) = oea:unstructured sns

[oea:ignoreUnstructured]

orderable

- * (UNDEFINED) multiple

- * (UNDEFINED)

+ * (oea:ignoreUnstructured) = oea:ignoreUnstructured sns IGNORE

+ * (oea:versionUnstructured) = oea:versionUnstructured sns VERSION

+ * (oea:copyUnstructured) = oea:copyUnstructured sns COPY

[oea:copyUnstructured]

orderable

- * (UNDEFINED) multiple

- * (UNDEFINED)

+ * (oea:copyUnstructured) = oea:copyUnstructured sns COPY

+ * (oea:ignoreUnstructured) = oea:ignoreUnstructured sns IGNORE

[oea:versionUnstructured]

orderable

- * (UNDEFINED) multiple

- * (UNDEFINED)

+ * (oea:ignoreUnstructured) = oea:ignoreUnstructured sns IGNORE

+ * (oea:versionUnstructured) = oea:versionUnstructured sns VERSION

Here is how i get the frozen node:

public Element getVersionedDOMHierarchy(Location location, int depth, int propertiesMode, Location version) {

        Node node = getJCRNode(location);

        Node requestedVersion = getJCRNode(version);

        Version v = null;

        try {

                String versionUUID = requestedVersion.getParent().getUUID();

                VersionHistory versions = node.getVersionHistory();

                VersionIterator versionIterator = versions.getAllVersions();

                while (versionIterator.hasNext()) {

                    Version currentVersion = (Version) versionIterator.nextVersion();

                    if (currentVersion.getUUID().equals(versionUUID)) {

                        SerializableVersion sv = new SerializableVersion(context,currentVersion);

                        System.out.println(sv.getType());

                        v = currentVersion;

                        break;

                    }

                }

                node=v.getFrozenNode();

        } catch {

.....

                    }

        Element domHierarchy = getDOMHierarchyRecursively(node, depth,

                propertiesMode);

        return domHierarchy;

    }

Once i got the frozen node, i'm using my usual function, but maybe i can't access in the versioning area the same way i do with normal nodes?

In getDOMHierarchyRecursively, here is how the frozen node is processed to get the children:

NodeIterator children = node.getNodes();

     while (children.hasNext()) {

     Node childNode = (Node) children.next();

     Element childAsDOM = getDOMHierarchyRecursively(childNode, depth-1, propertiesMode);

     if (childAsDOM != null) domElement.addContent(childAsDOM);

}

This piece of code works perfectly to get back children on normal nodes, but could it work on frozenNodes too?

Thank you for your support,

SJ.

[sjahan]

I just got a clue debugging through Modeshape code.

During the checkin of my document here is what i find:

int onParentVersion = node.getDefinition().getOnParentVersion(); ---> OPV of the Document (so IGNORE)

        for (NodeIterator childNodes = node.getNodes(); childNodes.hasNext();) {

            AbstractJcrNode childNode = (AbstractJcrNode)childNodes.nextNode();

            versionNodeAt(childNode, frozenVersionPath, systemBatch, onParentVersion); ---> In this, the children aren't processed because of the IGNORE of the parent, so it's perfectly normal that i don't get them.

        }

I thought OPV meant "what to do when my parent is checked in", but here, it uses OPV of the parent for the versioning of the children. Is it the way that it is supposed to be according to the JCR 2.0 Specs? I understood from the specs that it is supposed to be managed at the child level and not at the parent level... So when i check in the node, each child reacts according to its OPV and not to its parent's OPV.

[rhauch]

I thought OPV meant "what to do when my parent is checked in", but here, it uses OPV of the parent for the versioning of the children. Is it the way that it is supposed to be according to the JCR 2.0 Specs? I understood from the specs that it is supposed to be managed at the child level and not at the parent level... So when i check in the node, each child reacts according to its OPV and not to its parent's OPV.

I think you're right. Another contributor and I just did a review of that code and of the spec, and we think there are two bugs there.

First, the last parameter of the 'versionNodeAt' call should be 'childNode.getDefinition().getOnParentVersion()', not 'onParentVersion'. So you are correct.

Second, within the 'versionNodeAt' method, in the COPY case, the onParentVersionAction variable needs to be changed to OnParentVersion.COPY. This is because of something we just noticed in the JCR specification (in bullet items 5 and 6 of Section 3.13.9), where COPY implies "a copy of the entire subgraph rooted at C (regardless of the OPV values of the sub-items) is added to the frozen node" [emphasis mine]

Interestingly (and frustratingly), ModeShape passes the JCR TCK tests before and after these changes.

So thanks for catching this! Do you want the honor of logging a defect in our JIRA, or shall I? I already have a fix ready to commit, so it definitely will be in Beta3 (out in a few days).

[rhauch]

I went ahead and logged the defect: MODE-1172

Thanks again, SJ, for finding and reporting this!