1 2 Previous Next 26 Replies Latest reply on Sep 14, 2011 12:36 PM by Danny C Go to original post
      • 15. Re: Querying Repository (Filesystem Connector) return only Root node
        Danny C Newbie

        Hi Randall,

         

              Have tried running using the standalone app (not via App server), the Query still return only the Root node where the "folder1" is not listed by the Query as shown below together with the actual and full source code and configuration xml used. If you somehow manage got the Query to return not only the Root node but also the "folder1" (nt:folder) node, pls let me know and appreciate if you can let me know the full set of the Jar libraries that you are using.

         

        Many Thanks in advance.

         

        Program Output

        =============================================================================================

        run:

        02:03:33,965  INFO Started rebuilding indexes for repository 'Repo'

        02:03:33,991  INFO Completed rebuilding indexes for repository 'Repo'

        02:03:34,001  WARN The JAAS policy named 'modeshape-jcr' (nor the policy named 'other') could not be found; check the value of the 'JAAS_LOGIN_CONFIG_NAME' repository option in the configuration for the 'Repo' repository (or ignore this message if JAAS authentication is not being used)

        02:03:34,008  INFO Completed starting the "Repo" repository

         

        ---------------------nodeDump -------------------------------------------

        /

        /jcr:primaryType = mode:root

        /jcr:uuid = cafebabe-cafe-babe-cafe-babecafebabe

         

        ---------------------nodeDump -------------------------------------------

        /folder1

        /folder1/jcr:primaryType = nt:folder

         

        ---------------------nodeDump -------------------------------------------

        /jcr:system

         

        ---------------------QueryDump -------------------------------------------

        +---+-----------------+----------+----------+-----------+----------------+------------+------------------------------------------------------------------------------------+----------------+

        | # | jcr:primaryType | jcr:path | jcr:name | jcr:score | mode:localName | mode:depth | Location(nt:base)                                                                  | Score(nt:base) |

        +---+-----------------+----------+----------+-----------+----------------+------------+------------------------------------------------------------------------------------+----------------+

        | 1 | mode:root       | /        |          | 1.0       |                | 0          | </ && [{http://www.modeshape.org/1.0}uuid = cafebabe-cafe-babe-cafe-babecafebabe]> | 1.0            |

        +---+-----------------+----------+----------+-----------+----------------+------------+------------------------------------------------------------------------------------+----------------+

        BUILD SUCCESSFUL (total time: 2 seconds)

         

        *Above showing the NodeDump using getRootNode() return the Root and the "folder1" nodes where the QueryDump only return the Root node.

         

         

         

        Application Directory Structure

        =============================================================================================

        Since using relative path, during app run time, has set a working directory contain the files structure as below:


        [Working Directory] /repository/indexes/

        [Working Directory] /repository/workspace1/folder1/

        [Working Directory] /modeshape-config.xml

        [Working Directory] /log4j.properties


         

         

        modeshape-config.xml

        =============================================================================================

        <?xml version="1.0" encoding="UTF-8"?>

        <configuration xmlns:mode="http://www.modeshape.org/1.0" xmlns:jcr="http://www.jcp.org/jcr/1.0">

            <mode:repositories>

                <mode:repository jcr:name="Repo" mode:source="Store">

                    <mode:options jcr:primaryType="mode:options">

                        <mode:option jcr:name="rebuildQueryIndexOnStartup" mode:value="ifMissing"/>

                        <mode:option jcr:name="queryIndexDirectory" mode:value="./repository/indexes" />

                    </mode:options>

                </mode:repository>

            </mode:repositories>

         

            <mode:sources>

                <mode:source jcr:name="Store" mode:classname="org.modeshape.connector.filesystem.FileSystemSource"

                    mode:workspaceRootPath="./repository"

                    mode:defaultWorkspaceName="workspace1"

                    mode:creatingWorkspacesAllowed="false"

                    mode:extraPropertiesBehavior="store"

                    mode:updatesAllowed="true"

                    />   

            </mode:sources>

          </configuration>

         

         

        TestModeshapeLocal.java

        =============================================================================================

        package testmodeshapelocal;

         

        import java.io.IOException;

        import java.net.URL;

        import java.util.Collections;

        import java.util.Map;

        import javax.jcr.Node;

        import javax.jcr.NodeIterator;

        import javax.jcr.PropertyIterator;

        import javax.jcr.Repository;

        import javax.jcr.RepositoryException;

        import javax.jcr.Session;

        import org.modeshape.jcr.JcrConfiguration;

        import org.modeshape.jcr.JcrEngine;

        import org.xml.sax.SAXException;

        import javax.jcr.Property;

        import javax.jcr.Value;

        import javax.jcr.query.Query;

        import javax.jcr.query.QueryManager;

        import javax.jcr.query.QueryResult;

        import org.modeshape.jcr.api.AnonymousCredentials;

         

        public class TestModeshapeLocal {

         

            public static void main(String[] args) {

                JcrConfiguration config = new JcrConfiguration();

                try {

                    String url = "modeshape-config.xml";

                    config.loadFrom(url);

                } catch (SAXException e) {

                    System.err.println("Failed to read the configuration file");

                } catch (IOException e) {

                    System.err.println("Failed to load the configuration file");

                }

         

                // Create and start the engine ...

                JcrEngine engine = config.build();

                if (engine.getProblems().hasErrors()) {

                    System.err.println("Problems starting the engine.");

                    System.err.println(engine.getProblems());

                    System.exit(-1);

                }

         

                // Start the engine

                engine.start();

         

                Repository repository = null;

                Session session = null;

                try {

                    // Get the repository

                    repository = engine.getRepository("Repo");

         

                    // Create a session with anonymous access

                    session = repository.login(new AnonymousCredentials(),"workspace1");

         

                    //Try to list all node by iterator

                    Node node = session.getRootNode();

                    nodeTreeDump(node);

         

                    //Try to list all node by query

                    String language = null;

                    String expression = null;

                    Query query = null;

                    QueryResult result = null;

                    language = "JCR-SQL2";

                    expression = "SELECT * FROM [nt:base]";

                    query = session.getWorkspace().getQueryManager().createQuery(expression, language);

                    result = query.execute();

                    System.out.println("\n---------------------QueryDump -------------------------------------------");

                    System.out.println(result.toString());

         

                } catch (RepositoryException e) {

                    System.err.println("Failed to load the configuration file");

                } finally {

                    if (session != null) {

                        session.logout();

                    }

                    engine.shutdown();

                }

            }

         

            private static void nodeTreeDump(Node node) throws RepositoryException {

                System.out.println("\n---------------------nodeDump -------------------------------------------");

                // First output the node path

                System.out.println(node.getPath());

                // Skip the virtual (and large!) jcr:system subtree

                if (node.getName().equals("jcr:system")) {

                    return;

                }

         

                // Then output the properties

                PropertyIterator properties = node.getProperties();

                while (properties.hasNext()) {

                    Property property = properties.nextProperty();

                    if (property.getDefinition().isMultiple()) {

                        // A multi-valued property, print all values

                        Value[] values = property.getValues();

                        for (int i = 0; i < values.length; i++) {

                            System.out.println(

                                    property.getPath() + " = " + values[i].getString());

                        }

                    } else {

                        // A single-valued property

                        System.out.println(

                                property.getPath() + " = " + property.getString());

                    }

                }

         

                // Finally output all the child nodes recursively

                NodeIterator nodes = node.getNodes();

                while (nodes.hasNext()) {

                    nodeTreeDump(nodes.nextNode());

                }

            }

        }

         

         

        log4j.properties

        =============================================================================================

        # Direct log messages to stdout

        log4j.appender.stdout=org.apache.log4j.ConsoleAppender

        log4j.appender.stdout.Target=System.out

        log4j.appender.stdout.layout=org.apache.log4j.PatternLayout

        log4j.appender.stdout.layout.ConversionPattern=%d{ABSOLUTE} %5p %m%n

         

        # Root logger option

        log4j.rootLogger=INFO, stdout

         

        # Set up the default logging to be INFO level, then override specific units

        log4j.logger.org.modeshape=INFO

         

         

        Libraries jar included with the App (built using latest Modeshape Git source)

        =============================================================================================

        antlr-2.7.6.jar

        c3p0-0.9.1.jar

        commons-collections-3.1.jar

        commons-logging-1.1.1.jar

        dom4j-1.6.1.jar

        hibernate-annotations-3.5.2-Final.jar

        hibernate-c3p0-3.5.2-Final.jar

        hibernate-commons-annotations-3.2.0.Final.jar

        hibernate-core-3.5.2-Final.jar

        hibernate-entitymanager-3.5.2-Final.jar

        hibernate-jpa-2.0-api-1.0.0.Final.jar

        hibernate-validator-3.0.0.ga.jar

        jboss-common-core-2.2.14.GA.jar

        jboss-jaspi-api-1.0.0.GA.jar

        jboss-logging-spi-2.1.1.GA.jar

        jboss-reflect-2.0.2.GA.jar

        jboss-transaction-api-1.0.1.GA.jar

        jbosscache-core-3.2.4.GA.jar

        jbosssx-bare-3.0.0.Final.jar

        jbossxacml-2.0.4.jar

        jbossxb-2.0.1.GA.jar

        jcr-2.0.jar

        joda-time-1.6.jar

        log4j-1.2.16.jar

        lucene-analyzers-3.1.0.jar

        lucene-core-3.1.0.jar

        lucene-misc-3.1.0.jar

        lucene-regex-3.0.3.jar

        modeshape-cnd-2.6-SNAPSHOT.jar

        modeshape-common-2.6-SNAPSHOT.jar

        modeshape-connector-disk-2.6-SNAPSHOT.jar

        modeshape-connector-filesystem-2.6-SNAPSHOT.jar

        modeshape-connector-jcr-2.6-SNAPSHOT.jar

        modeshape-connector-store-jpa-2.6-SNAPSHOT.jar

        modeshape-graph-2.6-SNAPSHOT.jar

        modeshape-jcr-2.6-SNAPSHOT.jar

        modeshape-jcr-api-2.6-SNAPSHOT-jar-with-dependencies.jar

        modeshape-repository-2.6-SNAPSHOT.jar

        modeshape-search-lucene-2.6-SNAPSHOT.jar

        slf4j-api-1.6.1.jar

        slf4j-log4j12-1.6.1.jar

        • 16. Re: Querying Repository (Filesystem Connector) return only Root node
          Randall Hauch Master

          The indexes for a repository are not built until the Repository instance is first needed, which in this case is the following line:

                      repository = engine.getRepository("Repo");

           

          By default, indexing is done asynchronously, which means the re-indexing might not be complete by the time the query is performed. Though it's ugly, try putting a "Thread.sleep(3000)" call before you execute the query. Also, I'd suggest putting a small text file with a line or two of contents somewhere under the "./repository/workspace1" folder. Maybe just folders are a problem?

          • 17. Re: Querying Repository (Filesystem Connector) return only Root node
            Danny C Newbie

            Hi Randall,

             

                      I have tried "Thread.sleep(3000)"  for 3 & even 30 seconds and putting a 2kb text file inside "folder1" &  even Root folder. The problem still the same, only Root node is returned. Very strange. I am using Windows(Vista) for this test, later will test it with Linux (Ubuntu) see if it can work.

             

                      Btw, for this Intergration test ->   [modeshape] modeshape-integration-tests/src/test/java/org/modeshape/test/integration/filesystem/FileSystemRepositoryIntegrationTest.java . I am not quite a Maven guy, I am just using IDE to develop my app, so far I just manage to build Modeshape from source without Unit Test (-DskipTests) because the whole things usually fail somewhere with Unit Test included. How do I properly carry out the unit test , what specific maven command is needed in order to verify if this particular test is running well ?

             

                      Thank.

            • 18. Re: Querying Repository (Filesystem Connector) return only Root node
              Randall Hauch Master

                        Btw, for this Intergration test ->   [modeshape] modeshape-integration-tests/src/test/java/org/modeshape/test/integration/filesystem/FileSystemRepositoryIntegrationTest.java . I am not quite a Maven guy, I am just using IDE to develop my app, so far I just manage to build Modeshape from source without Unit Test (-DskipTests) because the whole things usually fail somewhere with Unit Test included. How do I properly carry out the unit test , what specific maven command is needed in order to verify if this particular test is running well ?

               

              You should be able to run a full integration build at the command-line using Maven 3.0 and this command:

               

              mvn clean install -Pintegration

               

              More details on this are here, and more information about setting up some of the tools is here.

               

              Most of the time, build problems are related to having the wrong version of Maven installed (2.x) or an older JDK. And although many of us use OS X or Linux, we do have periodic CI builds on Windows, and several community members use Windows (don't know which variant) and are able to build locally. Now, having said all that, you may indeed run into real problems building on Windows, so please let us know what you're seeing.

               

              BTW, we are tracking down a unit test in 'web/modeshape-web-jcr-rest-client' that fails on some boxes but not others.

              • 19. Re: Querying Repository (Filesystem Connector) return only Root node
                Danny C Newbie

                Hi Randall,

                 

                    Managed to isolate and now unit testing only the following with a new Maven's reactor using "mvn clean install -e -pl :modeshape-integration-tests -s settings.xml"

                 

                modeshape-integration-tests\src\test\java\org\modeshape\test\integration\filesystem\FileSystemRepositoryIntegrationTest.java

                 

                I am getting some lead, will update in a short while.

                • 20. Re: Querying Repository (Filesystem Connector) return only Root node
                  Danny C Newbie

                  Hi Randall,

                   

                  \modeshape-integration-tests\src\test\java\org\modeshape\test\integration\filesystem\FileSystemRepositoryIntegrationTest.java

                   

                       Have just unit tested the above FileSystemRepositoryIntegrationTest.java. Although it seems like @Test shouldFindAllNodesWhenQueryingContent() print out the whole nodes tree, this unit test pass because actually it is only addressing those new nodes created after session started, it DOES NOT address existing nodes inside the repository since before every test, the repository is first deleted in "beforeEach()"

                   

                      To illustrate the issue, we can just add some folders into the repository before running the unit @Test shouldFindAllNodesWhenQueryingContent() as below:

                   

                   

                  @Override

                      public void beforeEach() throws Exception {

                          // Delete the directory used for the FS store ...

                          FileUtil.delete("target/fsRepoWithProps");

                   

                          // Now make the root directory ...

                          new File("target/fsRepoWithProps/root").mkdirs();

                          new File("target/fsRepoWithProps/root/defaultWorkspace/folder1").mkdirs();

                          new File("target/fsRepoWithProps/root/defaultWorkspace/folder2").mkdirs();

                   

                          super.beforeEach();

                      }

                   

                  Then we run the unit @Test shouldFindAllNodesWhenQueryingContent(), below is what print=true; printQuery("SELECT * FROM [nt:base]", 5L, Collections.<String, String>emptyMap()); print out:

                   

                   

                  SELECT * FROM [nt:base]

                  +---+-----------------+---------------------------+-------------+-----------+----------------+------------+-------------------------------------------

                  ---------------------------------------------------------------------------------------------+----------------+

                  | # | jcr:primaryType | jcr:path                  | jcr:name    | jcr:score | mode:localName | mode:depth | Location(nt:base)

                                                                                                               | Score(nt:base) |

                  +---+-----------------+---------------------------+-------------+-----------+----------------+------------+-------------------------------------------

                  ---------------------------------------------------------------------------------------------+----------------+

                  | 1 | mode:root       | /                         |             | 1.0       |                | 0          | </ && [{http://www.modeshape.org/1.0}uuid

                  = cafebabe-cafe-babe-cafe-babecafebabe]>                                                     | 1.0            |

                  | 2 | nt:folder       | /meta                     | meta        | 1.0       | meta           | 1          | /{}meta

                                                                                                               | 1.0            |

                  | 3 | nt:folder       | /A                        | A           | 1.0       | A              | 1          | /{}A

                                                                                                               | 1.0            |

                  | 4 | nt:file         | /A/README.txt             | README.txt  | 1.0       | README.txt     | 2          | /{}A/{}README.txt

                                                                                                               | 1.0            |

                  | 5 | nt:resource     | /A/README.txt/jcr:content | jcr:content | 1.0       | content        | 3          | </{}A/{}README.txt/{http://www.jcp.org/jcr

                  /1.0}content && [{http://www.modeshape.org/1.0}uuid = 1e4fa9f7-3484-4409-8357-df42b60f3f8b]> | 1.0            |

                  +---+-----------------+---------------------------+-------------+-----------+----------------+------------+-------------------------------------------

                  ---------------------------------------------------------------------------------------------+----------------+

                   

                   

                         As shown above, we won't find the "folder1" or "folder2", we can even put some content file into "folder1" or "folder2", it won't show either. So, all existing nodes (except Root node) before session start won't be addressed by the Query. This behavior is exactly the same to some previous test done highlighted in some earlier discussion in this thread. Putting Thread.sleep(3000) before Query.execute() won't help either.

                   

                       Pls help . Thanks.

                  • 21. Re: Querying Repository (Filesystem Connector) return only Root node
                    Randall Hauch Master

                    ...this unit test pass because actually it is only addressing those new nodes created after session started, it DOES NOT address existing nodes inside the repository since before every test, the repository is first deleted in "beforeEach()"

                    I totally missed this distinction. Would you mind logging a (blocker) defect in our JIRA? I can take a look to figure out what's going on, and hopefully we'll have it fixed shortly.

                    • 22. Re: Querying Repository (Filesystem Connector) return only Root node
                      Danny C Newbie

                      Hi Randall,

                       

                      Just created the JIRA entry: MODE-1263

                       

                      Thanks.

                      • 23. Re: Querying Repository (Filesystem Connector) return only Root node
                        Randall Hauch Master

                        I just committed the fix for MODE-1263 to the 'master' branch, so it will be in the upcoming '2.6.0.Final' release.

                         

                        Danny, thanks again for doing all the work to track down the failure scenario and creating an integration test to reproduce the problem!

                        • 24. Re: Querying Repository (Filesystem Connector) return only Root node
                          Danny C Newbie

                          Hi Randall,

                           

                                I really would like to express my big thank to you in fixing this issue,  now we can proceed to implement our project using Modeshape filesystem connectors.  I am rebuilding the modeshape jar now and will proceed to the integration test. I did  look into the modeshape source trying to to hack down the issue but fail perhaps lacking understanding of the big picture of Modeshape.  Appreciate your time very much.

                           

                          Thank.

                          • 25. Re: Querying Repository (Filesystem Connector) return only Root node
                            Randall Hauch Master

                            I'm glad we were able to fix it for you. I wish it didn't take as long to track down the problem, but your test case nailed it. Don't worry about not being able to find the cause of the problem - ModeShape is a fairly complex beast.

                             

                            BTW, if this doesn't fix the problem for you, please reopen the JIRA issue ASAP.

                            • 26. Re: Querying Repository (Filesystem Connector) return only Root node
                              Danny C Newbie

                              Yeah, tested with both standalone and app server, this fix the issue. Great. Thanks.

                              1 2 Previous Next