-
15. Re: Querying Repository (Filesystem Connector) return only Root node
penkween Sep 12, 2011 2:33 PM (in response to penkween)Hi Randall,
Have tried running using the standalone app (not via App server), the Query still return only the Root node where the "folder1" is not listed by the Query as shown below together with the actual and full source code and configuration xml used. If you somehow manage got the Query to return not only the Root node but also the "folder1" (nt:folder) node, pls let me know and appreciate if you can let me know the full set of the Jar libraries that you are using.
Many Thanks in advance.
Program Output
=============================================================================================
run:
02:03:33,965 INFO Started rebuilding indexes for repository 'Repo'
02:03:33,991 INFO Completed rebuilding indexes for repository 'Repo'
02:03:34,001 WARN The JAAS policy named 'modeshape-jcr' (nor the policy named 'other') could not be found; check the value of the 'JAAS_LOGIN_CONFIG_NAME' repository option in the configuration for the 'Repo' repository (or ignore this message if JAAS authentication is not being used)
02:03:34,008 INFO Completed starting the "Repo" repository
---------------------nodeDump -------------------------------------------
/
/jcr:primaryType = mode:root
/jcr:uuid = cafebabe-cafe-babe-cafe-babecafebabe
---------------------nodeDump -------------------------------------------
/folder1
/folder1/jcr:primaryType = nt:folder
---------------------nodeDump -------------------------------------------
/jcr:system
---------------------QueryDump -------------------------------------------
+---+-----------------+----------+----------+-----------+----------------+------------+------------------------------------------------------------------------------------+----------------+
| # | jcr:primaryType | jcr:path | jcr:name | jcr:score | mode:localName | mode:depth | Location(nt:base) | Score(nt:base) |
+---+-----------------+----------+----------+-----------+----------------+------------+------------------------------------------------------------------------------------+----------------+
| 1 | mode:root | / | | 1.0 | | 0 | </ && [{http://www.modeshape.org/1.0}uuid = cafebabe-cafe-babe-cafe-babecafebabe]> | 1.0 |
+---+-----------------+----------+----------+-----------+----------------+------------+------------------------------------------------------------------------------------+----------------+
BUILD SUCCESSFUL (total time: 2 seconds)
*Above showing the NodeDump using getRootNode() return the Root and the "folder1" nodes where the QueryDump only return the Root node.
Application Directory Structure
=============================================================================================
Since using relative path, during app run time, has set a working directory contain the files structure as below:
[Working Directory] /repository/indexes/
[Working Directory] /repository/workspace1/folder1/
[Working Directory] /modeshape-config.xml
[Working Directory] /log4j.properties
modeshape-config.xml
=============================================================================================
<?xml version="1.0" encoding="UTF-8"?>
<configuration xmlns:mode="http://www.modeshape.org/1.0" xmlns:jcr="http://www.jcp.org/jcr/1.0">
<mode:repositories>
<mode:repository jcr:name="Repo" mode:source="Store">
<mode:options jcr:primaryType="mode:options">
<mode:option jcr:name="rebuildQueryIndexOnStartup" mode:value="ifMissing"/>
<mode:option jcr:name="queryIndexDirectory" mode:value="./repository/indexes" />
</mode:options>
</mode:repository>
</mode:repositories>
<mode:sources>
<mode:source jcr:name="Store" mode:classname="org.modeshape.connector.filesystem.FileSystemSource"
mode:workspaceRootPath="./repository"
mode:defaultWorkspaceName="workspace1"
mode:creatingWorkspacesAllowed="false"
mode:extraPropertiesBehavior="store"
mode:updatesAllowed="true"
/>
</mode:sources>
</configuration>
TestModeshapeLocal.java
=============================================================================================
package testmodeshapelocal;
import java.io.IOException;
import java.net.URL;
import java.util.Collections;
import java.util.Map;
import javax.jcr.Node;
import javax.jcr.NodeIterator;
import javax.jcr.PropertyIterator;
import javax.jcr.Repository;
import javax.jcr.RepositoryException;
import javax.jcr.Session;
import org.modeshape.jcr.JcrConfiguration;
import org.modeshape.jcr.JcrEngine;
import org.xml.sax.SAXException;
import javax.jcr.Property;
import javax.jcr.Value;
import javax.jcr.query.Query;
import javax.jcr.query.QueryManager;
import javax.jcr.query.QueryResult;
import org.modeshape.jcr.api.AnonymousCredentials;
public class TestModeshapeLocal {
public static void main(String[] args) {
JcrConfiguration config = new JcrConfiguration();
try {
String url = "modeshape-config.xml";
config.loadFrom(url);
} catch (SAXException e) {
System.err.println("Failed to read the configuration file");
} catch (IOException e) {
System.err.println("Failed to load the configuration file");
}
// Create and start the engine ...
JcrEngine engine = config.build();
if (engine.getProblems().hasErrors()) {
System.err.println("Problems starting the engine.");
System.err.println(engine.getProblems());
System.exit(-1);
}
// Start the engine
engine.start();
Repository repository = null;
Session session = null;
try {
// Get the repository
repository = engine.getRepository("Repo");
// Create a session with anonymous access
session = repository.login(new AnonymousCredentials(),"workspace1");
//Try to list all node by iterator
Node node = session.getRootNode();
nodeTreeDump(node);
//Try to list all node by query
String language = null;
String expression = null;
Query query = null;
QueryResult result = null;
language = "JCR-SQL2";
expression = "SELECT * FROM [nt:base]";
query = session.getWorkspace().getQueryManager().createQuery(expression, language);
result = query.execute();
System.out.println("\n---------------------QueryDump -------------------------------------------");
System.out.println(result.toString());
} catch (RepositoryException e) {
System.err.println("Failed to load the configuration file");
} finally {
if (session != null) {
session.logout();
}
engine.shutdown();
}
}
private static void nodeTreeDump(Node node) throws RepositoryException {
System.out.println("\n---------------------nodeDump -------------------------------------------");
// First output the node path
System.out.println(node.getPath());
// Skip the virtual (and large!) jcr:system subtree
if (node.getName().equals("jcr:system")) {
return;
}
// Then output the properties
PropertyIterator properties = node.getProperties();
while (properties.hasNext()) {
Property property = properties.nextProperty();
if (property.getDefinition().isMultiple()) {
// A multi-valued property, print all values
Value[] values = property.getValues();
for (int i = 0; i < values.length; i++) {
System.out.println(
property.getPath() + " = " + values[i].getString());
}
} else {
// A single-valued property
System.out.println(
property.getPath() + " = " + property.getString());
}
}
// Finally output all the child nodes recursively
NodeIterator nodes = node.getNodes();
while (nodes.hasNext()) {
nodeTreeDump(nodes.nextNode());
}
}
}
log4j.properties
=============================================================================================
# Direct log messages to stdout
log4j.appender.stdout=org.apache.log4j.ConsoleAppender
log4j.appender.stdout.Target=System.out
log4j.appender.stdout.layout=org.apache.log4j.PatternLayout
log4j.appender.stdout.layout.ConversionPattern=%d{ABSOLUTE} %5p %m%n
# Root logger option
log4j.rootLogger=INFO, stdout
# Set up the default logging to be INFO level, then override specific units
log4j.logger.org.modeshape=INFO
Libraries jar included with the App (built using latest Modeshape Git source)
=============================================================================================
antlr-2.7.6.jar
c3p0-0.9.1.jar
commons-collections-3.1.jar
commons-logging-1.1.1.jar
dom4j-1.6.1.jar
hibernate-annotations-3.5.2-Final.jar
hibernate-c3p0-3.5.2-Final.jar
hibernate-commons-annotations-3.2.0.Final.jar
hibernate-core-3.5.2-Final.jar
hibernate-entitymanager-3.5.2-Final.jar
hibernate-jpa-2.0-api-1.0.0.Final.jar
hibernate-validator-3.0.0.ga.jar
jboss-common-core-2.2.14.GA.jar
jboss-jaspi-api-1.0.0.GA.jar
jboss-logging-spi-2.1.1.GA.jar
jboss-reflect-2.0.2.GA.jar
jboss-transaction-api-1.0.1.GA.jar
jbosscache-core-3.2.4.GA.jar
jbosssx-bare-3.0.0.Final.jar
jbossxacml-2.0.4.jar
jbossxb-2.0.1.GA.jar
jcr-2.0.jar
joda-time-1.6.jar
log4j-1.2.16.jar
lucene-analyzers-3.1.0.jar
lucene-core-3.1.0.jar
lucene-misc-3.1.0.jar
lucene-regex-3.0.3.jar
modeshape-cnd-2.6-SNAPSHOT.jar
modeshape-common-2.6-SNAPSHOT.jar
modeshape-connector-disk-2.6-SNAPSHOT.jar
modeshape-connector-filesystem-2.6-SNAPSHOT.jar
modeshape-connector-jcr-2.6-SNAPSHOT.jar
modeshape-connector-store-jpa-2.6-SNAPSHOT.jar
modeshape-graph-2.6-SNAPSHOT.jar
modeshape-jcr-2.6-SNAPSHOT.jar
modeshape-jcr-api-2.6-SNAPSHOT-jar-with-dependencies.jar
modeshape-repository-2.6-SNAPSHOT.jar
modeshape-search-lucene-2.6-SNAPSHOT.jar
slf4j-api-1.6.1.jar
slf4j-log4j12-1.6.1.jar
-
16. Re: Querying Repository (Filesystem Connector) return only Root node
rhauch Sep 12, 2011 4:40 PM (in response to penkween)The indexes for a repository are not built until the Repository instance is first needed, which in this case is the following line:
repository = engine.getRepository("Repo");
By default, indexing is done asynchronously, which means the re-indexing might not be complete by the time the query is performed. Though it's ugly, try putting a "Thread.sleep(3000)" call before you execute the query. Also, I'd suggest putting a small text file with a line or two of contents somewhere under the "./repository/workspace1" folder. Maybe just folders are a problem?
-
17. Re: Querying Repository (Filesystem Connector) return only Root node
penkween Sep 13, 2011 3:32 AM (in response to penkween)Hi Randall,
I have tried "Thread.sleep(3000)" for 3 & even 30 seconds and putting a 2kb text file inside "folder1" & even Root folder. The problem still the same, only Root node is returned. Very strange. I am using Windows(Vista) for this test, later will test it with Linux (Ubuntu) see if it can work.
Btw, for this Intergration test -> [modeshape] modeshape-integration-tests/src/test/java/org/modeshape/test/integration/filesystem/FileSystemRepositoryIntegrationTest.java . I am not quite a Maven guy, I am just using IDE to develop my app, so far I just manage to build Modeshape from source without Unit Test (-DskipTests) because the whole things usually fail somewhere with Unit Test included. How do I properly carry out the unit test , what specific maven command is needed in order to verify if this particular test is running well ?
Thank.
-
18. Re: Querying Repository (Filesystem Connector) return only Root node
rhauch Sep 13, 2011 9:05 AM (in response to penkween)Btw, for this Intergration test -> [modeshape] modeshape-integration-tests/src/test/java/org/modeshape/test/integration/filesystem/FileSystemRepositoryIntegrationTest.java . I am not quite a Maven guy, I am just using IDE to develop my app, so far I just manage to build Modeshape from source without Unit Test (-DskipTests) because the whole things usually fail somewhere with Unit Test included. How do I properly carry out the unit test , what specific maven command is needed in order to verify if this particular test is running well ?
You should be able to run a full integration build at the command-line using Maven 3.0 and this command:
mvn clean install -Pintegration
More details on this are here, and more information about setting up some of the tools is here.
Most of the time, build problems are related to having the wrong version of Maven installed (2.x) or an older JDK. And although many of us use OS X or Linux, we do have periodic CI builds on Windows, and several community members use Windows (don't know which variant) and are able to build locally. Now, having said all that, you may indeed run into real problems building on Windows, so please let us know what you're seeing.
BTW, we are tracking down a unit test in 'web/modeshape-web-jcr-rest-client' that fails on some boxes but not others.
-
19. Re: Querying Repository (Filesystem Connector) return only Root node
penkween Sep 13, 2011 10:11 AM (in response to penkween)Hi Randall,
Managed to isolate and now unit testing only the following with a new Maven's reactor using "mvn clean install -e -pl :modeshape-integration-tests -s settings.xml"
modeshape-integration-tests\src\test\java\org\modeshape\test\integration\filesystem\FileSystemRepositoryIntegrationTest.java
I am getting some lead, will update in a short while.
-
20. Re: Querying Repository (Filesystem Connector) return only Root node
penkween Sep 13, 2011 1:06 PM (in response to penkween)Hi Randall,
\modeshape-integration-tests\src\test\java\org\modeshape\test\integration\filesystem\FileSystemRepositoryIntegrationTest.java
Have just unit tested the above FileSystemRepositoryIntegrationTest.java. Although it seems like @Test shouldFindAllNodesWhenQueryingContent() print out the whole nodes tree, this unit test pass because actually it is only addressing those new nodes created after session started, it DOES NOT address existing nodes inside the repository since before every test, the repository is first deleted in "beforeEach()"
To illustrate the issue, we can just add some folders into the repository before running the unit @Test shouldFindAllNodesWhenQueryingContent() as below:
@Override
public void beforeEach() throws Exception {
// Delete the directory used for the FS store ...
FileUtil.delete("target/fsRepoWithProps");
// Now make the root directory ...
new File("target/fsRepoWithProps/root").mkdirs();
new File("target/fsRepoWithProps/root/defaultWorkspace/folder1").mkdirs();
new File("target/fsRepoWithProps/root/defaultWorkspace/folder2").mkdirs();
super.beforeEach();
}
Then we run the unit @Test shouldFindAllNodesWhenQueryingContent(), below is what print=true; printQuery("SELECT * FROM [nt:base]", 5L, Collections.<String, String>emptyMap()); print out:
SELECT * FROM [nt:base]
+---+-----------------+---------------------------+-------------+-----------+----------------+------------+-------------------------------------------
---------------------------------------------------------------------------------------------+----------------+
| # | jcr:primaryType | jcr:path | jcr:name | jcr:score | mode:localName | mode:depth | Location(nt:base)
| Score(nt:base) |
+---+-----------------+---------------------------+-------------+-----------+----------------+------------+-------------------------------------------
---------------------------------------------------------------------------------------------+----------------+
| 1 | mode:root | / | | 1.0 | | 0 | </ && [{http://www.modeshape.org/1.0}uuid
= cafebabe-cafe-babe-cafe-babecafebabe]> | 1.0 |
| 2 | nt:folder | /meta | meta | 1.0 | meta | 1 | /{}meta
| 1.0 |
| 3 | nt:folder | /A | A | 1.0 | A | 1 | /{}A
| 1.0 |
| 4 | nt:file | /A/README.txt | README.txt | 1.0 | README.txt | 2 | /{}A/{}README.txt
| 1.0 |
| 5 | nt:resource | /A/README.txt/jcr:content | jcr:content | 1.0 | content | 3 | </{}A/{}README.txt/{http://www.jcp.org/jcr
/1.0}content && [{http://www.modeshape.org/1.0}uuid = 1e4fa9f7-3484-4409-8357-df42b60f3f8b]> | 1.0 |
+---+-----------------+---------------------------+-------------+-----------+----------------+------------+-------------------------------------------
---------------------------------------------------------------------------------------------+----------------+
As shown above, we won't find the "folder1" or "folder2", we can even put some content file into "folder1" or "folder2", it won't show either. So, all existing nodes (except Root node) before session start won't be addressed by the Query. This behavior is exactly the same to some previous test done highlighted in some earlier discussion in this thread. Putting Thread.sleep(3000) before Query.execute() won't help either.
Pls help . Thanks.
-
21. Re: Querying Repository (Filesystem Connector) return only Root node
rhauch Sep 13, 2011 12:03 PM (in response to penkween)...this unit test pass because actually it is only addressing those new nodes created after session started, it DOES NOT address existing nodes inside the repository since before every test, the repository is first deleted in "beforeEach()"
I totally missed this distinction. Would you mind logging a (blocker) defect in our JIRA? I can take a look to figure out what's going on, and hopefully we'll have it fixed shortly.
-
22. Re: Querying Repository (Filesystem Connector) return only Root node
penkween Sep 13, 2011 12:50 PM (in response to penkween) -
23. Re: Querying Repository (Filesystem Connector) return only Root node
rhauch Sep 14, 2011 10:37 AM (in response to penkween)I just committed the fix for MODE-1263 to the 'master' branch, so it will be in the upcoming '2.6.0.Final' release.
Danny, thanks again for doing all the work to track down the failure scenario and creating an integration test to reproduce the problem!
-
24. Re: Querying Repository (Filesystem Connector) return only Root node
penkween Sep 14, 2011 10:57 AM (in response to penkween)Hi Randall,
I really would like to express my big thank to you in fixing this issue, now we can proceed to implement our project using Modeshape filesystem connectors. I am rebuilding the modeshape jar now and will proceed to the integration test. I did look into the modeshape source trying to to hack down the issue but fail perhaps lacking understanding of the big picture of Modeshape. Appreciate your time very much.
Thank.
-
25. Re: Querying Repository (Filesystem Connector) return only Root node
rhauch Sep 14, 2011 11:18 AM (in response to penkween)I'm glad we were able to fix it for you. I wish it didn't take as long to track down the problem, but your test case nailed it. Don't worry about not being able to find the cause of the problem - ModeShape is a fairly complex beast.
BTW, if this doesn't fix the problem for you, please reopen the JIRA issue ASAP.
-
26. Re: Querying Repository (Filesystem Connector) return only Root node
penkween Sep 14, 2011 12:36 PM (in response to penkween)Yeah, tested with both standalone and app server, this fix the issue. Great. Thanks.