Programmatic configuration of workspace projections

[kkrauth]

Hi, first and foremost I'm a complete beginner with ModeShape. I was looking at the example projects from the getting started guide and got stuck while attempting to programatically configure projections for a federated repository source.

The example application creates a federated repository for Vehicles using a plain string within the XML configuration file:

<mode:projection jcr:name="Cars projection" mode:source="Cars" mode:workspaceName="workspace1">

          <mode:projectionRules>/Vehicles/Cars => /Cars</mode:projectionRules>

</mode:projection>

Since I will be creating repositories dynamically based on information coming from a database, I must do everything programatically. Looking at the Projection class API, the constructor looks as follows:

Projection(String sourceName, String workspaceName, boolean readOnly, Projection.Rule... rules)

The varargs are of type Projection.Rule, the constructor of which requires two Path instances (repository and source paths). The only problem with this is, I don't see a way to obtain an instance of a Path during config time. The PathFactory class, and all other methods that return Paths depend on having an instance of JcrEngine (or the ExecutionContext) which can only be obtained after the configuration has been built.

Does this mean I need to initialize all the underlying repositories before creating the federated one? Is that how things get executed when the XML configuration is loaded, or am I missing something obvious?

Thanks in advance!

[rhauch]

You can create an ExecutionContext instance, and then pass it to the JcrConfiguration constructor as well as use it in the ProjectionParser.rulesFromString(...) method.

Best regards

[kkrauth]

Hi Randall,

thanks for that tip! I managed to construct a valid Rule using the ProjectionParser and the default ExecutionContext. Code below for those that might get stuck trying to do the same:

ProjectionParser pp = ProjectionParser.getInstance();
Rule rule = pp.ruleFromString("/stagein/project1 => /", ExecutionContext.DEFAULT_CONTEXT);
ArrayList<Projection> projections = new ArrayList<Projection>();
projections.add(new Projection("project1", "sources", true, rule));

However, I'm still struggling with mapping the federated repository XML configuration from the example project to its programmatic equivalent. I tried initially with an approach such as this:

config.repositorySource("stagein_sources")
            .usingClass(FederatedRepositorySource.class)
            .setProperty("defaultWorkspaceName", "virtual")
            .setProperty("projections", projections.toArray());

config.repository("stagein")
            .setSource("stagein_sources");

Configuring a federated repository in such a manner feels somewhat hacky, especially since I haven't found a full list of available properties in the documentation yet. Indeed it doesn't work either, quitting with the following config exception:

Error starting the "stagein" repository (check the configuration): null

What I tried instead, is creating an instance of FederatedRepositorySource myself as follows:

FederatedRepositorySource frs = new FederatedRepositorySource();
frs.setName("stagein_source"); 
frs.addWorkspace("virtual", projections, true); 
config.loadFrom(frs);

This doesn't work either, and the error message indicates it's because the repository source is not initialized:

java.lang.IllegalArgumentException: Internationalization field "federatedRepositorySourceMustBeInitialized" in class org.modeshape.graph.GraphI18n: 2 parameters supplied, but 1 parameter required: "The "{0}" federated repository source must be initialized before it can be used" => "The "name"

However, the initialize() method takes a RepositoryContext as a parameter, indicating that this should be used with a repository that's already been set up and is running. The documentation also suggest this by saying that JcrConfiguration.loadFrom() method should actually be used with a running repository instance, and that it will also overwrite any existing configuration within the config variable. That tells me that such an approach is likely not suitable for configuring a federated repository at application startup?

I'm unsure which approach is preferred, or possible. The reference guide provides configuration examples using XML so unfortunately it doesn't help me. I'm sorry if this is a little verbose, I'm trying to relay my thought process, and hopefully the end result will be a useful resource for other ModeShape initiates such as myself

[rhauch]

Configuring a federated repository in such a manner feels somewhat hacky, especially since I haven't found a full list of available properties in the documentation yet.

There actually aren't that many properties on a federated repository source, because all of the information is in the projections. Note that the sources being federated are configured separately and have their own properties.

I think you're finding this difficult because there's a public method missing on the FederatedRepositorySource to set the workspace information via the JcrConfiguration class. We do this internally, but those methods aren't available outside the project.

If you'd log an JIRA issue stating that it's not possible to programmatically configure a JcrEngine with a federated repository, we'll try to get that fixed for 2.7. In the meantime, about the only recourse is using a configuration file.

[kkrauth]

Good to know that this is not currently possible. It felt like I was running around in circles for far too long towards the end, especially since the documentation only listed name and retryLimit as available configuration properties.

I have logged a JIRA issue to track the enhancement request:

https://issues.jboss.org/browse/MODE-1325

Thanks for looking into it!

[rhauch]

I started looking into this today, and it's going to require a fairly significant amount of work. The JcrConfiguration is currently written to only deal with a single level of properties (represented as attributes in XML) on each repository source, but the federation connector requires 5 levels of XML. So while it is definitely possible to do, it will take a fair amount of work to change and add this feature. One workaround is to generate the configuration XML and then parse that.

Note that with ModeShape 3.0 we've completely changed how the configuration is done: each repository has a completely separate configuration file, and each configuration file is a simple JSON file. Almost everything has sensible defaults, so a repository can actually be created only with a single name (though by default there are no sequencers or external content sources). Additionally, nearly all parts of the configuration (including sequencers and external content sources) can be changed while the repository is deployed and running.

I'm wondering whether this 2.7 feature is really worth it for 2.7, considering the amount of work, that there is a realistic workaround, and that configuration of a 3.0 repository will use a different mechanism that already supports this feature. If this is really needed, we can probably find the time to do this.

Thoughts?

[kkrauth]

Actually the project I'm working on just got pushed back to February timeframe. If 3.0 release will be getting into usable shape around that time, I would definitely rather use that from the start. Based on your description it sounds like it will be a joy to configure In addition, dynamic recofiguration of a running repository is going to be an extremely useful feature and I'm looking forward to trying it out!

EDIT: Just noticed on JIRA that the planned release date for 3.0 is January 24th, so that sounds perfect!

Message was edited by: Kosta Krauth

[rhauch]

Our goal is to release 3.0.0.Final be end of January or mid February at the latest, with probably a relatively quick 3.1 for bug fixes. Hopefully that will be good timing for you.

So if it's okay with you, I'm going to mark MODE-1325 as 'WontFix'.

[kkrauth]

That's perfect. Thanks a lot for the help