About multiple connectors, workspaces and repositories.

[sjahan]

Hi everyone,

Another day, another question!

Sorry to bother you with that but i read the documentation about the connector framework. We're kind of interested by that feature.

What troubles me is that the project i'm working on is already connected through modeshape to two repositories, but i wasn't in the team when this has been done and i don't know if the setup is correct...

We have a MySQL database server, with two databases on it. There are the "Portal" db and the "Repository" db.

The modeshape set up is the following:

In my configuration file, i have two sources configured, each one dealing with one of the dbs. I also have two repositories, each one dealing with one source.

In my code dealing with the modeshape API, I have two JCRConfigurations, one for each repository. Each one loads its repository parameters from the configuration file and then build a JCREngine (so if i use the two connectors at the same time, i would have two different JCREngines).

Until now, these two repositories aren't directly related, so that's not embarassing to deal with them as two independant JCREngines.

But here is the federation feature which is coming:

What i would like to do is to have some connectors from various sources, but i would like to access the data through only one JCR tree. Until now, as i never use my two JCREngines at the same time, i never got interested by this feature, and i'm sure each connector is managing a different tree.

Is it possible to do this? Is there something like a mounting point or something, to tell a connector to add a branch to a JCR tree?

Because right now, i could eventually add some connectors, but it would be quite boring to deal with several tree, i should manage to move objects from one tree to another so that they could be processed, etc. this would be quite a complex setup.

Well, if you notice something strange in the setup or if you have a clue about how should i configure Modeshape to manage several connectors/datasource in the same JCR tree, this would be really great!

Thank you by advance and have a great day,

SJ.

[sjahan]

In fact, i could sum up my question by:

Are there any good patterns (repositories, workspaces, JCR trees) to deal with multiple sources of data using modeshape?

I guess our current setup isn't efficient at all to deal with several sources/connectors and the ability to deal with data accross this different sources.

Thank you

SJ.

[bcarothers]

Randall's on vacation until Tuesday, so I'll take a weak stab at this. 

If I'm understanding you correctly, you have multiple repository sources and you'd like to be able to refer to them in a single JCR repository so that you can move content around between the repository sources.  This is certainly possible and described in the Reference Guide, Chapter 14.5. Configuration.  There's also an example of this in the Repository QuickStart.  The configuration file is in GitHub at https://github.com/ModeShape/modeshape/blob/master/docs/examples/gettingstarted/repositories/src/main/resources/configRepository.xml, but I'd really recommend downloading the source code and playing around with the example to get a good feel for how it works.

Personally, I'd suggest combining the existing repositories into a single JcrEngine if that's not too disruptive for your project.  You can only "mount" (we would say "federate") repository sources within the same JcrEngine.

[rhauch]

Thanks for the spot-on answer, Brian.

The only thing I'd add is that you'll want to think of one repository as being the primary repository for your application. From your client application's perspective and from ModeShape's JCR implementation layer, this primary repository will contain all of the content, even though technically some of the content is stored in one source and the remainder is "projected into the repository" from other source(s).

This notion is important for a couple of reasons. First, the federating repository is still configured with a single source - it's just that this source is a federating source that dynamically reads content from multiple other sources and will use its projection rules to combine the content from those sources into a single hierarchical structure. Second, the primary repository still will need a system workspace, and you'll have to properly configure (or programmatically define) all of the namespaces and node type definitions for all of the content. (Thus the primary repository must have the node type definitions and namespaces used in the other repository.) Third, ModeShape will still have to index all of the content so that it can affectively provide answers to your queries.

I hope this makes sense. The most challenging aspects of federation are getting your mind around what is happening, and then setting up your configuration to do what you want. As Brian suggested, get the example running and look at the code and configuration for it. Then play with the configuration's projection rules, and see what happens. The example's configuration is pretty simplistic, and ModeShape is capable of handling much more complicated configuration. But as is so often the case get something simple working and slowly add more complexity only as needed.

Best regards.