Neo4j Connector for Modeshape 3.0

[duncanmcintyre]

The new Modeshape architecture in 3.0 seems to be geared towards simple blob-stores, and there is no more discussion of connectors to e.g. SVN. Is this intentional, and would that preclude the possibility of creating a new storage connector for 3.0 which would use Neo4j?

I am looking at building a CMS with semantic annotations and it seems logical to store the content in a graph database of some sort. I'd rather not reinvent the wheel so somehow piggybacking on top of JCR would be great.

[rhauch]

In ModeShape 2, we used connectors for both storage and accessing external systems. The idea was that connectors would abstract where/how ModeShape accessed/stored/updated all information. Unfortunately, we learned that the semantics of storing content owned by the repository are different than the semantics of accessing content owned and managed by external systems.

So in ModeShape 3's architecture (see Next Generation ModeShape for an overview), we started making this decision. All content owned by a repository will be stored (and cached) in an Infinispan cache, and we'll reintroduce the "connector" concept to handle how ModeShape accesses external systems that already are managing their own content indendently of ModeShape (such as file systems, SVN/Git repositories, other JCR repositories, etc.).

Now, we still want to have some flexibility in where the ModeShape-owned content is persisted. Infinispan already has the notion of "cache loaders" (aka "cache stores"), and relying so heavily on Infinispan for caching and storage means we can simply reuse the cache loaders to keep this abstraction. We already get the benefit of Infinispan's existing cache loaders for file system, JDBC data sources, Cassandra, BerkleyDB, and cloud storage (Amazon S3, Rackspace Cloudfiles, etc.). Other people have even created cache stores (e.g., Datomic).

So IIUC, you're asking whether we could store content in Neo4J. I think the answer is definitely 'yes' by implementing an Infinispan cache loader for Neo4J, although ModeShape would still largely use it as a blob store and wouldn't utilize many of the really nice graph-oriented features. In the longer run we probably will abstract out a bit more of the logic of using Infinispan so that we could plug in more rich data stores, but this is farther out on our roadmap.

We choose Infinispan over a graph-oriented database (or other storage technology) because it offers a fantastic capability for caching plus a really wide range of scalability. We think that Infinispan's data grid technology (storing and distributing the content in-memory across a potentially geographically-distributed cluster of machines) has tremendous potential for very large and fast repositories. The fact that it also works for small repositories is also important. In short, we don't think Infinispan is just another kind of data store; its the next generation of highly scalable, highly available, highly performant data storage.

What do you think?

[virtualdatabase]

Any update on accessing Neo4j content (inclusive of 'metadata') using ModeShape?

[rhauch]

There's been no community effort on this.