6 Replies Latest reply on Mar 26, 2014 1:33 PM by mike.conway

    ModeShape and iRODS Custom Connector

    mike.conway

      Hi all,

       

      Just a note that I'm in the process of developing a ModeShape custom connector for the iRODS Data Grid, we're really interested in WebDav and Fedora Repository integration for folks running iRODS. At any rate, I just have some code roughed in based on the File System connector and I'm just now hacking unit tests to learn how it all works, mostly wrestling with what the configs do right now. 

       

      FWIW the code is being developed over at GitHub: https://github.com/DICE-UNC/jargon-modeshape

       

      And I'll ping back when it's actually usable.  I didn't know if any devs out there were contemplating this use case but could be a fun addition to the mix of connectors available when it's done!  I expect another week of development.

       

      We're at https://www.irods.org, it's an open source policy managed data grid that supports federation, and lots of cool capabilities.  This is part of NSF's DataNet Federation Consortium project http://datafed.org/

       

      Cheers,

      Mike Conway - DICE Center - UNC Chapel Hill

        • 1. Re: ModeShape and iRODS Custom Connector
          rhauch

          That sounds really interesting. Please keep us informed about how it goes, and ping us if you run into any challenges. The latest form of the FileSystemConnector in the 'master' branch (for 4.0) already has support for events via the Java 7 watch service. Take a look if you're interested in an example of generating events.

           

          BTW, interested in writing a guest blog post to talk more about it? Perhaps when you get farther along?

          • 2. Re: ModeShape and iRODS Custom Connector
            mike.conway

            I think that would be great fun!  I have my connector running, and am more or less hacking on tests to understand what ModeShape wants. An interesting problem I'll need to solve is to connect our system metadata (which are arbitrary attribute/value/unit triples attached to files and collections) to the ModeShape extra properties store. Being able to bi-directionally understand these AVUs and these JCR multi-value extra properties is slightly tricky.

             

            I seem to be moving along, and perhaps next week I could declare victory.  I'll let you know.

             

            Cheers

            MC

            • 3. Re: ModeShape and iRODS Custom Connector
              vasilievip

              Mike,

               

              Please take a look at this connector:

              modeshape/connectors/modeshape-connector-cmis/src/main/java/org/modeshape/connector/cmis/mapping/LocalTypeManager.java a…

               

              there is type import implemented with configuration driven mapping for naming which sounds like issue you have with custom metadata.

               

              Thanks

              • 4. Re: ModeShape and iRODS Custom Connector
                rhauch

                An interesting problem I'll need to solve is to connect our system metadata (which are arbitrary attribute/value/unit triples attached to files and collections) to the ModeShape extra properties store. Being able to bi-directionally understand these AVUs and these JCR multi-value extra properties is slightly tricky.

                 

                Mike, the only problem with patterning your connector off of the file system connector is that the file system connector's "extra property storage" is very unique, and exists only because there's no way (via Java) to store additional file- and folder-related properties/tags on the file system itself. ModeShape certainly does not expect that a connector performs this way. Plus, the file system connector is very much tied to understanding only "nt:file", "nt:folder" and "nt:resource" nodes, whereas the connector framework can actually expose any kind of node types. Have a look at the Git Connector or the JDBC Metadata Connector for some very different examples.

                • 5. Re: ModeShape and iRODS Custom Connector
                  mike.conway

                  OK thanks.  I took it that I could just overhead the *Properties() methods and map to our AVU 'triples' under the covers.  At any rate I expect I'll get that all wrong at first, and am sort of relying on developing learning tests to figure all of this out (being a JCR newbie).

                   

                  I'll review the Git connector as you suggest.  I had found that on GrepCode!

                   

                  PS the other thing I'm doing is playing with the WebDav connector, which will be one of the immediate bits of low-hanging fruit to this exercise.  If anyone has this running with Tomcat or other container that would be great to know.

                   

                  Cheers

                  MC

                  • 6. Re: ModeShape and iRODS Custom Connector
                    mike.conway

                    Thanks for the tips on the props.  I have had a bit of time to play with this in the most recent code, and I've at least got some inklings of getting properties to be added to the node with custom irods:avu properties in my .cnd file like so:

                     

                    //------------------------------------------------------------------------------

                    // N A M E S P A C E S

                    //------------------------------------------------------------------------------

                     

                     

                    <irods='http://www.irods.org/jcr/irods/1.0'>

                     

                     

                     

                     

                    //------------------------------------------------------------------------------

                    // N O D E T Y P E S

                    //------------------------------------------------------------------------------

                     

                     

                    [irods:irodsobject]  mixin

                    - irods:avu (String) multiple

                     

                     

                     

                     

                    The way I had conceived of this, I'm actually using the underlying capability of iRODS to store the extra properties, and then extending ExtraPropertiesStore to do round trip updates between Node properties and iRODS AVUs.  While certainly I've got it all still hacked up, I thought that mapped nicely. 

                     

                    Just as an update, I have indeed gotten the webdav service to work, and using BitKinnex as well as WebDrive have gotten Windows to map a drive, and to basic listing/put/get so it's getting there.

                     

                    So I need to think about what my nodes are and play with some of the properties, my bigger thing is to think about custom authenticators so that I can map Basic Auth credentials to an iRODS account.  I may have to tackle this first.

                     

                    I'm also interested in optimizing some of this connection handling.  Right now I have 'finally' blocks to close the iRODS connections, though that seems less than ideal, as I may still open and close connections multiple times in an operation.  I do wonder whether there's any idea of a filter where I can enforce 'connection per request' handling.  Alternately, I can implement some sort of connection cache and timeout arrangement to help optimize on the connector side.

                     

                    But we're still plugging away. I have to say it gets pretty complicated to absorb all this at once, but I think we'll get there! 

                     

                    Cheers

                    MC