6 Replies Latest reply on Aug 20, 2013 10:29 AM by jay709

    how to create federated node via RESTful API?

    jay709

      Hi All,

       

      Could anyone please tell me if I could create a federated node via the current RESTful API?

      If so, please attached the detailed sample content, local file system as external source is enough for me.

       

      Thanks a lot,

       

      Jay

        • 1. Re: how to create federated node via RESTful API?
          rhauch

          If you configure your repository to have a federation projection, then creating an external node inside that projection is exactly the same as creating a node anywhere else.

           

          However, while it is possible to create or remove a federation projection using the FederationManager in ModeShape's public API, it is not possible to create or remove a projection via the RESTful (or WebDAV) services.

          • 2. Re: how to create federated node via RESTful API?
            jay709

            Thanks Randall for your quick response.

             

            Let me put more detail about my scenario:

             

            1) I have a fileSystemConnector on the folder "/federations";

            2) I have some files and directories in the folder already, "file1.txt","file2.txt","dir1/file3.txt",...

            3) I would not call REST API to create the external nodes, as I do not want to create/overwrite those files; (make sense?)

            4) But I need to access those files, so I have to create the fedration projection under an internal node, via the FederationManager.createExternalProjection method, in which a federated node is created, and it would link to  those files above;

             

            So, how can the user access those existing files if there is NOT REST API for them to create the federation projection?

            Should modeshape expose the a REST API to invoke FederationManager.createExternalProjection method for this case?

            Or any other way to inject those external nodes over existing files?

             

            Thanks once again,

             

             

            Jay

            • 3. Re: how to create federated node via RESTful API?
              rhauch

              Let me put more detail about my scenario:

               

              1) I have a fileSystemConnector on the folder "/federations";

              2) I have some files and directories in the folder already, "file1.txt","file2.txt","dir1/file3.txt",...

              3) I would not call REST API to create the external nodes, as I do not want to create/overwrite those files; (make sense?)

              4) But I need to access those files, so I have to create the fedration projection under an internal node, via the FederationManager.createExternalProjection method, in which a federated node is created, and it would link to  those files above;

               

              All of this seems correct. If you set up the projection inside the repository configuration, then the existing files and folders will simply appear (to JCR Sessions or REST clients) as nodes inside your repository. You can even add files and folders on the file system, and ModeShape will see them and expose them to it's clients (presuming the time-to-live value on the configured file system connector is pretty short).

               

               

              So, how can the user access those existing files if there is NOT REST API for them to create the federation projection?

              If you haven't already, please carefully read the Federation Concepts and Terminology section in our documentation. In it, we try to describe what a projection is, and what external and internal nodes are. Once you've defined a projection, then ModeShape will use the specific connector to access external information and project them as external nodes inside the repository, even when the external information (and thus the external nodes projected by the connector) change. Those external nodes are "nodes" in every sense of the term, except that they are not stored by ModeShape but instead treated as being managed/owned "externally". (ModeShape will always "own" and persist internal nodes.)

               

              So, once you define an external source and a projection (either in the external source configuration or programmatically via the FederationManager), the repository will use that projection and nodes will appear under it. Even upon restart, the repository will attempt to re-establish the projection (via the connector).

               

              Let me describe this with a more detailed example:

               

              There is a directory on the file system at path "/usr/local/dir1", and that directory contains several files (e.g., "file1.txt", "file2.txt") and a directory "dir2" with a file "file3.txt". IOW:

               

                /usr/local/dir1/file1.txt

                /usr/local/dir1/file2.txt

                /usr/local/dir1/dir2/file1.txt

               

              Now, in your repository configuration, you define an external source as follows:

               

                 "externalSources" : {

                      "files" : {

                          "classname" : "org.modeshape.connector.filesystem.FileSystemConnector",

                          "directoryPath" : "/usr/local",

                          "readonly" : true,

                          "projections" : [

                              "default:/usrfiles => /dir1"

                          ]

                      }

                  },

               

              Here, the repository will create a projection in the "default" workspace such that the "/usrfiles" node is actually a federated node that doesn't really exist in the repository but is instead a proxy for the "/usr/local/dir1" directory on the file system. Thus, the "/usrfiles" node will be a node with a primary type of "nt:folder" and three child nodes:

               

              1. /usrfiles/file1.txt (primary type of "nt:file", with a "jcr:content" child node of type "nt:resources")
              2. /usrfiles/file2.txt (primary type of "nt:file", with a "jcr:content" child node of type "nt:resources")
              3. /usrfiles/dir2 (primary type of "nt:folder"

               

              The "/usrfiles/dir2" will contain one node:

              1. /usrfiles/dir2/file3.txt (primary type of "nt:file", with a "jcr:content" child node of type "nt:resources")

               

              All of these nodes are "external nodes". Now, if you were to connect to this repository with the JCR API or via the RESTful service, you would see these nodes. You can even modify these nodes, create new children under any of them, or even remove them. For example, if you were to remove the "/usrfiles/file2.txt" node, then ModeShape would actually have the file system connector "remove" that external node, and the file system connector translates that into a "remove file" operation on the local file system.

               

              Likewise, if an additional "file4.txt" file were to appear on the local file system under the "dir2" directory, then the next time a JCR session or a RESTful service client would ask for the children of "/usrfiles/dir2" node they would see the "file3.txt" and "file4.txt" node.

               

              To summarize, projections are created infrequently, yet the nodes within those projections are entirely dynamic and will reflect the underlying system (as quickly as the time-to-live attribute is defined). Because projections are created and removed infrequently, we chose not to expose such operations in the RESTful API.

               

              consider a directory on the

              • 4. Re: how to create federated node via RESTful API?
                jay709

                Thanks Randall for the great explanation.

                 

                And please help me on the further questions:

                 

                1) For the fileSystemConnector, does it initialize/re-establish all the external nodes in the projected folder everytime when it starts/restarts? That means those external nodes are not loaded from repository when the engine starts,like those general internal nodes do. If so, should we conern about the time comsum issue during starting if there are tons of files in the folder, especially if the connector is slow?

                 

                2) When you mentioned time-to-live attribute, what is the exact property name I can use in the config file, like "cacheTtlSeconds"? And could you please point me out what is the class in the fileSystemConnector maintain such feature? And what is the exact class in the fileSystemConnector to detect the change in the folder?

                 

                Thanks,

                 

                Jay

                • 5. Re: how to create federated node via RESTful API?
                  rhauch

                  1) For the fileSystemConnector, does it initialize/re-establish all the external nodes in the projected folder everytime when it starts/restarts? That means those external nodes are not loaded from repository when the engine starts,like those general internal nodes do. If so, should we conern about the time comsum issue during starting if there are tons of files in the folder, especially if the connector is slow?

                  ModeShape never handles internal or external nodes this way. All nodes are read when needed, and maintained in an internal cache i) until the node is changed, or ii) for a maximum amount of time (whichever happens first). For external nodes, there is also the connectors time-to-live value, which is generally small and thus very often is the reason why a cached representation is discarded.

                   

                  For example, if you set the "cacheTtlSeconds" to 1 second on an external source that uses a file system connector, and then the first time ModeShape needs an external node it will ask the connector (which will read the file system to build the representation). If ModeShape is asked for the same node within 1 second, it will respond with its cached respresentation; but after 1 second, any request will result in ModeShape again asking the connector for the node's representation.

                   

                   

                  2) When you mentioned time-to-live attribute, what is the exact property name I can use in the config file, like "cacheTtlSeconds"? And could you please point me out what is the class in the fileSystemConnector maintain such feature?

                   

                  You can look in the file system connector's documentation. for the complete list of attributes to use in JSON configuration files or EAP's XML configuration files. Some of these are specific to the file system connector (which in the code are represented by fields in the FileSystemConnector class) or are more general and inherited from fields in the WritableConnector or Connector base classes. Note that the Connector base class contains some fields that are set automatically by ModeShape during connector initialization, while others are set reflectively only when the configuration makes use of the attribute (we tried to document which is which in the JavaDoc for every field).

                   

                   

                  And what is the exact class in the fileSystemConnector to detect the change in the folder?

                   

                  When I talked about ModeShape knowing that an external node has changed, I was referring to a ModeShape client explicitly modifying that node.

                   

                  ModeShape does provide a way that a connector implementation can generate events to signal to ModeShape event listeners that external nodes have changed (see MODE-1710 for details). However, none of our connectors currently make use of this feature. In particular, the file system connector does not do this because JDK 6 does not provide a way to watch for changes to underlying files. (We do want to add this when we switch to JDK 7.)

                   

                  Of course, if the TTL value is small enough, ModeShape will (within reason) obtain the latest representation of the external node from the connector/external system.

                   

                  Hope this helps.

                  • 6. Re: how to create federated node via RESTful API?
                    jay709

                    Thank you very much, Randall, you do help me a lot to understand the federation.