3 Replies Latest reply on Jun 27, 2012 12:05 PM by rhauch

    Writes in a clustered environment

    fcarriedos

      Hi there,

       

      i have been checking both documentation and the forum to find out how does Modeshape behave when writing in a clustered environment, does it support concurrent writes on the same disk or is it an exclusive operation for each node at a time?

       

      About how i am planning to deploy Modeshape would be the following way:

       

      • read / write access directly to hard disk (making this fast enough would be a matter of using an appropriate filesystem and maybe special subsystem)
      • read / write for the application level through REST interface.
      • downloading stored files through the REST interface.

       

      Thanks for your time in advance.

       

      Best regards.

        • 1. Re: Writes in a clustered environment
          rhauch

          Both ModeShape 2 and ModeShape 3 can be clustered but are architected differently, so I'll try to answer your question from both perspectives.

           

          ModeShape 2

           

          ModeShape does not use a global-level write lock; instead, how a repository behaves in a cluster depends on the type of connector that is used. The DB (aka JPA) connector relies on database transactions and locking (e.g., locking depends on the DBMS and database configuration), whereas the Infinispan connector relies upon how the cache locks each entry (that is, each node) as its updated. The disk storage connector also locks at the equivalent of the node level, but this requires all the processes in the cluster to use the same file system area. The file system connector (that exposes existing files and directories as nodes) does not have any locking mechanism at all. The SVN connector just delegates to the SVN repository.

           

          ModeShape 3

           

          ModeShape 3 does not use a global-level write lock, but uses something more akin to node-level write locks.

           

          Each repository writes content to a single Infinispan cache and uses JTA transactions, and you can control the transactional semantics of this cache (e.g., either XA-compliant or non-XA compliant). Note that by default each "Session.save()" operation persists the changes using a single transaction; however, you can of course explicitly use transactions in your JCR application and ModeShape and Infinispan will simply participate in those transactions. Regardless, each transaction locks only the node-level entries in the cache that are being modified, so the only time you might run into some delays during concurrent write are if the same nodes are being modified concurrently by separate Sessions.

           

          The primary driver for ModeShape 3's rearchitecture has been to leverage Infinispan's performance and flexibility for clustering. You can configure the Infinispan cache to write the information into the in-memory cache (including distributing the data around the cluster) and optionally writing the updated content to disk, databases, or cloud storage (see Infinispan Cache Loaders for more detail). Note that you can even control whether Infinispan writes through to storage (e.g., as part of the transaction) or writes behind (e.g., after the transaction). For large clusters, you may even consider using Infinispan as a data grid, where multiple copies of your content are distributed around the cluster and where the grid can be expanded as needed.

           

          Note that it is also possible to store the Lucene indexes in Infinispan (in separate caches than the regular content), and we're working on the ability to store large binary values (again, in separate caches).

          • 2. Re: Writes in a clustered environment
            fcarriedos

            Hi Randall,

             

            I assume then that, when using the filesystem connector, synchronizing writes on disk relays completely on my own and Modeshape will deal correctly with incomplete files (there will be many concurrent writes and much of the time, partially writen files will be present in the folder specified as workspace). Could this last point be an issue when talking about performance? On the other hand, if i plan to use DB connector, then i assume that if the DBMS product i use allows row level locking i will get row level locking on Modeshape as well, isn't it?

             

            Thank you very much for your detailed response!

            • 3. Re: Writes in a clustered environment
              rhauch

              The file system connector should be tolerant of files and folders appearing. In fact, I think that the only time the connector will have a problem with partially-written files is when the JCR client attempts to read the file content before it is completely written. The connector shouldn't really ever load the file content until the InputStream is obtained from a BINARY value in the "jcr:data" property. Other than that, it is really just accessing java.io.File information (e.g., the name, possibly the size, timestamps, etc).

              1 of 1 people found this helpful