3 Replies Latest reply on Dec 14, 2011 10:16 AM by rhauch

    Modeshape 2.6 how to avoid loading binary content in memory ...

    clau_babau

      Hi,

       

        We have problems when trying to delete nodes from the repository especially when those nodes contain binary content. As we are using JPA connector with Oracle database server, we retrieved statistics and we have seen that a simple delete from mode_simple_node or mode_large_values takes less than a second but our delete operation lasts for more than a couple of seconds. When activating the Hibernate logging to DEBUG, we have observed that many SQL queries are generated during a delete operation, queries that add overhead like for example there's a query that loads binary content into memory. If by chance you have a couple of nodes with binary content and that binary size is quite high it takes up time to load the data into memory and also consumes heap space.

         We want to avoid loading binary data in memory.

        Is there a possibility to do that? If not can it be added in the next releases so that binary content is loaded in memory by Modeshape only if requested through an option added to mode:source definition for JPA connector?

       

      Regards,

      Claudiu

        • 1. Re: Modeshape 2.6 how to avoid loading binary content in memory ...
          rhauch

          Unfortunately, the way the JPA connector is written in 2.x means that the connector (and JPA) end up loading some of the binary values even when not needed. I'm not sure if we can easily change that for 2.7, but the situation is definitely changing in 3.0. For example, I'm nearly finished with a feature that dramatically changes how we're dealing with Binary values. With 3, content of a binary value will be loaded only when using the binary's InputStream to read the content, and even then it will be streamed/buffered from the underlying storage. Similarly, creating a new binary values is also done with an InputStream, and will also involve streaming to the underlying storage.

           

          IOW, in ModeShape 3.0 the binary values will never be fully loaded into heap memory.

           

          Best regards,

          Randall

          • 2. Re: Modeshape 2.6 how to avoid loading binary content in memory ...
            clau_babau

            Thanks, Randall.

             

            When is the ETA for 3.0 supposed to take place?

            Is there a chance to access the source code for 3.0?

             

            Claudiu

            • 3. Re: Modeshape 2.6 how to avoid loading binary content in memory ...
              rhauch

              We plan to release our first alpha in a few weeks (before the holidays), another alpha in early January, a feature complete beta by mid-January, and go through a few more quick betas before the final release in mid-to-late February.

               

              As to the source, we're working in the '3.x' branch of our main Git repository, but it's a bit of a mess as it includes both the new stuff ('modeshape-jcr-redux') and the old stuff ('modeshape-jcr'). Before we release the first alpha, we'll remove the old stuff and move the new stuff into 'modeshape-jcr'. We're redoing our documentation, and what we have (which is not much yet) is available here.