5 Replies Latest reply on Jan 25, 2013 12:50 PM by Samier Nadji

    teiid-buffer disk space usage

    Samier Nadji Newbie

      Hi,

       

      I am not able to figure out how to control how much disk space Teiid uses. I tried setting "buffer-service-max-buffer-space" and "buffer-service-max-file-size", but it seems that Teiid is still using up the 2.5G space available on the harddrive.

       

      Some more info:

      • The heavy disk usage occurs when I run several simultaneous SQL queries that return large amount of data.
      • I am running Teiid 8.2
      • I tried setting the max-file and max-space properties to very low values, but that didn't seem to change anything. The following is what it's set at now:

       

      [standalone@localhost:9999 /] /subsystem=teiid:read-attribute(name=buffer-service-max-buffer-space)

      {

          "outcome" => "success",

          "result" => 512L

      }

      [standalone@localhost:9999 /] /subsystem=teiid:read-attribute(name=buffer-service-max-file-size)                                                                  {

          "outcome" => "success",

          "result" => 10L

      }

       

       

      Did I misunderstand the use of these properties? Do I need to do anything else?

       

      Thanks

        • 1. Re: teiid-buffer disk space usage
          Steven Hawkins Master

          > The heavy disk usage occurs when I run several simultaneous SQL queries that return large amount of data.

           

          You'll want to double check the plan and whether you are using scrollable cursors.  If you effectively need results just streamed, then only a minimal amount of disk space should be consumed per user query.

           

          > I tried setting the max-file and max-space properties to very low values, but that didn't seem to change anything. The following is what it's set at now:

          > Did I misunderstand the use of these properties? Do I need to do anything else?

           

          These, as well as most configuration properties, only take affect after a Teiid restart as they affect the configuration state and not the running service state.  Also you should typically not reduce the max file size as that will lead to greater file fragmentation than is necessary.  The setting that you need is the buffer-service-max-buffer-space.

           

          If your setting of 512MB is not enforced after a restart, let me know.  You'll also want to consider that the current enforcement of the max buffer space setting is a hard error.  The query requesting the additional space will be stopped.

           

          Steve

          • 2. Re: teiid-buffer disk space usage
            Samier Nadji Newbie

            Hi Steve,

             

            I am not using scrollable cursors explicitly. And, I don’t see anything in the query plan returned by teiid that indicates the use of cursors. However, the query I am running federates data from 2 datasources. It produces a plan that has a semi-join where results from one query are used in another so was expecting some diskspace usage.

             

            Yes, I was aware that JBoss had to be recycled after changing the settings. Restarting JBoss did not seem to enforce my buffer-service-max-buffer-space setting. I tried several times (even  trying to change the max file size to see if that helps)

             

            It’s good to know about the hard error enforcement of the disk space setting.

             

            Thanks.

            • 3. Re: teiid-buffer disk space usage
              Steven Hawkins Master

              Samier,

               

              I'll see if I can reproduce anything similar locally.

               

              Steve

              • 4. Re: teiid-buffer disk space usage
                Steven Hawkins Master

                Samier,

                 

                There is an issue here:  https://issues.jboss.org/browse/TEIID-2357

                 

                The fix will be in 8.3 beta1.  Thanks for bringing this to our attention,

                 

                Steve

                • 5. Re: teiid-buffer disk space usage
                  Samier Nadji Newbie

                  Thanks Steve. We'll try with beta1.