5 Replies Latest reply on Jan 25, 2013 12:50 PM by snadji

teiid-buffer disk space usage

snadji Jan 22, 2013 9:01 PM

Hi,

I am not able to figure out how to control how much disk space Teiid uses. I tried setting "buffer-service-max-buffer-space" and "buffer-service-max-file-size", but it seems that Teiid is still using up the 2.5G space available on the harddrive.

Some more info:

The heavy disk usage occurs when I run several simultaneous SQL queries that return large amount of data.
I am running Teiid 8.2
I tried setting the max-file and max-space properties to very low values, but that didn't seem to change anything. The following is what it's set at now:

[standalone@localhost:9999 /] /subsystem=teiid:read-attribute(name=buffer-service-max-buffer-space)

{

"outcome" => "success",

"result" => 512L

}

[standalone@localhost:9999 /] /subsystem=teiid:read-attribute(name=buffer-service-max-file-size) {

"outcome" => "success",

"result" => 10L

}

Did I misunderstand the use of these properties? Do I need to do anything else?

Thanks

1. Re: teiid-buffer disk space usage

shawkins Jan 23, 2013 8:47 AM (in response to snadji)

> The heavy disk usage occurs when I run several simultaneous SQL queries that return large amount of data.

You'll want to double check the plan and whether you are using scrollable cursors. If you effectively need results just streamed, then only a minimal amount of disk space should be consumed per user query.

> I tried setting the max-file and max-space properties to very low values, but that didn't seem to change anything. The following is what it's set at now:
> Did I misunderstand the use of these properties? Do I need to do anything else?

These, as well as most configuration properties, only take affect after a Teiid restart as they affect the configuration state and not the running service state. Also you should typically not reduce the max file size as that will lead to greater file fragmentation than is necessary. The setting that you need is the buffer-service-max-buffer-space.

If your setting of 512MB is not enforced after a restart, let me know. You'll also want to consider that the current enforcement of the max buffer space setting is a hard error. The query requesting the additional space will be stopped.

Steve
Actions
2. Re: teiid-buffer disk space usage

snadji Jan 23, 2013 11:59 AM (in response to shawkins)

Hi Steve,

I am not using scrollable cursors explicitly. And, I don’t see anything in the query plan returned by teiid that indicates the use of cursors. However, the query I am running federates data from 2 datasources. It produces a plan that has a semi-join where results from one query are used in another so was expecting some diskspace usage.

Yes, I was aware that JBoss had to be recycled after changing the settings. Restarting JBoss did not seem to enforce my buffer-service-max-buffer-space setting. I tried several times (even trying to change the max file size to see if that helps)

It’s good to know about the hard error enforcement of the disk space setting.

Thanks.
Actions
3. Re: teiid-buffer disk space usage

shawkins Jan 23, 2013 11:11 PM (in response to snadji)

Samier,

I'll see if I can reproduce anything similar locally.

Steve
Actions
4. Re: teiid-buffer disk space usage

shawkins Jan 24, 2013 4:23 PM (in response to shawkins)

Samier,

There is an issue here: https://issues.jboss.org/browse/TEIID-2357

The fix will be in 8.3 beta1. Thanks for bringing this to our attention,

Steve
Actions
5. Re: teiid-buffer disk space usage

snadji Jan 25, 2013 12:50 PM (in response to shawkins)

Thanks Steve. We'll try with beta1.
Actions

Go to original post