About threads used in the Teiid

[meltedmetal]

Dear all,

Would you like to do some explanation about the consumption of thread?

So we could be better to configure the maxActivePlans and maxThreads.

For a JDBC client that connect to the VDB, if a thread that is used to handle the connection is consumed?

When a VDB have 2 underlying data sources, if at least 2 threads are used to handle the connections between them and the VDB?

BTW, we used Teiid 7.2.

Thank you!

Best Regards,

S.Q

[shawkins]

Hi S.Q,

First off there is an issue with maxActivePlans not being read from the configuration.  If you want to patch an older branch I've attached a specific patch to https://issues.jboss.org/browse/TEIID-1505 - don't use the fisheye change set as the issue was worked as part of a larger set of changes.

The basic threading design separates:

- socket operations are processed by nio worker threads configured via the maxSocketThreads.  Threads for a particular transport are shared amongst all sockets.  See the Netty documenation for more.  For JDBC, all IO operations are isolated from any long running task involving sources.  See also https://issues.jboss.org/browse/TEIID-1502 for a note about ODBC.

- the query engine has a pool of threads for all Teiid processing operations, which includes source queries.  Source queries are executed and have results fetched in separate threads from the processing thread with as much parallism as possible.  See also https://issues.jboss.org/browse/TEIID-1474 for an upcoming change to help guide this parallism.  Non-transactional query plans that are waiting on source operations relinquish their processing thread so that other work can be performed.   Since obtaining and using a source connection is a blocking operation and can consume a thread indefinitely allowing too many plans to be active at once could easily lead to pool contention in waiting for source connections.  Thus the need to limit the maxActivePlans.

JDBC socket threads can then be a rather constained number, such as the max available processors on your system.

The settting for maxActivePlans is a little more complicated since maxThreads, maxActivePlans, and data source pool sizes are all interrelated.  Generally you will want maxThreads set to something greater than the maximum number of concurrent source queries you can ideally allow based upon your data source pool sizes.  Then you would set maxActivePlans to maxThreads/(average user query parallism) so that high client loads do not flood your source pools.

I hope this helps.  The docs have been updated for 7.4 on threading.  Let me know if we need to keep expanding on this.

Thanks,

Steve