9 Replies Latest reply on Jan 18, 2013 4:58 AM by anilallewar Branched from an earlier discussion.

    Thread usage in Teiid

    anilallewar

      Ramesh,

       

      While working through the local connection issue, I have one more question regarding Teiid architecture.

       

      Teiid internally uses a async thread pool to handle requests; so what is that used for? Based on the logs it looks like you use (and if my understanding is correct create) worker threads on the fly to delegate query execution to the actual datasource using the translator. So is the thread pool used for the I/O threads and they create new worker threads to do the actual work?

       

      We would also like to know how does the thread pool affect performance and does it become a bottleneck in web applications? Suppose I have a web container that uses a HTTP thread pool of 100 i.e. it supports 1000 concurrent requests. So in this case suppose most of the requests come to Teiid how does the thread pool handle it and what will be the optimal configuration of the thread pool?

       

      Anil

        • 1. Re: Teiid "Local Connection".
          shawkins

          > Teiid internally uses a async thread pool to handle requests; so what is that used for?

           

          There is a main thread pool, known as the engine/dqp or process worker pool.  The engine pool threads perform all internal server side work - processing plans, source queries, etc.  There is also a thread pool for each socket transport, which simply has the responsibility of performing the IO work and then hands off any long run processing to the engine pool.

           

          > Based on the logs it looks like you use (and if my understanding is correct create) worker threads on the fly to delegate query execution to the actual datasource using the translator.  So is the thread pool used for the I/O threads and they create new worker threads to do the actual work?

           

          No engine pool threads can have their named changed to reflect their task, this does not imply that a new thread is being created.

           

          > We would also like to know how does the thread pool affect performance and does it become a bottleneck in web applications? Suppose I have a web container that uses a HTTP thread pool of 100 i.e. it supports 1000 concurrent requests. So in this case suppose most of the requests come to Teiid how does the thread pool handle it and what will be the optimal configuration of the thread pool?

           

          So this breaks down a couple of ways. 

           

          First if you are in a local (in-vm) connection scenario and have left useCallingThread as the default of true:

          - you by-pass the socket work and pool altogether

          - you aren't limited by the max active plans setting

          - your usage of the engine thread pool will be based solely on the asynch work (asynch connector requests, stream-ing xpath processsing, etc.) that is spawned by the process thread (which is the same thread from your web app). 

           

          So the consideration here would be to appropriately size the engine thread pool to handle the desired amount of concurrency given processing factors (which will depend upon how much your plans fan-out, the user request source concurrency setting, if the connector work is actually forked, etc.) and system constraints.  Obviously unless you are on a highly scalable platform this does not mean something as simple as assuming 1000 concurrent requests * 2 teiid asynch accesses - as that implies a thread pool that is much too large.  You also need to consider your resource limits, such as your connection pools sizes (here also you are unlikely to maintian pools with 1000s of connections) since you won't typically need more threads available than connections (and that works the other way as well, just increasing threads without additional connections will just put contention on the source connection pools).

           

          The other scenario is using socket connections to Teiid:

          - you typically don't need to worry about sizing the socket pool(s) as they default to the number of processors on the system (setting it higher is not necessary as these threads only perform IO work).

          - you may need to adjust the max active plans setting, since that setting limits the amount of socket based plans that Teiid can work concurrently as to not flood resource pools

          - engine asynch thread usage is the same as with the in-vm scenario, but the engine thread pool has the additional responsibility of processing the plans (which in the local scenario was handled by the calling thread).  Plan processing work though should be just cpu bound/non-blocking so you would only need to raise the pool max compared by approxiimately the number of processors available on the system.

           

          And regardless of the scenario there are mechanisms to control the amount of asynch work spawned by the Teiid engine.  If a request is transactional, then it is performed serially.  If a translator is marked as non-forkable, then it will be accessed by the processing thread.  If the user source request concurreny is set to some number other than zero, then you can limit the number of connector requests that a spawned concurrently for a user query.  In particular when this setting is 1, then all access will be performed serially by the processing thread.

           

          Mostly what this all means is that there is no one size fits all approach that I can give you, but hopefully this will help if you need to do some tuning of the default configuration.

           

          Steve

          • 2. Re: Teiid "Local Connection".
            markaddleman

            I'm curious about the memory implications of useCallingThread=true. 

            First if you are in a local (in-vm) connection scenario and have left useCallingThread as the default of true:

            - you by-pass the socket work and pool altogether

            - you aren't limited by the max active plans setting

            - your usage of the engine thread pool will be based solely on the asynch work (asynch connector requests, stream-ing xpath processsing, etc.) that is spawned by the process thread (which is the same thread from your web app).

            When useCallingThread=true and by-pass the socket work, do we skip serialization?  What about when we use local connections but useCallingThread=false?

            • 3. Re: Teiid "Local Connection".
              shawkins

              > When useCallingThread=true and by-pass the socket work, do we skip serialization?  What about when we use local connections but useCallingThread=false?

               

              Skipping serialization is due to the in-vm connection.  So that happens with local (regardless of the useCallingThread setting) and embedded connections.  Setting useCallingThread to false from a threading perspective will look like the socket case.  The calling thread will submit work to the engine, which will acquire a thread, and the calling thread will continue.  However the calling thread will then typically block on the relevant blocking JDBC method call.

               

              Steve

              • 4. Re: Teiid "Local Connection".
                markaddleman

                Is this correct?

                 

                Connection Method
                useCallingThread value
                Data Serialization between Server & Client
                maxActivePlans
                Threading
                EmbeddedTrueNo serializationIgnored
                • Query is processed on the JDBC client thread. 
                • Async work may be started based on the query structure and data sources involved.
                EmbeddedFalseNo serializationHonored
                • Query is processed using Teiid engine thread pool
                • Additional async work may be started based on the query structure and data sources involved
                RemoteN/ASerialization will occurHonored
                • Query is processed using Teiid engine thread pool
                • Additional async work may be started based on the query structure and data sources involved

                 

                So, except for a little thread hopping embedded useCallingThread has very little affect on embedded connections?

                • 5. Re: Teiid "Local Connection".
                  shawkins

                  > Is this correct?

                   

                  Yes with the clarification that embedded means in-vm whether to Teiid embedded or within the AS server to the local Teiid instance.

                   

                  > So, except for a little thread hopping embedded useCallingThread has very little affect on embedded connections?

                   

                  Setting it to false could have a large negative impact on performance depending upon the system, engine configuration etc.  In theory it should not need to be explicitly set.  It was really added to just provide backwards compatibility to the old processing model in cases issues arose with using the calling thread.

                  • 6. Re: Teiid "Local Connection".
                    markaddleman

                    Thanks!

                    • 7. Re: Teiid "Local Connection".
                      anilallewar

                      >No engine pool threads can have their named changed to reflect their task, this does not imply that a new thread is being created.

                      1. Would it be fair to say then that all the server work (async like loading metadata, processing plan etc) is done by the async thread pool we define in the configuration file?

                      2. And we have an additional thread pool (defined by max-threads) that handles the I/O work (querying datasource, connections to source etc) that does the work for each socket?

                      3. So am I correct in stating that the socket connections get serviced by the engine threads and the engine threads delgate the request to processing threads limited by max-threads?

                      4. Or the sockets always goes to the processor thread (available in max-threads pool) and delegates any async processing (query plans etc) to the async thread pool?

                           Based on my understanding (4) seems to be how the requests are handled for Teiid remote socket connections.

                       

                      If (3) is the valid case, the number of concurrent requests that my web app would serve is limited by the async thread pool. So suppose the concurrent users that I desire to support is 50 (or 50 per node in a cluster), then my async thread pool should be say 51 (1 more to minimize deadlocks) and the processor thread pool should be say 51 * 2 if say on average my teiid VDB goes to 2 sources.

                       

                      If (4) is the valid case, then my processor thread pool should at-least be 51 * 2 (and probably some more) if say on average my teiid VDB goes to 2 sources and the async thread pool should be somewhere between 30-40 for the async tasks.

                       

                      In both cases, I am assuming that the max active plans should be set to the max concurrent request threads which is 50 assuming that all 50 requests might require therotically different plans.

                       

                      >- you typically don't need to worry about sizing the socket pool(s) as they default to the number of processors on the system (setting it higher is not necessary as these threads only perform IO work).

                      So the number of processors we are referring here is the teiid engine thread pool i.e. not async thread pool that we configure using max-threads?

                       

                      I am having a use case where request for large number of rows from the same table run fairly quickly(2 mins) but if we have a loop that takes up each row in the resultset and then trying to get its dependencies also, then it takes almost 8 hours to execute. I am assuming this could be because of max active plans or could it be related to memory buffer and allocating off-heap memory would help?

                       

                       

                       

                      • 8. Re: Teiid "Local Connection".
                        shawkins

                        > 1. Would it be fair to say then that all the server work (async like loading metadata, processing plan etc) is done by the async thread pool we define in the configuration file?

                        > 2. And we have an additional thread pool (defined by max-threads) that handles the I/O work (querying datasource, connections to source etc) that does the work for each socket?

                         

                        Assuming a full Teiid deployment, metadata loading is actually handled by a different AS thread pool.  The engine thread pool handles all non-socket work.

                         

                        > 3. So am I correct in stating that the socket connections get serviced by the engine threads and the engine threads delgate the request to processing threads limited by max-threads?

                        > 4. Or the sockets always goes to the processor thread (available in max-threads pool) and delegates any async processing (query plans etc) to the async thread pool?

                        > Based on my understanding (4) seems to be how the requests are handled for Teiid remote socket connections.

                        > So the number of processors we are referring here is the teiid engine thread pool i.e. not async thread pool that we configure using max-threads?

                         

                        Each socket transport has a dedicated thread pool for NIO work.  Those pools (it can be more than one if you have both a JDBC and ODBC socket transport configured) default in size to the number of processors - or can be set explicitly via the transport max-socket-threads. 

                        The engine has a thread pool for everything else - this means both processing plans (in the non-local / embedded case) and additional asynch work such as source queries.  The engine thread pool is restricted by the Teiid subsystem max-threads setting that defaults to 64.

                         

                        > If (3) is the valid case, the number of concurrent requests that my web app would serve is limited by the async thread pool. So suppose the concurrent users that I desire to support is 50 (or 50 per node in a cluster), then my async thread pool should be say 51 (1 more to minimize deadlocks) > and the processor thread pool should be say 51 * 2 if say on average my teiid VDB goes to 2 sources.

                        > If (4) is the valid case, then my processor thread pool should at-least be 51 * 2 (and probably some more) if say on average my teiid VDB goes to 2 sources and the async thread pool should be somewhere between 30-40 for the async tasks.

                        > In both cases, I am assuming that the max active plans should be set to the max concurrent request threads which is 50 assuming that all 50 requests might require therotically different plans.

                         

                        If you want to have the engine work (work meaning constructing the output - if the end of the result is reached and all we have to do is serve results from the buffer, then the plan is no longer considered active) concurrently on 50 plans, then yes you would set the Teiid subsystem max-active-plans to 50. 

                         

                        From there (assuming socket connections) you would likely:

                        - not need to adjust the socket thread pool size

                        - set the Teiid subsystem max-threads to accomodate the processing and source work meaning roughly 50 + 2*50 = 150 (assuming that you mean on average a plan accesses two sources concurrently).

                        - ensure that you have enough source connections to satisify ~100 concurrent accesses

                         

                        > I am having a use case where request for large number of rows from the same table run fairly quickly(2 mins) but if we have a loop that takes up each row in the resultset and then trying to get its dependencies also, then it takes almost 8 hours to execute. I am assuming this could be because of max active plans or could it be related to memory buffer and allocating off-heap memory would help?

                         

                        Where is the loop being performed?  On the client or the server?  How many requests are spawned by each row?  Do you know what is the expected execution time for the row-by-row processing on a per-row and overall basis compared to your actual performance?  You will see detail/debug logs if the engine is queuing plans due to the max active setting.  You can also use the debug/trace level to inspect buffer manager activity.  You'll want to do some more digging to understand what is happening.

                         

                        Steve

                        • 9. Re: Teiid "Local Connection".
                          anilallewar

                          Thanks Steve !! This answers a lot of my questions about performance.

                           

                          I'll get back to you with specifics once I have more details about that particular use case.