1 2 Previous Next 18 Replies Latest reply on Jun 19, 2015 8:48 AM by Ramesh Reddy

    Caching questions

    Pranav K Novice

      Hi,

       

      I had a few questions regarding caching which I was exploring in Teiid Embedded. It would be great if someone could answer them

      1. On what basis is the default scope of the query (vds/session/user) calculated in case of ResultSet Caching?

      2. Can internal materialization work at the source level (rather than view level)? This also came from the fact that I see "isMaterialized" being a property of the generic Table class.

       

      Thanks,

      Pranav

        • 1. Re: Caching questions
          Steven Hawkins Master

          > 1. On what basis is the default scope of the query (vds/session/user) calculated in case of ResultSet Caching?

           

          It defaults to vdb and then based upon the determinism of the functions (and applicability of row filtering and column masking) invoked can narrow the scope.  We don't yet have any understanding if the data from the source is scoped for one reason or another.

           

          > 2. Can internal materialization work at the source level (rather than view level)? This also came from the fact that I see "isMaterialized" being a property of the generic Table class.

           

          No.  That is just a bit of left over metadata from the Table class representing both Tables and Views.

          • 2. Re: Caching questions
            Ramesh Reddy Master

            Typically for (2) we suggest to create a base layer where view defines source as is (except for name changes etc) then use the materialized flag on it. I guess TEIID-3434 could be used for caching sources beginning 8.11

            • 3. Re: Caching questions
              Steven Hawkins Master

              > I guess TEIID-3434 could be used for caching sources beginning 8.11

               

              That issue is mostly unrelated.  Not supporting materialized source tables is more of a planning issue in how we substitute the materialization tables.

              • 4. Re: Caching questions
                Pranav K Novice

                Thanks Steve and Ramesh for your replies.

                 

                I have couple of more questions here with respect to caching in Teiid embedded,

                1. I am seeing that when I enable internal materialization on a view model with scope as ‘user’ and a certain ttl (using the cache hint), two separate caches are formed when I query the VDB (containing this view) using two different users. What is the main idea from Teiid’s side behind this ‘user’ scope when these 2 different caches formed would be identical and not individually configurable (having same ttl etc)?
                2. What exactly is the meaning/behavior of ‘session’ scope from Teiid Embedded’s point of view? Does it mean the lifetime of a connection made to a virtual database?

                 

                Thanks again,

                Pranav

                • 5. Re: Caching questions
                  Ramesh Reddy Master

                  Panav,

                   

                  They behave just like how they are named.

                   

                  user scoping - means the cached entities are cached at user level, That means every user gets a different version of the cache, but same user with different sessions will be directed to use same cache. You want to use this scope when there is user specific data on the result

                   

                  session scope - is finest level of caching, that means ever connection you make can have its own copy of the cache, even the same user. You want use this scope when session specific details are on the result. As long as "connection aka session" is active this holds good irrespective of its embedded mode vs server.

                   

                  Ramesh..

                  • 6. Re: Caching questions
                    Pranav K Novice

                    Thanks for that information Ramesh,

                     

                    What I intend to ask for the case of user-scoped caching is, that whether I can configure the individual user caches differently (because they are formed as separate caches after all). For example user X wants his cache with a ttl of 10000 and user Y wants the view results to be cached with ttl of 20000. Would a scenario like this be possible?

                    • 7. Re: Caching questions
                      Steven Hawkins Master

                      No, that is not currently possible.

                      • 8. Re: Caching questions
                        Pranav K Novice

                        Thanks for the previous answer Steve.

                        I have one more question regarding Internal Materialization. How/can we configure the caches (eviction, expiration, cache loaders) thus formed using this approach?  I believe that for the case of resultset cache we can alter the infinispan-config.xml to change properties of the cache. What would be an interface to interact with cache configuration for the case of Materialized view?

                        I am using Teiid Embedded.

                        Thanks!

                        Pranav

                        • 9. Re: Caching questions
                          Steven Hawkins Master

                          Internal materialization loading is currently only on demand, but there is https://issues.jboss.org/browse/TEIID-2469

                           

                          Beyond that there is the TTL metadata that can be in the cache hint or extension metadata properties and system procedures to refresh the whole view or single rows.

                           

                          Internal materialization doesn't rely on infinispan.

                          • 10. Re: Caching questions
                            Pranav K Novice

                            Thanks again Steve, couple of more questions prompted from your reply-

                             

                            1) So, Internal materialization is done solely on the memory heap?

                            2) How would these caches then be in sync in a distributed scenario ? I am using Teiid embedded depolyed in a jetty based server, and I have implemented a functionality by which I am able to handle the modeling process (creation of source and view model, then deploying the VDB) in a clustered scenario through events that I broadcast throughout the cluster to bring the metadata in sync. Especially considering that the tables are loaded on-demand, what would be a way keep these materialization tables in sync on a cluster?

                            3) And just to confirm on resultset caching, the caches thus formed would be automatically replicated as per the clustering mode selected (distributed, replicated) using Infinispan, right?

                            • 11. Re: Caching questions
                              Steven Hawkins Master

                              > 1) So, Internal materialization is done solely on the memory heap?

                               

                              No they utilize the buffer manage and spill to disk as needed.

                               

                              > what would be a way keep these materialization tables in sync on a cluster?

                               

                              The same replication logic is used in embedded as the server.  When a new cluster member joins they'll pull the current materialization tables from another member.  Using any of the standard mechanisms for updated/refresh, the changes will be broadcast.

                               

                              > 3) And just to confirm on resultset caching, the caches thus formed would be automatically replicated as per the clustering mode selected (distributed, replicated) using Infinispan, right?

                               

                              Yes, the entries are made available on the other cluster members based upon the infinispan configuration and then the actual data is pulled on demand.

                              • 12. Re: Caching questions
                                Himanshu Kapoor Newbie

                                Hi Steve,

                                 

                                It would be helpful if you can explain the second point above. As you described when a new member is joined in a cluster they will pull current materialization table from other. 

                                 

                                How does node discovers each other in a cluster in case of embedded teiid for the internal materialization. Does the settings done in jgroups file is used for node discovery here also the way it is used by infinispan for result set cache or there is some other mechanism in place ?

                                 

                                Does teiid embedded provides any way for VDS deployment on various nodes in a cluster ?

                                 

                                 

                                Thanks

                                Himanshu Kapoor

                                • 13. Re: Caching questions
                                  Steven Hawkins Master

                                  > Does the settings done in jgroups file is used for node discovery here also the way it is used by infinispan for result set cache or there is some other mechanism in place ?

                                   

                                  Yes, it is based upon the jgroups settings file set on the EmbeddedConfiguration.  If no file is set, then we'll assume a standalone mode.

                                   

                                  > Does teiid embedded provides any way for VDS deployment on various nodes in a cluster ?

                                   

                                  Currently no.  As we rely on AS logic to handle all deployments in the full version, we haven't yet provided a similar layer for Teiid embedded.

                                  • 14. Re: Caching questions
                                    Pranav K Novice

                                    Thanks, this is just in continuation with your answer related to VDB deployment. When I configure the jgroups file for clustered deployment of Internal Materialized Views (by setting materialized = true for view model), I get something like the following log message on Teiid startup for each view model contained in its respective vdb:

                                    -------------------------------------------------------------------

                                    GMS: address=PRANAVPC-5850, cluster=vdb_sf.1, physical address=152.142.297.101:7800

                                    ------------------------------------------------------------------


                                    I guess the bold part was what could have been slightly misdirecting with respect to clustered deployment of vdb. What does this point to then?

                                    1 2 Previous Next