7 Replies Latest reply on Jun 22, 2017 7:26 AM by rareddy

    teiid caching and performance optimization parameters

    sanjeewa111

      Hi,

       

      I'm developing a data virtualization layer using teiid which will serve data to the front end side. I need to performance tune teiid to access large volume of data and provide the result data to the front end side with optimum performance. I'm thinking about using caching to improve performance. I have the fallowing questions

       

      1. When result set caching or materialized views are used when will teiid start using disk storage instead of main memory. How memory and disk space utilized when caching is enabled
      2. How to handle max-reserve-kb, memory-buffer-space like parameters in the buffer manager. what are the optimum values. what are the references I can refer
        • 2. Re: teiid caching and performance optimization parameters
          sanjeewa111

          What is the recommended value for memory allocation for JVM for teiid in production environment. current configuration is 512MB. We have lots of data (millions) so the cache will be in effect. Is there any recommended guidance for JVM memory allocation for teiid.

           

          Thanks above document u sent has lot of details that I need.

          • 3. Re: teiid caching and performance optimization parameters
            rareddy

            You can configure whatever memory you have dependent on your usecase.  512 MB is too small, I will start a minimum at 8 GB for a small server, we have gone all the way up to 128 GB. But note that after 16GB you need to look into large memory sizes for it.

            • 4. Re: teiid caching and performance optimization parameters
              flora_dim

              Hi Ramesh,

               

              We noticed that there are several places to modify JVM configurations within Teiid ecosystem. We found out that by changing JAVA_OPTS in bin/domain.conf we can change the JVM settings for domain controller and host controller, and that by modifying host.xml and domain.xml we can change the JVM settings for each server. Our understanding is that domain controller and host controller are primarily for the management of the cluster, and hence for caching we have to modify the JVM configuration of actual servers using host.xml and domain.xml as opposed to modifying domain.conf

               

              1. Is our assumption correct? If not kindly point out the correct location to modify the JVM settings for optimizing the caching.

              2. When you say "start at a minimum of 8GB for a small server " do you mean JVM settings under <servers> in host.xml ?

               

              Thank you.

              • 5. Re: teiid caching and performance optimization parameters
                rareddy

                If you are working standalone environment you want to change in the standalone.conf, in domain mode you want to edit the host.xml files and each node settings can be different. Also when you change in domain.conf it will only apply to host controller node.

                • 6. Re: teiid caching and performance optimization parameters
                  flora_dim

                  Hi Ramesh,

                   

                  Here is a brief description of our cluster setup. We have setup a two node Teiid cluster in domain mode. One node acts as the master and the other as the slave. In our setup we run only one server per node, hence we have only one server specified under the <servers> in host.xml in each node.

                  <servers>

                          <server name="server-one" group="main-server-group">

                              <jvm name="default">

                              </jvm>

                          </server>

                  </servers>

                   

                  We are implementing caching on this cluster and are studying on recommended JVM specifications for domain controller, host controller and the servers.

                   

                  1. What are the recommended JVM settings for domain controller, host controller and for the individual servers ?

                  2. We assume that for caching Teiid uses JVM associated with servers ( and not the JVM associated with the domain controller and host controller). Is this assumption correct?

                   

                  Thank you.

                  • 7. Re: teiid caching and performance optimization parameters
                    rareddy

                    1) for individual servers the previous standalone suggestions apply here. There is no one size fits all, you would have estimate based on your usage and sizes of your resultsets.

                    2) yes.