There are many resources on the web that suggest various settings to reduce the JVM GC pause times. In Teiid correctly allocate your batch sizes so that you are not holding on to too many rows that lead to big memory dumps. Check if there too many temp files being created in the "tmp/teiid/buffer" directory, this will indicate if your results being paged excessively or not.
If you are a customer please contact Redhat Technical support as they may have general JBoss AS JVM tuning guidelines to reduce the GC pauses.
EDS 5.3/ Teiid 7.7 has direct memory option, where you can configure Teiid to allocate direct memory instead of from heap, which is managed by Teiid directly for its purposes. This may also reduce some of the issues you are seeing. This version also auto calculates lot of tuple batch sizes according to the query it is executing.
> I would like to know what is the recommended configuration for Teiid JVM when dealing with very large volume of data in the range of millions of records?
If you're just returning forward-only results without joins/ordering/etc., then there shouldn't be much to configure speically for Teiid with regards to memory as we should be dealing with a bounded amount of data per request. What is your max active plans setting? If that is too high, then you may be attempting to fullful too many requests concurrently.
You'll also want to check the hotspots for memory usage with a profiler is possible to see what is producing the most garbarge - Teiid buffering, the source drivers, etc. If it's Teiid buffering then you'll want to target the buffer manager settings to improve the situation, which may include restricting the overall buffer memory available and possibly increasing the processing memory so that memory is allocated more on a query basis.
There have also been memory management improvements in every Teiid release. For instance Teiid will more appropriately shape internal batches (and source batches with a recent 8.3 change) and you can move alot of the memory consumption off heap using direct memory starting with 7.6.