-
1. Re: Does Teiid Support Importing Files Under Particular S3 Bucket Dynamically?
rareddy Aug 13, 2018 12:37 PM (in response to rinku.singh.111)Amazon-S3 translator does not support glob based pattern like *.json to select the all files in a Bucket. However, this translator gives you a procedure called "list()" and table based on this procedure is called "Bucket". So in effect, you can get the list of files using "select * from Bucket", then using that you can read each file (or just updated files) in virtual procedure perhaps.
Ramesh..
-
2. Re: Does Teiid Support Importing Files Under Particular S3 Bucket Dynamically?
rinku.singh.111 Aug 14, 2018 5:18 AM (in response to rareddy)Hello rareddy,
Thank you. I am following this example where is it required to mention the file name and then import.
Amazon S3 Translator · GitBook
Can you please help me with a small example or step by step procedure to achieve it.
Appreciate your help on this.
-
3. Re: Does Teiid Support Importing Files Under Particular S3 Bucket Dynamically?
rareddy Aug 14, 2018 10:24 AM (in response to rinku.singh.111)If you got the above example to work, then it is the process of making it work for many files. For that I am saying to follow the something like below pseudo code
create virtual procedure myrecords () returns table (x, y, z) begin loop on (select key from bucket) as files begin insert into #records FROM SELECT SP.x as x, SP.y as y, SP.z as z FROM (EXEC s3.getTextFile(name=>files.key)) AS f, TEXTTABLE(f.file COLUMNS e1 integer, e2 string, e3 double HEADER) AS SP; end select * from #records end
Note the above is written very loosely, you need to correct it for reading JSON files, return format etc. Above gives an idea what I was thinking about, do not expect to work as is.
Ramesh..