TEIID30504 File not found

[joshuabezaleel]

Hi everyone,

I really need help from anyone since I've been stuck in this problem for a while.

I got one CSV file as a data source and I want to access it using the Java code that I have defined all the VDBMetaData. I already make sure that I got this CSV file (let's say it's called "sekolah_1") at the folder's absolute path that I supplied to the ParentDirectory property, but I still got this error log.

01:07:28,897 INFO  [org.teiid.RUNTIME.VDBLifeCycleListener] (MSC service thread 1-7)  TEIID40120 VDB test.1 will be removed from the repository

01:07:28,917 INFO  [org.teiid.RUNTIME.VDBLifeCycleListener] (MSC service thread 1-7)  TEIID40119 VDB test.1 removed from the repository

01:07:28,918 INFO  [org.teiid.RUNTIME] (MSC service thread 1-7)  TEIID50026 VDB "test.1[physicalModel{text-connector=text-connector, file, java:/textconnector-file}, sekolah_1VirtualModel{}]" undeployed.

01:07:28,919 INFO  [org.teiid.RUNTIME.VDBLifeCycleListener] (MSC service thread 1-7)  TEIID40120 VDB test.1 will be removed from the repository

01:07:28,925 INFO  [org.jboss.as.server.deployment] (MSC service thread 1-1)  WFLYSRV0028: Stopped deployment test-vdb.xml (runtime-name: test-vdb.xml) in 29ms

01:07:28,929 INFO  [org.jboss.as.server.deployment] (MSC service thread 1-4)  WFLYSRV0027: Starting deployment of "test-vdb.xml" (runtime-name: "test-vdb.xml")

01:07:29,172 INFO  [org.teiid.RUNTIME.VDBLifeCycleListener] (MSC service thread 1-7)  TEIID40118 VDB test.1 added to the repository

01:07:29,173 INFO  [org.teiid.RUNTIME] (MSC service thread 1-7)  TEIID50029 VDB test.1 model "physicalModel" metadata is currently being loaded. Start Time: 5/6/19 1:07 AM

01:07:29,182 INFO  [org.teiid.RUNTIME] (MSC service thread 1-7)  TEIID50029 VDB test.1 model "sekolah_1VirtualModel" metadata is currently being loaded. Start Time: 5/6/19 1:07 AM

01:07:29,193 INFO  [org.teiid.RUNTIME] (Worker9_async-teiid-threads16)  TEIID50030 VDB test.1 model "sekolah_1VirtualModel" metadata loaded. End Time: 5/6/19 1:07 AM

01:07:29,201 INFO  [org.teiid.RUNTIME] (Worker8_async-teiid-threads17)  TEIID50030 VDB test.1 model "physicalModel" metadata loaded. End Time: 5/6/19 1:07 AM

01:07:29,215 INFO  [org.teiid.RUNTIME.VDBLifeCycleListener] (Worker8_async-teiid-threads17)  TEIID40003 VDB test.1 is set to ACTIVE

01:07:29,247 INFO  [org.teiid.PROCESSOR.MATVIEWS] (Worker13_QueryProcessorQueue30) 0VAplV+Z/oFo TEIID30013 Loading materialized view table #MAT_PG_CATALOG.PG_TYPE

01:07:29,255 INFO  [org.teiid.PROCESSOR.MATVIEWS] (Worker12_QueryProcessorQueue29) /yLpxMFzprKc TEIID30013 Loading materialized view table #MAT_PG_CATALOG.MATPG_DATATYPE

01:07:29,317 INFO  [org.teiid.PROCESSOR.MATVIEWS] (Worker13_QueryProcessorQueue32) 0VAplV+Z/oFo TEIID30014 Loaded materialized view table #MAT_PG_CATALOG.PG_TYPE with row count 43.

01:07:29,335 INFO  [org.teiid.PROCESSOR.MATVIEWS] (Worker12_QueryProcessorQueue29) /yLpxMFzprKc TEIID30014 Loaded materialized view table #MAT_PG_CATALOG.MATPG_DATATYPE with row count 51.

01:07:29,346 INFO  [org.jboss.as.server] (management-handler-thread - 5)  WFLYSRV0016: Replaced deployment "test-vdb.xml" with deployment "test-vdb.xml"

01:07:29,877 WARN  [org.teiid.CONNECTOR] (Worker13_QueryProcessorQueue34) o4ky0U7PVg8K Connector worker process failed for atomic-request=o4ky0U7PVg8K.0.3.2: org.teiid.translator.TranslatorException: File not found sekolah_1.csv

at org.teiid.translator.file.FileExecutionFactory$FileProcedureExecution.execute(FileExecutionFactory.java:94)

at org.teiid.dqp.internal.datamgr.ConnectorWorkItem.execute(ConnectorWorkItem.java:358)

at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)

at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

at java.lang.reflect.Method.invoke(Method.java:498)

at org.teiid.dqp.internal.datamgr.ConnectorManager$1.invoke(ConnectorManager.java:216)

at com.sun.proxy.$Proxy36.execute(Unknown Source)

at org.teiid.dqp.internal.process.DataTierTupleSource.getResults(DataTierTupleSource.java:302)

at org.teiid.dqp.internal.process.DataTierTupleSource$1.call(DataTierTupleSource.java:108)

at org.teiid.dqp.internal.process.DataTierTupleSource$1.call(DataTierTupleSource.java:104)

at java.util.concurrent.FutureTask.run(FutureTask.java:266)

at org.teiid.dqp.internal.process.FutureWork.run(FutureWork.java:61)

at org.teiid.dqp.internal.process.DQPWorkContext.runInContext(DQPWorkContext.java:280)

at org.teiid.dqp.internal.process.ThreadReuseExecutor$RunnableWrapper.run(ThreadReuseExecutor.java:115)

at org.teiid.dqp.internal.process.ThreadReuseExecutor$3.run(ThreadReuseExecutor.java:206)

at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)

at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)

at java.lang.Thread.run(Thread.java:748)

Caused by: javax.resource.ResourceException: File not found sekolah_1.csv

at org.teiid.translator.FileConnection$Util.getFiles(FileConnection.java:93)

at org.teiid.translator.file.FileExecutionFactory$FileProcedureExecution.execute(FileExecutionFactory.java:92)

... 18 more

01:07:29,880 WARN  [org.teiid.PROCESSOR] (Worker12_QueryProcessorQueue35) o4ky0U7PVg8K TEIID30020 Processing exception for request o4ky0U7PVg8K.0 'TEIID30504 text-connector: File not found sekolah_1.csv'. Originally TeiidProcessingException FileConnection.java:93. Enable more detailed logging to see the entire stacktrace.

Any kind of help is much appreciated. Thank you very much

Best,

Joshua

[rareddy]

Post your data source configuration? When you mentioned VDBMetadata it looked like you are using Teiid Embedded? here is one example https://wiki.uiowa.edu/display/ICTSit/Creating+a+View+from+text+table+using+Teiid

[joshuabezaleel]

Hi Ramesh,

So currently I am using the runtime of WildFly Java EE Server.

The purpose of the code that I attached here are just simply to virtualize all of the CSV files in a folder to be of its own table where the name of the table is the name of the CSV file and the columns are the first line of the CSV files denoting the column.

What I found weird is that with the code that I attached earlier, I can make it work with this single CSV file (sales.csv)

PERIOD,PRODUCT,STORE,CITY,SALES

2016,computer,cihampelas,bandung,12

2016,computer,dago,bandung,13

2016,computer,melawai,jakarta,14

2016,phone,cihampelas,bandung,15

2016,phone,dago,bandung,16

2016,phone,melawai,jakarta,17

2017,computer,cihampelas,bandung,18

But I got the error of "TEIID30504 text-connector: File not found sekolah.csv" when I tried to change the CSV file into another single CSV file (sekolah.csv) in the same folder.

NPSN,NAMA

20219328,SMP Negeri 23

20219413,SMP Negeri 32

20219400,SMP Negeri 41

This is the DDL for the Virtual Model for the first CSV (sales.csv)

CREATE VIEW sales (

            period bigdecimal,

            product string,

            city string,

            store string,

            sales bigdecimal

        ) AS

          SELECT file.period, file.product, file.city, file.store, file.sales

            FROM (EXEC physicalModel.getTextFiles('sales.csv')) AS f,

            TEXTTABLE(f.file COLUMNS period bigdecimal, product string, city string, store string, sales bigdecimal HEADER) AS file;

and this is the DDL for the Virtual Model for the second CSV (sekolah.csv)

CREATE VIEW sekolah (

            npsn bigdecimal,

            nama string

        ) AS

          SELECT file.npsn, file.nama

            FROM (EXEC physicalModel.getTextFiles('sekolah.csv')) AS f,

            TEXTTABLE(f.file COLUMNS npsn bigdecimal, nama string HEADER) AS file;

Also, another question, can a VDB contain one physical model to be a text-file (CSV) connector but many - more than one - virtual model for each of the text-file in a folder?

Any kind of help is much appreciated. Thank you very much.

Best,

Joshua

[rareddy]

Instead of defining "sales.csv", you can write DDL as "*.csv" then it will pick up all the files matching the pattern in the folder defined. Note, all the csv match to to the patttern must have similar format of data. Yes, you can have multiple virtual models, but you do not need that. You can use a single virtual model and multiple virtual views in that model for each table if each csv file format is different. read about file translator here File Translator · GitBook

Ramesh..

[joshuabezaleel]

Hi Ramesh,

1. The CSV files in the folder that I am going to input will definitely have different first line/columns (pattern if that's what you mean) one to another (different CSV files). If I am going to implement what you recommend (single virtual model with multiple virtual views), do I have to specify which of the CSV files a view is corresponding to (which means defining the name of the CSV file e.g. "sales.csv" and not *.csv") or they will automatically be paired to the CSV file that has the same pattern as the one that we define at the VIEW DDL even though we do not specify which file it is (use "*.csv" instead of "view_name.csv")?

For example at the second scenario, we just use *.csv, and define a VIEW for a school csv data, and it will be connected to the school.csv and not sales.csv even though that they are in the same parent folder.

2. I might actually found the solution, or should I say how to make things work for the problem that I stated previously, which is to put/copy the .csv data source files to my eclipse workspace's resource folder ($eclipse-workspace/$project-name/src/main/resources) to make the error logs of "TEIID30504 text-connector: File not found $filename.csv" disappeared. It's not even the project's workspace resource folder, but other project's (let's say that I'm currently running a project at $eclipse-workspace/$project-a, I need to put the .csv data source files at the $eclipse-workspace/$project-b/src/main/resources to make things enable to work). The project-b was indeed my other project that I used to experimentally ran Teiid way earlier though. But, (1) I don't know why copying the files to the resource folder make things work and later can be queried and (2) I don't think this is a feasible implementation because I am planning to use a really large size CSV file as a data source and thus this is not a feasible approach. Do you know why this happens and is there any other approach that I can use?

I really appreciate any kind of help. Thank you very much.

Have a good day.

Best,

Joshua

[joshuabezaleel]

I also tried to run the $project-a that also use Teiid, but I also need to put/copy the CSV file to be in the resource folder of the project ($eclipse-workspace/$project-a/src/main/resources), instead of just inputting the path of the folder that contains the CSVs.

I already make sure that the ParentDirectory for the connection property is the folder which the path that I input earlier, so I still can't understand why this happens.

Any kind of help is really much appreciated because I desperately need this. Thank you very much.

Best, Joshua

[rareddy]

1) There is automatic mapping, for each view you want to define the .csv that dependens om, if you want you can do like use like "sales*.csv" and "customer*.csv". Or put them in seperate directories if you want to use "*.csv" for different types if csv files. If they are only differ at first line it is different issue.

2) Placing csv in resource directory is not correct. Note that there is Designer (eclipse) tool and Teiid server that may be on same machine or a seperate machine. You need to make sure that the file path you are providing is resolvable (reachable) from the Teiid server, if not you will see those file not found errors. So, if they both on same machine, provide an absolute path to the parent directory, if Teiid Server is on remote then you need to put it on a shared drive where server can read from, or local to server.

Ramesh..