6 Replies Latest reply on May 8, 2019 10:14 AM by rareddy

    TEIID30504 File not found

    joshuabezaleel

      Hi everyone,

       

      I really need help from anyone since I've been stuck in this problem for a while.

       

      I got one CSV file as a data source and I want to access it using the Java code that I have defined all the VDBMetaData. I already make sure that I got this CSV file (let's say it's called "sekolah_1") at the folder's absolute path that I supplied to the ParentDirectory property, but I still got this error log.

       

      01:07:28,897 INFO  [org.teiid.RUNTIME.VDBLifeCycleListener] (MSC service thread 1-7)  TEIID40120 VDB test.1 will be removed from the repository

      01:07:28,917 INFO  [org.teiid.RUNTIME.VDBLifeCycleListener] (MSC service thread 1-7)  TEIID40119 VDB test.1 removed from the repository

      01:07:28,918 INFO  [org.teiid.RUNTIME] (MSC service thread 1-7)  TEIID50026 VDB "test.1[physicalModel{text-connector=text-connector, file, java:/textconnector-file}, sekolah_1VirtualModel{}]" undeployed.

      01:07:28,919 INFO  [org.teiid.RUNTIME.VDBLifeCycleListener] (MSC service thread 1-7)  TEIID40120 VDB test.1 will be removed from the repository

      01:07:28,925 INFO  [org.jboss.as.server.deployment] (MSC service thread 1-1)  WFLYSRV0028: Stopped deployment test-vdb.xml (runtime-name: test-vdb.xml) in 29ms

      01:07:28,929 INFO  [org.jboss.as.server.deployment] (MSC service thread 1-4)  WFLYSRV0027: Starting deployment of "test-vdb.xml" (runtime-name: "test-vdb.xml")

      01:07:29,172 INFO  [org.teiid.RUNTIME.VDBLifeCycleListener] (MSC service thread 1-7)  TEIID40118 VDB test.1 added to the repository

      01:07:29,173 INFO  [org.teiid.RUNTIME] (MSC service thread 1-7)  TEIID50029 VDB test.1 model "physicalModel" metadata is currently being loaded. Start Time: 5/6/19 1:07 AM

      01:07:29,182 INFO  [org.teiid.RUNTIME] (MSC service thread 1-7)  TEIID50029 VDB test.1 model "sekolah_1VirtualModel" metadata is currently being loaded. Start Time: 5/6/19 1:07 AM

      01:07:29,193 INFO  [org.teiid.RUNTIME] (Worker9_async-teiid-threads16)  TEIID50030 VDB test.1 model "sekolah_1VirtualModel" metadata loaded. End Time: 5/6/19 1:07 AM

      01:07:29,201 INFO  [org.teiid.RUNTIME] (Worker8_async-teiid-threads17)  TEIID50030 VDB test.1 model "physicalModel" metadata loaded. End Time: 5/6/19 1:07 AM

      01:07:29,215 INFO  [org.teiid.RUNTIME.VDBLifeCycleListener] (Worker8_async-teiid-threads17)  TEIID40003 VDB test.1 is set to ACTIVE

      01:07:29,247 INFO  [org.teiid.PROCESSOR.MATVIEWS] (Worker13_QueryProcessorQueue30) 0VAplV+Z/oFo TEIID30013 Loading materialized view table #MAT_PG_CATALOG.PG_TYPE

      01:07:29,255 INFO  [org.teiid.PROCESSOR.MATVIEWS] (Worker12_QueryProcessorQueue29) /yLpxMFzprKc TEIID30013 Loading materialized view table #MAT_PG_CATALOG.MATPG_DATATYPE

      01:07:29,317 INFO  [org.teiid.PROCESSOR.MATVIEWS] (Worker13_QueryProcessorQueue32) 0VAplV+Z/oFo TEIID30014 Loaded materialized view table #MAT_PG_CATALOG.PG_TYPE with row count 43.

      01:07:29,335 INFO  [org.teiid.PROCESSOR.MATVIEWS] (Worker12_QueryProcessorQueue29) /yLpxMFzprKc TEIID30014 Loaded materialized view table #MAT_PG_CATALOG.MATPG_DATATYPE with row count 51.

      01:07:29,346 INFO  [org.jboss.as.server] (management-handler-thread - 5)  WFLYSRV0016: Replaced deployment "test-vdb.xml" with deployment "test-vdb.xml"

      01:07:29,877 WARN  [org.teiid.CONNECTOR] (Worker13_QueryProcessorQueue34) o4ky0U7PVg8K Connector worker process failed for atomic-request=o4ky0U7PVg8K.0.3.2: org.teiid.translator.TranslatorException: File not found sekolah_1.csv

      at org.teiid.translator.file.FileExecutionFactory$FileProcedureExecution.execute(FileExecutionFactory.java:94)

      at org.teiid.dqp.internal.datamgr.ConnectorWorkItem.execute(ConnectorWorkItem.java:358)

      at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

      at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)

      at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

      at java.lang.reflect.Method.invoke(Method.java:498)

      at org.teiid.dqp.internal.datamgr.ConnectorManager$1.invoke(ConnectorManager.java:216)

      at com.sun.proxy.$Proxy36.execute(Unknown Source)

      at org.teiid.dqp.internal.process.DataTierTupleSource.getResults(DataTierTupleSource.java:302)

      at org.teiid.dqp.internal.process.DataTierTupleSource$1.call(DataTierTupleSource.java:108)

      at org.teiid.dqp.internal.process.DataTierTupleSource$1.call(DataTierTupleSource.java:104)

      at java.util.concurrent.FutureTask.run(FutureTask.java:266)

      at org.teiid.dqp.internal.process.FutureWork.run(FutureWork.java:61)

      at org.teiid.dqp.internal.process.DQPWorkContext.runInContext(DQPWorkContext.java:280)

      at org.teiid.dqp.internal.process.ThreadReuseExecutor$RunnableWrapper.run(ThreadReuseExecutor.java:115)

      at org.teiid.dqp.internal.process.ThreadReuseExecutor$3.run(ThreadReuseExecutor.java:206)

      at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)

      at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)

      at java.lang.Thread.run(Thread.java:748)

      Caused by: javax.resource.ResourceException: File not found sekolah_1.csv

      at org.teiid.translator.FileConnection$Util.getFiles(FileConnection.java:93)

      at org.teiid.translator.file.FileExecutionFactory$FileProcedureExecution.execute(FileExecutionFactory.java:92)

      ... 18 more

       

       

      01:07:29,880 WARN  [org.teiid.PROCESSOR] (Worker12_QueryProcessorQueue35) o4ky0U7PVg8K TEIID30020 Processing exception for request o4ky0U7PVg8K.0 'TEIID30504 text-connector: File not found sekolah_1.csv'. Originally TeiidProcessingException FileConnection.java:93. Enable more detailed logging to see the entire stacktrace.

       

      Any kind of help is much appreciated. Thank you very much

       

      Best,

      Joshua

        • 1. Re: TEIID30504 File not found
          rareddy

          Post your data source configuration? When you mentioned VDBMetadata it looked like you are using Teiid Embedded? here is one example https://wiki.uiowa.edu/display/ICTSit/Creating+a+View+from+text+table+using+Teiid

          • 2. Re: TEIID30504 File not found
            joshuabezaleel

            Hi Ramesh,

             

            So currently I am using the runtime of WildFly Java EE Server.

            The purpose of the code that I attached here are just simply to virtualize all of the CSV files in a folder to be of its own table where the name of the table is the name of the CSV file and the columns are the first line of the CSV files denoting the column.

            What I found weird is that with the code that I attached earlier, I can make it work with this single CSV file (sales.csv)

             

            PERIOD,PRODUCT,STORE,CITY,SALES

            2016,computer,cihampelas,bandung,12

            2016,computer,dago,bandung,13

            2016,computer,melawai,jakarta,14

            2016,phone,cihampelas,bandung,15

            2016,phone,dago,bandung,16

            2016,phone,melawai,jakarta,17

            2017,computer,cihampelas,bandung,18

             

            But I got the error of "TEIID30504 text-connector: File not found sekolah.csv" when I tried to change the CSV file into another single CSV file (sekolah.csv) in the same folder.

             

            NPSN,NAMA

            20219328,SMP Negeri 23

            20219413,SMP Negeri 32

            20219400,SMP Negeri 41

             

            This is the DDL for the Virtual Model for the first CSV (sales.csv)

            CREATE VIEW sales (

                        period bigdecimal,

                        product string,

                        city string,

                        store string,

                        sales bigdecimal

                    ) AS

                      SELECT file.period, file.product, file.city, file.store, file.sales

                        FROM (EXEC physicalModel.getTextFiles('sales.csv')) AS f,

                        TEXTTABLE(f.file COLUMNS period bigdecimal, product string, city string, store string, sales bigdecimal HEADER) AS file;

             

            and this is the DDL for the Virtual Model for the second CSV (sekolah.csv)

            CREATE VIEW sekolah (

                        npsn bigdecimal,

                        nama string

                    ) AS

                      SELECT file.npsn, file.nama

                        FROM (EXEC physicalModel.getTextFiles('sekolah.csv')) AS f,

                        TEXTTABLE(f.file COLUMNS npsn bigdecimal, nama string HEADER) AS file;

             

            Also, another question, can a VDB contain one physical model to be a text-file (CSV) connector but many - more than one - virtual model for each of the text-file in a folder?

             

            Any kind of help is much appreciated. Thank you very much.

             

            Best,

            Joshua

            • 3. Re: TEIID30504 File not found
              rareddy

              Instead of defining "sales.csv", you can write DDL as "*.csv" then it will pick up all the files matching the pattern in the folder defined. Note, all the csv match to to the patttern must have similar format of data. Yes, you can have multiple virtual models, but you do not need that. You can use a single virtual model and multiple virtual views in that model for each table if each csv file format is different. read about file translator here File Translator · GitBook

               

              Ramesh..

              • 4. Re: TEIID30504 File not found
                joshuabezaleel

                Hi Ramesh,

                 

                1. The CSV files in the folder that I am going to input will definitely have different first line/columns (pattern if that's what you mean) one to another (different CSV files). If I am going to implement what you recommend (single virtual model with multiple virtual views), do I have to specify which of the CSV files a view is corresponding to (which means defining the name of the CSV file e.g. "sales.csv" and not *.csv") or they will automatically be paired to the CSV file that has the same pattern as the one that we define at the VIEW DDL even though we do not specify which file it is (use "*.csv" instead of "view_name.csv")?

                For example at the second scenario, we just use *.csv, and define a VIEW for a school csv data, and it will be connected to the school.csv and not sales.csv even though that they are in the same parent folder.

                 

                2. I might actually found the solution, or should I say how to make things work for the problem that I stated previously, which is to put/copy the .csv data source files to my eclipse workspace's resource folder ($eclipse-workspace/$project-name/src/main/resources) to make the error logs of "TEIID30504 text-connector: File not found $filename.csv" disappeared. It's not even the project's workspace resource folder, but other project's (let's say that I'm currently running a project at $eclipse-workspace/$project-a, I need to put the .csv data source files at the $eclipse-workspace/$project-b/src/main/resources to make things enable to work). The project-b was indeed my other project that I used to experimentally ran Teiid way earlier though. But, (1) I don't know why copying the files to the resource folder make things work and later can be queried and (2) I don't think this is a feasible implementation because I am planning to use a really large size CSV file as a data source and thus this is not a feasible approach. Do you know why this happens and is there any other approach that I can use?

                 

                I really appreciate any kind of help. Thank you very much.

                Have a good day.

                 

                Best,

                Joshua

                • 5. Re: TEIID30504 File not found
                  joshuabezaleel

                  I also tried to run the $project-a that also use Teiid, but I also need to put/copy the CSV file to be in the resource folder of the project ($eclipse-workspace/$project-a/src/main/resources), instead of just inputting the path of the folder that contains the CSVs.

                  I already make sure that the ParentDirectory for the connection property is the folder which the path that I input earlier, so I still can't understand why this happens.

                   

                  Any kind of help is really much appreciated because I desperately need this. Thank you very much.

                   

                  Best, Joshua

                  • 6. Re: TEIID30504 File not found
                    rareddy

                    1) There is automatic mapping, for each view you want to define the .csv that dependens om, if you want you can do like use like "sales*.csv" and "customer*.csv". Or put them in seperate directories if you want to use "*.csv" for different types if csv files. If they are only differ at first line it is different issue.

                     

                    2) Placing csv in resource directory is not correct. Note that there is Designer (eclipse) tool and Teiid server that may be on same machine or a seperate machine. You need to make sure that the file path you are providing is resolvable (reachable) from the Teiid server, if not you will see those file not found errors. So, if they both on same machine, provide an absolute path to the parent directory, if Teiid Server is on remote then you need to put it on a shared drive where server can read from, or local to server.


                    Ramesh..