8 Replies Latest reply on Nov 26, 2016 11:11 AM by Ramesh Reddy

    Teiid Embedded connect to Mongodb

    Mark Tawk Novice

      I'm using Teiid 9.0.0 Embedded and trying to connect to Mongodb.

      My Mongodb Model Meta Data deployed successfully.

      Is there a dynamic way to get Mongodb collections into Teiid metadata?

      If not, what is the best practice to create them into the model meta data of Teiid Embedded? I did not find a corresponding java example.

        • 1. Re: Teiid Embedded connect to Mongodb
          Ramesh Reddy Master

          Mark,

           

          If you defined your VDB as dynamic VDB with metadata element in the model like

           

          <vdb name="nothwind" version="1">
            <model name="northwind">
            <source name="local" translator-name="mongodb" connection-jndi-name="java:/mongoDS"/>
            </model>
          <vdb>
          

          then on deploy of the VDB, the collections from the mongodb source are read and will try to figure out the model structure automatically. You can log into the VDB using the JDBC and see what is the model structure it created or use the getSchema method on Admin API to see the resulting DDL of the model, or use the web-console check out the schema on the model.


          Ramesh..

          • 2. Re: Teiid Embedded connect to Mongodb
            Mark Tawk Novice

            Here is the java code I'm using and i'm getting an empty schema:

             

            String dbName = "test";

                MongoDBManagedConnectionFactory mongoDBManagedConnectionFactory = new MongoDBManagedConnectionFactory();

                mongoDBManagedConnectionFactory.setRemoteServerList("localhost:27017");

                mongoDBManagedConnectionFactory.setDatabase(dbName);

            //
            String dsName = "java:/mongoDS";
            String vdbName = "mongovdb";
            String modelName = "mongomodel";
            //
            this.embeddedServer.addConnectionFactory(dsName, mongoDBManagedConnectionFactory.createConnectionFactory());
            //
            ModelMetaData mongoDBModelMetaData = createModelMetaData(vdbName, modelName, dsName, false, dbName, null, "mongodb", null);
            this.embeddedServer.deployVDB(vdbName, mongoDBModelMetaData);
            • 3. Re: Teiid Embedded connect to Mongodb
              Ramesh Reddy Master

              How does your "createModelMetaData" look like?

              • 4. Re: Teiid Embedded connect to Mongodb
                Mark Tawk Novice
                private ModelMetaData createModelMetaData(String vdbName, String modelName, String dsName, Boolean withView, String catalog, String schema, String translator, ArrayList<String> declaredFunctions) {
                  ModelMetaData modelMetaData = new ModelMetaData();
                  modelMetaData.setName(modelName);
                  //
                  if (withView) {
                  modelMetaData.addProperty("importer.tableTypes", "TABLE,VIEW");
                  } else {
                  modelMetaData.addProperty("importer.tableTypes", "TABLE");
                  }
                  //
                  if (catalog != null)
                  modelMetaData.addProperty("importer.catalog", catalog);
                  //
                  if (schema != null) {
                  modelMetaData.addProperty("importer.schemaPattern", schema);
                  modelMetaData.addProperty("importer.useFullSchemaName", "true");
                  }
                  //
                  modelMetaData.addSourceMapping(vdbName, translator, dsName);
                  if (declaredFunctions != null && declaredFunctions.size() > 0) {
                  modelMetaData.setSchemaSourceType("DDL,NATIVE");
                  for (String declaredFct : declaredFunctions) {
                  declaredFct = "CREATE FOREIGN FUNCTION " + declaredFct;
                  modelMetaData.setSchemaText(declaredFct);
                  }
                  }
                  return modelMetaData;
                }
                
                • 5. Re: Teiid Embedded connect to Mongodb
                  Ramesh Reddy Master

                  Here is slightly modified example that works.

                   

                  Ramesh..

                  • 6. Re: Teiid Embedded connect to Mongodb
                    Mark Tawk Novice

                    I have tried the example, i'm getting an exception in MongoDBExecutionFactory.java when executing the query.

                     

                    The exception is occuring when getting the value of ObjectId (I'm using mongo-java-driver-3.3.0.jar):

                     

                    else if (value instanceof org.bson.types.ObjectId) {
                        org.bson.types.ObjectId id = (org.bson.types.ObjectId) value;
                        value = id.toStringBabble();
                    }

                     

                    ERROR CONNECTOR:67 (Worker2_QueryProcessorQueue5) -  - Connector worker process failed for atomic-request=A5bq8mB7yF8M.0.0.0 -

                    java.lang.NoSuchMethodError: org.bson.types.ObjectId.toStringBabble()Ljava/lang/String;

                      at org.teiid.translator.mongodb.MongoDBExecutionFactory.retrieveValue(MongoDBExecutionFactory.java:542)

                      at org.teiid.translator.mongodb.MongoDBQueryExecution.next(MongoDBQueryExecution.java:129)

                      at org.teiid.dqp.internal.datamgr.ConnectorWorkItem.handleBatch(ConnectorWorkItem.java:435)

                      at org.teiid.dqp.internal.datamgr.ConnectorWorkItem.more(ConnectorWorkItem.java:235)

                      at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

                      at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)

                      at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

                      at java.lang.reflect.Method.invoke(Method.java:606)

                      at org.teiid.dqp.internal.datamgr.ConnectorManager$1.invoke(ConnectorManager.java:211)

                      at com.sun.proxy.$Proxy12.more(Unknown Source)

                      at org.teiid.dqp.internal.process.DataTierTupleSource.getResults(DataTierTupleSource.java:309)

                      at org.teiid.dqp.internal.process.DataTierTupleSource$1.call(DataTierTupleSource.java:112)

                      at org.teiid.dqp.internal.process.DataTierTupleSource$1.call(DataTierTupleSource.java:108)

                      at java.util.concurrent.FutureTask.run(FutureTask.java:262)

                      at org.teiid.dqp.internal.process.FutureWork.run(FutureWork.java:65)

                      at org.teiid.dqp.internal.process.DQPWorkContext.runInContext(DQPWorkContext.java:276)

                      at org.teiid.dqp.internal.process.ThreadReuseExecutor$RunnableWrapper.run(ThreadReuseExecutor.java:119)

                      at org.teiid.dqp.internal.process.ThreadReuseExecutor$3.run(ThreadReuseExecutor.java:210)

                      at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)

                      at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)

                      at java.lang.Thread.run(Thread.java:745)

                    • 7. Re: Teiid Embedded connect to Mongodb
                      Mark Tawk Novice

                      The exception is reproduced when getting the ObjectId value.

                      If we specify in the select all fields other then ObjectId field, the query executes successfully.

                      • 8. Re: Teiid Embedded connect to Mongodb
                        Ramesh Reddy Master

                        According to Teiid's maven build it is still using mongodb java driver 2.13.1 so try with that. You can also submit a JIRA to update the driver and MongoDB version along with above exception to move to newer version.