5 Replies Latest reply on Jan 18, 2013 8:43 AM by Ramesh Reddy Branched from an earlier discussion. Branched to a new discussion.

    Teiid and OData4j

    Anil Allewar Newbie

      Ramesh,

       

      I am leading the team that Amit is working as a development lead on and I will try to answer your question. We are working on a solution that provides federated data from a single platform. The platform has to be built such that anyone can come in and plug-in their datasources and they will be exposed by the product as REST web services using OData. Since we wanted a common abstraction layer fpr the native datasources (could be custom database, standard datasource like Oracle, web services etc), this abstraction is provided by Teiid.

       

      We pick up the metadata provided by the Teiid translator (through the connection) and build a JPA model on the fly using the dynamic JPA functionality provided by EclipseLink. We need the ability to support any model that the user wants to add in and hence we are required to build the JPA model for the EntityManagerFactory on the fly.

       

      We also needed to extend OData4j because OData provides only one service at 1 endpoint whereas we wanted to have a root service and all the datasources added are accessed through sub-URLs that we create dynamically based on the VBDs that we are exposing.

       

      I hope this information is sufficient to explain the use case we are trying to solve which I must admit requires extension to the way OData4j supports service endpoint. But I think if we remove that extension we should be able to provide support to expose 1 VDB as an OData service endpoint.

       

      I'll check with my client to see if I can share any technical documentation with you so that we can collobarate on this.

       

      We use Teiid and OData4j extensively (also trying to add and submit features to OData4j) and would love to colloborate with you to take this forward!! And thanks for the awesome work you guys are doing

        • 1. Re: Teiid "Local Connection".
          Ramesh Reddy Master

          Anil,

           

          Thanks for that explanation. Neat! May be some day you can blog about how using these different technologies you solved your usecase.

          We also needed to extend OData4j because OData provides only one service at 1 endpoint whereas we wanted to have a root service and all the datasources added are accessed through sub-URLs that we create dynamically based on the VBDs that we are exposing.

          Yes, I saw that as an issue too. Apart from the multiple service end points, I thought the sticking point is the root url to serve the metadata document.  I did not wanted to deploy a WAR file for each VDB. I solved this by rather extending one of the RestEasy listener interfaces and deploying a single WAR file, such that my OData endpoint is

           

          http://<host>:<port>/odata/<vdb-name>.<version>

           

          so, I did not have to muck with OData4J. The issue I have seen with OData4J is, it did not readily work with RestEasy which is the REST engine on JBoss AS7. I had submitted few code modifications for this to the OData4J team along with some other visitor changes. At the same time RestEasy's support for "$" in the URL was not correct, I have submitted those fixes to the RestEasy team, they already acknowledged they are going to accept them.  My code is all checked in, you can test it from building from source or wait for 8.3 beta1. There are few other features incomplete, but I would love somebody like you to validate the usage of it.

           

          Open a thread on Teiid Development when you are ready to share any information on this.

           

          Thanks


          Ramesh..

          • 2. Re: Teiid "Local Connection".
            Anil Allewar Newbie

            Yes, I saw that as an issue too. Apart from the multiple service end points, I thought the sticking point is the root url to serve the metadata document. I did not wanted to deploy a WAR file for each VDB. I solved this by rather extending one of the RestEasy listener interfaces and deploying a single WAR file, such that my OData endpoint is

             

            http://<host>:<port>/odata/<vdb-name>.<version>

             

            so, I did not have to muck with OData4J. The issue I have seen with OData4J is, it did not readily work with RestEasy which is the REST engine on JBoss AS7. I had submitted few code modifications for this to the OData4J team along with some other visitor changes. At the same time RestEasy's support for "$" in the URL was not correct, I have submitted those fixes to the RestEasy team, they already acknowledged they are going to accept them. My code is all checked in, you can test it from building from source or wait for 8.3 beta1. There are few other features incomplete, but I would love somebody like you to validate the usage of it.

            I agree that since OData4j doesn't support RestEasy, we went with the Jersey option and removed the default JAX-RS provider that is shipped with JBoss 7.1. I see that you have already initiated a dialogue with Chaminda regarding contributing from our project back to Teiid.

             

            I had a quick look at your code and see that you are executing OData queries by creating SQL using visitors. I think it might be a better idea if you can use JPA because that would ensure that you can support CRUD directly from the OEntity objects, support tree of entities, sequence generation and propogation to children etc and that might integrate nicely with the JPA2 translator you have. That is the way we have gone

             

            Please let me know what you feel would be a better idea.

            • 3. Re: Teiid "Local Connection".
              Ramesh Reddy Master

              Anil,

              I agree that since OData4j doesn't support RestEasy, we went with the Jersey option and removed the default JAX-RS provider that is shipped with JBoss 7.1. I see that you have already initiated a dialogue with Chaminda regarding contributing from our project back to Teiid.

              Yes, the issues were at both ends, I have submitted patches to both the projects as I noted in previous post. Moreover, not using RestEasy is an option for Teiid project.  Yes, I have started dialogue with Chaminda when I started on this about month and half two months ago, unfortunately I did not get any response from your team.

               

               

              I had a quick look at your code and see that you are executing OData queries by creating SQL using visitors. I think it might be a better idea if you can use JPA because that would ensure that you can support CRUD directly from the OEntity objects, support tree of entities, sequence generation and propogation to children etc and that might integrate nicely with the JPA2 translator you have. That is the way we have gone

               

              Please let me know what you feel would be a better idea.

              JPA2 translator does not implicitly provide any advantages in this scenario, lets leave that one from this discussion.  From your description, I put together the following

               

              odata.png

               

              Is the first flow representative of what you are mentioning?

               

              If yes, then you have you have the JPA layer above the Teiid using eclipse link. Now, let follow the data from Source (backwards)

               

              (4) Teiid translator getting data from remote data source, this may already an abstract layer which is converting data from source layer into relational terms + Marshall/unmarshell expense.

              (3) Teiid query engine reading from translator in relational form

              (2) Teiid's resultset is being converted to a JPA entity

              (1) JPA entity is being converted back into a XML/ATOM/JSON for OData protocol

               

              I believe, that the conversion in the middle to JPA is not necessary and that is the direction I have taken. I think with more work, we can provide all the feature you are looking for with out additional JPA layer and at the same time keeping the usage of OData4J to its core services. If JPA model is sufficient alone, I do not think Odata4J folks would have started building JDBC model, which I considered to use, but it was too inadequate for Teiid needs.

               

              Hope this sheds some rationale of decisions we made.

               

              Thanks

               

              Ramesh..

              • 4. Re: Teiid "Local Connection".
                Anil Allewar Newbie

                Ramesh,

                 

                The first flow is indeed representative of what I was thinking.

                 

                I agree with you in the fact that whether we use JPA/JDBC as the middle layer is of no relevance to either Teiid or OData4j. The only reason I thought JPA would make sense it because it is already a mature technology and provide the tools for additional functionality like navigation links, tree of entities that we would have to add separately through JDBC producer.

                 

                If you look at the JDBC producer issue logged in OData4j, it mentions that "Although one could argue this goes against the spirit of the odata entity model, it would be extremely useful for very simple database models and lower the barrier of entry for some" and I would agree that it would solve simple data model representation and do anticipate issues when the data model has lots of relationships and gets complex.

                 

                So my guess is you would have to do lots of work to provide the same OData functionality as JDBC producer which otherwise is now readily available as a JPA producer and EDM model mapper that comes standard with OData4j.

                 

                Thanks,

                Anil

                • 5. Re: Teiid "Local Connection".
                  Ramesh Reddy Master

                  Anil,

                  I agree with you in the fact that whether we use JPA/JDBC as the middle layer is of no relevance to either Teiid or OData4j. The only reason I thought JPA would make sense it because it is already a mature technology and provide the tools for additional functionality like navigation links, tree of entities that we would have to add separately through JDBC producer.

                  I agree, JPA code looks  more mature than JDBC in the OData4J project, JDBC is still in works.

                   

                  If you look at the JDBC producer issue logged in OData4j, it mentions that "Although one could argue this goes against the spirit of the odata entity model, it would be extremely useful for very simple database models and lower the barrier of entry for some" and I would agree that it would solve simple data model representation and do anticipate issues when the data model has lots of relationships and gets complex. 

                  I agree that OData entity model is much more closer to JPA entity model and this would seem as natural fit. SQL is lot more complex, I am sure most of the OData queries can be mapped to SQL. As per the navigation details it really depends upon the relationships that are defined on the Schema. IMO, this is where Teiid shines. If the source itself does not have these relationships defined, or any other additional details, Teiid provides extensive framework like abstract view definitions and metadata framework to fill in the gap to the user, to provide better OData access, which will not be available through JDBC producer in the OData4J.

                   

                  yeah, this could be lot of work, I believe we got good start so far, we will keep improving. Any help your team can provide on this will be deeply appreciated.

                   

                  Thanks

                   

                  Ramesh..