4 Replies Latest reply on Nov 25, 2014 7:28 PM by rareddy

    Using jBoss Data Virtualization platform

    prabodh1987

      Hi,

       

      I am fairly new to the entire jBoss middleware suite.

       

      I am working on an application in which I am looking to connect to disparate data sources. To save time, I am looking to implement some data virtualization tool in my application.

       

      However, the challenge over here is that my source database, table structure and the SQL queries that I will fire will be completely dynamic. The source could be anything from flat file to relational database like MS SQL Server or Teradata or Oracle to No SQL databases to Big Data.

       

      I came across jBoss Data Virtualization which supports all of these databases, however I am not sure if it will satisfy my most important requirement of the entire data access being dynamic.

       

      I am still struggling to set up the server. However, it looks like I have to use teiid designer to create virtual databases at design time and then to query these virtual databases at run time. This will not work for me. I need the entire data access to be dynamic.

       

      Can someone please guide me on this? Any links to samples are greatly appreciated!!

       

      Thanks in advance!

        • 1. Re: Using jBoss Data Virtualization platform
          rareddy

          Prabodh,

           

          Yes, it is possible. Teiid has another type of VDB called Dynamic VDB. See VDBs - Teiid 8.10 (draft) - Project Documentation Editor

          - Dynamic VDB is XML file, you can build this file programatically

          - You can use Admin API to deploy and undeploy VDBs, so when you think you have new source, or remove source create a XML VDB file and deploy it

          - Once deployed, this VDB gets its metadata from sources directly, or you can supply the DDL inside the XML file that represents the model.

          - Once the VDB is active, you can connect to it and issue SQL queries against it.

           

          Note that every time you want to add a new source or remove a old source, you need to create new XML file and deploy it. If there are users connected to the previous VDB, you can either kill those sessions or deploy the new VDB as different version such that any new connections will go to the new VDB. If your client is REST based, then since those sessions are short lived you do not need to worry about it. So, it matters what kind of client you are using?

           

          If do not want write XML, you even do much tighter integration with what Teiid calls as "MetadataRepository", which represents a VDB. Think like some SVN, GIT, RDBMS etc which can supply the schema information about your sources inside the VDB. Obviously this is much more involved topic, so either ask specific questions for us to help with, or seek professional help from RedHat.

           

          HTH.

           

          Ramesh..

          1 of 1 people found this helpful
          • 2. Re: Using jBoss Data Virtualization platform
            shawkins

            However, it looks like I have to use teiid designer to create virtual databases at design time and then to query these virtual databases at run time. This will not work for me. I need the entire data access to be dynamic.

             

            Teiid Designer is not required.  You can create what we call vdbs with just an xml file or a zip - Dynamic VDBs - Teiid 8.10 (draft) - Project Documentation Editor

             

            It depends on what you mean by dynamic as to whether Data Virtualization will be a good fit.  If you mean that you want the user to enter any query against some combination of sources, then Data Virtualization will work well.  If you want to have sources that are added/removed dynamic and associated tables/views/procedures are created dynamically, then Data Virtualization may not be a good fit.  Any structural metadata change for Teiid currently requires a new vdb revision to be deployed.

             

            > Can someone please guide me on this? Any links to samples are greatly appreciated!!

             

            A good place to start: Home - Teiid Examples - Project Documentation Editor

            1 of 1 people found this helpful
            • 3. Re: Using jBoss Data Virtualization platform
              prabodh1987

              Thanks rareddy and shawkins.

               

              Let me try to rephrase the question.

               

              How does a VDB handle changes in the structure of the source DB's? For e.g., if a new table is added or if an existing table is modified, does the developer have to recreate the VDB and deploy it to the server?

              • 4. Re: Using jBoss Data Virtualization platform
                rareddy

                It does *not* handle changes in source systems. The metadata of the source is retrieved at the deploy time of the VDB, as long as you willing to re-deploy/re-load the VDB then at that time based on some event, it can can a fresh copy of the schema from source. The question you need to ask is, how does your client know that there is new source and new table or new column available for access?

                 

                May be you are going about the usecase incorrectly with respect to DV, if you wan explain what you are trying to achieve may be can help, if DV is right fit or not.


                Ramesh..