1 2 3 Previous Next 34 Replies Latest reply on Dec 16, 2014 3:15 PM by rareddy Go to original post
      • 30. Re: External Materialization with Postgresql
        shawkins

        [TEIID-3259] dynamic sql cannot use into with updates - JBoss Issue Tracker covers the exception seen in #25.


        > We have a requirement to publish content of external mat view (mongodb collection) to a message queue. Does current version support this feature? Or else, do we need to extend mongodb translator or the materialization script to achieve this?


        We do have replication of our internal materialized views over jgroups, but we don't have built-in integration with message queues.  I'm not sure what this would look like as a feature.  What would the message format be and how/when would the contents be published?



        • 31. Re: External Materialization with Postgresql
          tunguyen825

          The message format is JSON (mongodb document is fine) and we like to send them after materialization to amqp topic. After discussing with our team here, we have two ideas below:

          1. Allow user to add a custom plugin to do the publishing. For example, can we specify a custom script in "teiid_rel:MATVIEW_AFTER_LOAD_SCRIPT"?

          2. Bypass database materialization: can we just go straight to amqp without storing data in mongodb? This will be nice if we can extend a generic AMQP implementation to do this

          Thanks,

          • 32. Re: External Materialization with Postgresql
            rareddy

            So, the idea is grab the MongoDB JSON document and push to AMQP? Or  you want a insert an event into AMQP notifying that the materialization is done?

             

            On (1) you could write a UDF, that gives a Java code hook. I am not sure I follow (2), if you do not want materialization why use this feature, I guess what is end usecase you are trying to achieve?

            • 33. Re: External Materialization with Postgresql
              tunguyen825

              Yes, so basically we want to convert data from Sybase to MongoDB and push to AMQP for further processing.

              Do you have example on how to add UDF into materialization process? On (2), like i said above, we might not need to store data into mongodb but need the ability to converting our data to JSON and send to AMQP.

              • 34. Re: External Materialization with Postgresql
                rareddy

                What you are misinterpreting is, Teiid is converting the data to JSON document then inserting into MongoDB. That is not the case, Teiid uses *MongoDB API* to insert documents, the persistence of the documents is handled by MongoDB. So, there is no way Teiid can get you access to raw JSON document to be forwarded to AMQP.

                 

                If you just want to convert the data into JSON, then you need to look into JSON functions JSON Functions - Teiid 8.10 (draft) - Project Documentation Editor, then push result into AMQP.  As I mentioned before you can use UDF Support for User-Defined Functions (Non-Pushdown) - Teiid 8.10 (draft) - Project Documentation Editor design a function like

                 

                CREATE FUNCTION AMQUP_PUSH(String/clob)

                 

                then use it in your SQL to call it.

                 

                SELECT AMQP_PUSH(JSONOBJECT(...)) FROM Systable.table

                 

                Ramesh..

                1 2 3 Previous Next