5 Replies Latest reply on Jun 25, 2012 12:00 PM by rokhmanov

    DataNotAvailableException and Continuous Execution

    rokhmanov

      Hi,

       

      This is a new question related to my work with Continuous Execution in Teiid ( see previous here: "Proper ways to close Continuous execution" https://community.jboss.org/thread/201379?tstart=0 )

      What behavior is expected on client when translator raises DataNotAvailableException exception during processing of continuous async execution?

      Per my understanding, system should stop polling translator for a new data on a period, passed as parameter with the exception. I observe this:

      1. If DataNotAvailableException raised from inside of the next() method, immediately after the very first iteration of results complete (results.next() is returning false) - the system freezes indefinitely, and JBoss should be restarted. The thread dump of a blocked state attached, along with the sample code;

       

      Should the client perform any steps to resume execution, or may be translator call the ExecutionContext.dataAvailable() in such case?

       

      Thanks,

      Andriy

        • 1. Re: DataNotAvailableException and Continuous Execution
          rokhmanov

          One additional (low priority) thought please - after async continuous execution started, it should be closed by client, otherwise it will remain running as long as jvm is alive, or exception is thrown by translator. I think there might be the (exceptional) cases when client who started the query is no longer available. Is there any easy way exist to identify such "orphaned" running queries in Teiid and close them? I don't like to make things too complex (introduce keepalives or such), but at least having the ability to retrieve a number or list of currently running async continuous executions in system will be a nice thing. Client application can (when desired) retrieve the list, compare with the currently known running executions and warn if lists are different.

          • 2. Re: DataNotAvailableException and Continuous Execution
            shawkins

            Andriy,

             

            Client interaction should not be required.  I'll see if I can reproduce why your request didn't restart, but from just the code snippet it's not clear that you are resetting the results or would then next call to next() simply throw another DataNotAvailableException?  It would probably make more sense to have the translator code return null (if you truly are at the end of the results for that iteration) and then throw a DataNotAvailable exception if you asked about results prior to having the next set ready - which you would indicate to the engine with dataAvailable().

             

            You can also use the get call on the client that specifies a timeout to prevent the client from waiting too long for results.

             

            For the additional thought, since continuous clients are limited to in process connections there isn't really isn't such a thing as a down client.  Or were you thinking of detecting if our threads get hung calling a callback method?  For socket connections there is already a ping keep-alive mechanism.  You can retrieve a list of running queries off of the admin interface, but there's currently no marker for in-process or continuous executions.

             

            Steve

            • 3. Re: DataNotAvailableException and Continuous Execution
              rokhmanov

              Steven, you right about continuous clients running in connection scope, I haven't thought about it deeply before posting. There is no need to have anything in addition on Teiid side, client can keep track of open statements by itself and reopen a particular connection to get rid of stale items when needed.

               

              I also have a clarification and update to my first question. My task is to "slow down" the rate of continuously executed requests on a translator side (some of our backend systems might have rarely changing data, and banging them with a high-rate re-requests will be a waste of CPU). I thought that throwing DataNotAvailableException at the moments when resultset iteration complete will pause the next resultset creation for a delay time. It happens indeed, I just have to return null from the next() method execution after delay is not needed any more, like you said above.

               

              The only thing I noticed is that throw the single DataNotAvailableException is not enough to make querying paused. It is probably something again I did not understand correctly, but please see this code:

               

              ...
                   static AtomicInteger pass = new AtomicInteger(0);
              ...
                   @Override
                   public java.util.List next() throws org.teiid.translator.TranslatorException ,org.teiid.translator.DataNotAvailableException 
                   {          
                      try {
                          if (results.next()) {
                               List vals = new ArrayList(columnDataTypes.length);
                              for (int i = 0; i < columnDataTypes.length; i++) {
                                  Object value = this.executionFactory.retrieveValue(results, i+1, columnDataTypes[i]);
                                  vals.add(value);
                              }
                              System.out.println("!!! ROW: " + Arrays.toString(vals.toArray()));
                              return vals;
                          } else {
                               if (pass.incrementAndGet() < 10)
                                    {
                                       System.out.println("!!! Data not available ... " + pass);
                                       throw new DataNotAvailableException(5000);                                                 
                                    }
                          }
                      } catch (SQLException e) {
                          throw new TranslatorException(e,
                                  JDBCPlugin.Util.getString("JDBCTranslator.Unexpected_exception_translating_results___8", e.getMessage())); //$NON-NLS-1$
                      }
                      return null;
                   };
              
              
              

               

              It generates the log below. Take a note on time stamp of first, second, third and all other occurrences of "Data not available" message. Looks like the actual 5 seconds delay happens only after 2nd exception thrown.

              11:28:12,365 INFO  [stdout] (Worker1_QueryProcessorQueue1) !!! ROW: [1034, SAP, SAP AG]♪
              11:28:12,365 INFO  [stdout] (Worker1_QueryProcessorQueue1) !!! ROW: [1036, TM, Toyota Motor Corporation]♪
              11:28:12,365 INFO  [stdout] (Worker1_QueryProcessorQueue1) !!! ROW: [1046, CA, CA Technologies]♪
              11:28:12,365 INFO  [stdout] (Worker1_QueryProcessorQueue1) !!! Data not available ... 1♪
              11:28:12,375 INFO  [stdout] (Worker1_QueryProcessorQueue3) !!! Data not available ... 2♪
              11:28:17,385 INFO  [stdout] (Worker0_QueryProcessorQueue7) !!! Data not available ... 3♪
              11:28:22,395 INFO  [stdout] (Worker1_QueryProcessorQueue11) !!! Data not available ... 4♪
              11:28:27,397 INFO  [stdout] (Worker0_QueryProcessorQueue15) !!! Data not available ... 5♪
              11:28:32,407 INFO  [stdout] (Worker1_QueryProcessorQueue19) !!! Data not available ... 6♪
              11:28:37,417 INFO  [stdout] (Worker1_QueryProcessorQueue23) !!! Data not available ... 7♪
              11:28:42,427 INFO  [stdout] (Worker0_QueryProcessorQueue27) !!! Data not available ... 8♪
              11:28:47,437 INFO  [stdout] (Worker1_QueryProcessorQueue31) !!! Data not available ... 9♪
              11:28:52,447 INFO  [stdout] (Worker0_QueryProcessorQueue37) !!! Close !!!♪
              
              • 4. Re: DataNotAvailableException and Continuous Execution
                shawkins

                Andriy,

                 

                I'll update the DataNotFoundException javadocs, but in short using a positive delay is a replacement for a polling mechanism and does not make hard guarentees when the next poll will be.  The delay is instead the maximum amount of time before the plan will be requeued for execution.  There are several scenarios that would cause the delay to be shorter, such as multiple sources where one source returns a shorter delay or if the engine is believes more work is to be done before allowing the plan to sit idle.  The latter is likely the case here because we are using a separate thread to pull translator results and it is preforming a read ahead, which may cause the first DataNotAvailableException to be processed concurrently with the first set of results.

                 

                In any case if you are going to use the dataAvailable method, then you may not even want to use a polling approach and instead use a negative delay, which just instructs the engine to let the plan idle.  Here also you may have the same issue that the first exception may not cause the plan to fully idle.

                 

                Steve 

                • 5. Re: DataNotAvailableException and Continuous Execution
                  rokhmanov

                  Thanks Steven for explanation, I think it is clear now.