-
1. Re: DataNotAvailableException and Continuous Execution
rokhmanov Jun 22, 2012 8:05 PM (in response to rokhmanov)One additional (low priority) thought please - after async continuous execution started, it should be closed by client, otherwise it will remain running as long as jvm is alive, or exception is thrown by translator. I think there might be the (exceptional) cases when client who started the query is no longer available. Is there any easy way exist to identify such "orphaned" running queries in Teiid and close them? I don't like to make things too complex (introduce keepalives or such), but at least having the ability to retrieve a number or list of currently running async continuous executions in system will be a nice thing. Client application can (when desired) retrieve the list, compare with the currently known running executions and warn if lists are different.
-
2. Re: DataNotAvailableException and Continuous Execution
shawkins Jun 22, 2012 8:59 PM (in response to rokhmanov)Andriy,
Client interaction should not be required. I'll see if I can reproduce why your request didn't restart, but from just the code snippet it's not clear that you are resetting the results or would then next call to next() simply throw another DataNotAvailableException? It would probably make more sense to have the translator code return null (if you truly are at the end of the results for that iteration) and then throw a DataNotAvailable exception if you asked about results prior to having the next set ready - which you would indicate to the engine with dataAvailable().
You can also use the get call on the client that specifies a timeout to prevent the client from waiting too long for results.
For the additional thought, since continuous clients are limited to in process connections there isn't really isn't such a thing as a down client. Or were you thinking of detecting if our threads get hung calling a callback method? For socket connections there is already a ping keep-alive mechanism. You can retrieve a list of running queries off of the admin interface, but there's currently no marker for in-process or continuous executions.
Steve
-
3. Re: DataNotAvailableException and Continuous Execution
rokhmanov Jun 23, 2012 1:30 PM (in response to shawkins)Steven, you right about continuous clients running in connection scope, I haven't thought about it deeply before posting. There is no need to have anything in addition on Teiid side, client can keep track of open statements by itself and reopen a particular connection to get rid of stale items when needed.
I also have a clarification and update to my first question. My task is to "slow down" the rate of continuously executed requests on a translator side (some of our backend systems might have rarely changing data, and banging them with a high-rate re-requests will be a waste of CPU). I thought that throwing DataNotAvailableException at the moments when resultset iteration complete will pause the next resultset creation for a delay time. It happens indeed, I just have to return null from the next() method execution after delay is not needed any more, like you said above.
The only thing I noticed is that throw the single DataNotAvailableException is not enough to make querying paused. It is probably something again I did not understand correctly, but please see this code:
... static AtomicInteger pass = new AtomicInteger(0); ... @Override public java.util.List next() throws org.teiid.translator.TranslatorException ,org.teiid.translator.DataNotAvailableException { try { if (results.next()) { List
-
4. Re: DataNotAvailableException and Continuous Execution
shawkins Jun 25, 2012 9:31 AM (in response to rokhmanov)Andriy,
I'll update the DataNotFoundException javadocs, but in short using a positive delay is a replacement for a polling mechanism and does not make hard guarentees when the next poll will be. The delay is instead the maximum amount of time before the plan will be requeued for execution. There are several scenarios that would cause the delay to be shorter, such as multiple sources where one source returns a shorter delay or if the engine is believes more work is to be done before allowing the plan to sit idle. The latter is likely the case here because we are using a separate thread to pull translator results and it is preforming a read ahead, which may cause the first DataNotAvailableException to be processed concurrently with the first set of results.
In any case if you are going to use the dataAvailable method, then you may not even want to use a polling approach and instead use a negative delay, which just instructs the engine to let the plan idle. Here also you may have the same issue that the first exception may not cause the plan to fully idle.
Steve
-
5. Re: DataNotAvailableException and Continuous Execution
rokhmanov Jun 25, 2012 12:00 PM (in response to shawkins)Thanks Steven for explanation, I think it is clear now.