4 Replies Latest reply on Jun 23, 2006 12:58 PM by clebert.suconic

    JDBC Proxy - Measurement Demarcation

      Proposed Phase 1 Measurement Demarcation (Start/Stop)

      I think that the demarcations are split into two catgeories roughly as follows:

      1. Simple: These are flat measurements against one API call. (eg. Connection.PrepareStatement). These do not need much extra definition as the proxy can instrument every single API call (subject to some user defined filtering) in the JDBC package.

      2. Nested: These are measurements that are applied to a defined set of nested API calls. The most common (only ?) one is a SQL query that needs to measure the prepare, execution and fetch of a query.

      Let's assume the simple demarcation does not need much more definition.
      Here's my proposal for the nested using the example of a query. The result set end point is tricky since result sets may not necessarilly be closed or fully fetched, but we need something. This is especially challenging when dealing with serialized result sets that do not hold open resources such as javax.sql.rowset.WebRowSet

      Start Points


      Connection.prepareCall
      Connection.prepareStatement
      Statement.executeQuery
      Statement.executeBatch


      End Points (Needs Work)

      ResultSet.close
      ResultSet.next()==false


      We should also capture the average reading of each stat (time, cpu, waits etc.) for the rows retrieved when iterating the result set, as well as the same stats for the entire result set retrieval.

      I am continuing to profile some of the production we're running to see what other demarcations emerge as useful.

      //Nicholas

        • 1. Re: JDBC Proxy - Measurement Demarcation
          clebert.suconic

          Just adding something in top of that:

          The application server will probably cache prepareCall and prepareStatement. So, I think we will need to measure prepares in separate.
          This will provide information to users if they need to increase preparedStatementCache or not.

          Say... if a method is spending too much time preparing statements (and caling too many prepare statements) that means the user needs to increase prepared statement cache, or change his code.

          As for queries, we need to define what we want to measure. We could measure:

          - Time used to prepare Statement (we really need this, so this could give users information if they need to increase their statement's cache)
          - Time used to execute the query
          - Time used to navigate the query.

          I really like the idea of measuring the navigation of the query, but then here it's going to be difficult (I guess) to guess if the user is closing the statement properly, and how to treat nested calls.

          We will be nesting invocations. Say if you have this code:

          public void methodA() throws SQLException
          {
           Rset rset = preparedStatement.executeQuery();
          
           while (rset.next())
           {
           anotherEJB.invokeMethod(rset.getString(1));
           }
          
           rset.close();
          }
          



          Two things could happen on this code:

          I - The user could get an exception, than we would never get the rset.close() called on the proxy driver.
          II - Do we want to show the EJB call nested to the statement open?


          Nicholas,

          You're much closer to production systems then I am these days. What's your feeling about this?

          • 2. Re: JDBC Proxy - Measurement Demarcation

            Clebert;

            Good point on the cached prepared statements. On that note, the start should be either early (no cached ps) or late (a cached ps).

            On the topic of your sample code:

            1. I will have to test this on a variety of different drivers. In many cases I think the result set will be closed when it is finalized which should happen quickly since in this example it is a locally scoped reference. If this is the case, the the added time (from the exception to finalization) would be a tip pointing to the fact that the result set was not being closed immediately (like in a finally block). If this finalize hook does not exist, the proxy can implement it (more resource conservation !!) and close the real result set.

            2. In this example, the call to the EJB shoudl absolutely be included. There are some serious performance implications here and we should not supress that bad news that the code is doing something unadvised. In this scenario though, the row by row timings (as per my suggested average derivation or some other means) would not show this overhead, as I percieve we would be measuring only the resultSet.next() times which are not impacted by the ejb call. On the other hand, the total elapsed measurements for the entire retrieval will most certainly show it (and should). This introduces some tertiary useful information which is a measurement that reflects how fast the code is **handling** the result set in contrast with how fast the database is returning the data.

            //Nicholas

            • 3. Re: JDBC Proxy - Measurement Demarcation
              clebert.suconic

              for 1) As we will be nesting calls... we need to have the close called before exiting the method. Trusing on finalize could mean the method is poping the stack of methods before the query.

              Then what we can do is... if the method closes (pop) before the query.close().
              we will do a pop again, and probably set an error on the node (query) saying that a rset.close() wasn't called.


              Clebert

              • 4. Re: JDBC Proxy - Measurement Demarcation
                clebert.suconic

                for 2) Lets do it... I liked the idea also!