8 Replies Latest reply on Jan 17, 2018 7:45 AM by walkerca

    Rollback for DB call with transport-level error

    nickarls

      (more of a generic REST/EJB question)

       

      I have a REST interface against a stateless EJB that stores XML messages to the database and processes them.

       

      I have a problem, however, when the DB operation is OK but e.g. the client times out. The EJB thinks everything went fine but the client gets the exception.

      Is there any way of having the transaction demarcation on a high-enough-level to be able to handle this "broken pipe"-situation?

       

      Thanks in advance,

      - Nik

        • 1. Re: Rollback for DB call with transport-level error
          walkerca

          Hi,

           

          If I were in this situation, I'd involve JMS and an Asynchronous EJB to implement fire-and-forget (rather than request-response).  Of course, there are a bunch of ramifications to this approach.  You'd need to communicate errors back using another JMS queue or put a record in a table that could be polled later for reconciliation.  If the clients time out a lot, you could be having a negative impact on them, causing a backup down the line while everyone is synchronously waiting on you.

           

          However, here's a hack that seems to work.  Try streaming your HTTP response.  At least on WildFly, this gave me "preparing output" if I waited for the results in a browser and a "Connection gone; rollback" if I hit the stop button 3 seconds into the operation.

           

          @Path("/hw")

          public class HelloWorldResource {

           

              @GET

              @Produces(MediaType.TEXT_PLAIN)

              @Path("/hwStreamed")

              public Response streamedHW() {

           

                  StreamingOutput stream = os -> {

           

                      try {

                          Thread.sleep(10_000);  // kill me in the browser

                      } catch(InterruptedException exc) {

                          exc.printStackTrace();

                      }

           

                      try {

                          Writer writer = new BufferedWriter(new OutputStreamWriter(os));

                          writer.write("Hello, World! (Streamed)");

                          writer.flush();

                          writer.close();

                          System.out.println("preparing output");

                      } catch(Exception exc) {

                          System.err.println("Connection gone; rollback");

                      }

                  };

                  return Response.ok().entity( stream ).build();

              }

          }

          1 of 1 people found this helpful
          • 2. Re: Rollback for DB call with transport-level error
            nickarls

            Yes, it is semantically a bit tricky because

             

            a) separate messages can also be lost/delayed

            b) the service is happy when it completes it's work (response is written), and has completed by the time the response might actually be read (and an error occurs)

             

            Your approach is certainly worth looking into (having control of the response long enough to retain control for the service). Another approach we are looking into is having the client re-transmit the message in case of an IO-error and having the message return the old response (side-effect free) since all messages have an unique ID which could be tracked to see if the actual work has already been done

            • 3. Re: Rollback for DB call with transport-level error
              walkerca

              I've done the re-transmits before and ended up replacing that for a JMS approach.  If you're expecting heavy traffic, this can present problems.  When your system is under stress, that mechanism can lead to a DDOS that you can't get out from under.  As you start bogging down, the clients get more timeouts.  So, they send more messages.  Your processing gets slower...

              • 4. Re: Rollback for DB call with transport-level error
                nickarls

                An additional complication is that we have others integrating to the system (in some cases Oracle DB:s doing the REST calls) so it's non-trivial for them to implement the JMS

                • 5. Re: Rollback for DB call with transport-level error
                  jaikiran

                  Based on what you note, you probably won't need JMS for this and the Async servlet support introduced in Servlet 3.0 should probably be an option in this case Asynchronous Support in Servlet 3.0 | Oracle Enterprise Tech Tips Blog

                  1 of 1 people found this helpful
                  • 6. Re: Rollback for DB call with transport-level error
                    nickarls

                    Thanks, I'll look into it. The problem is not that much that the actual message processing would be long-running - the problem is that if there is a network error when reading the response the client has no way if the receipt lost was an "ok" or an "error"...

                    • 7. Re: Rollback for DB call with transport-level error
                      walkerca

                      Hi,

                       

                      I'd definitely keep the REST interface if that wasn't clear.  The solution I have in mind would be a REST resource paired with an EJB method optionally with @Asynchronous.  The EJB method would put the payload -- probably JSON coming from the REST call -- on the JMS queue.

                       

                      I might do some light validation in the REST resource, basically anything RAM based like a required field check unless the parsing was too intensive.  Then you can return the synchronous 400 error if there's a problem right out of the gate.

                       

                      Finally, for development and testing by the external clients, I would allow a flag (ex "validateOnly=true") to be passed in.  This would skip the EJB call for the queue but run the payload through the full complement of validations available downstream.

                       

                      Good luck

                      • 8. Re: Rollback for DB call with transport-level error
                        walkerca

                        I still like to be able to tap into and inject messages into the queue for processing.  I've had good luck with the EJB @Asynchronous though.