4 Replies Latest reply on Sep 16, 2008 3:27 AM by tom.baeyens

    Major refactoring of the jbpm3 core test suite

    thomas.diesler

      Folks,

      https://jira.jboss.org/jira/browse/JBPM-1733

      I the search for a solution to the "schemaCreate() on setUp()" issue it became apparent that dropping/creating the schema on setup for every test (i.e. 300 times) during the testrun is not a valid option.

      It is only slightly better, if not equally bad, to clear the schema on setup. This is because the code for clearing the schema currently does this is a very "unintelligent" way by dropping/recreating foreign key constraints before deleting table content in an arbitrary order. This approach is invalid because it requires the jbpmtest user to have alter table privileges and also executes a very large number of DLL statements.

      Rather the opposite should be tested, which is that an "ordinary" jbpm user does not have the privileges to alter jbpm tables.

      JUnit works on the premises that a test sets up its own fixture and tears it down afterwards. Generally, it is incorrect to delegate the cleanup of test N to setup of test N+1. Only test N can know what records have been created and should be removed afterwards.

      I refactored the testsuite according to the above, with the result that neither createSchema() nor clearSchema() is called during setUp or tearDown.

      As a result, the execution time of mysql tests drops from 25min to 8min

      http://jbws.dyndns.org:8280/hudson/job/jBPM-Matrix/container=jboss422,database=mysql,jdk=jdk1.6/buildTimeTrend

      and the execution time of postgresql tests drops from 5h to 20min, which allows them to run at all.

      http://jbws.dyndns.org:8280/hudson/job/jBPM-Matrix/container=jboss422,database=postgresql,jdk=jdk1.6/buildTimeTrend

        • 1. Re: Major refactoring of the jbpm3 core test suite
          tom.baeyens

          can you explain how you make sure that each test starts from a clean DB ?

          i also played with the idea that each test should delete its own generated data explicitely. but that turned out to be very impractical and tedious. usually when you forget to delete a record (and this can be very unexpected) it is not a problem so you won't know at that time. then when after a while a test might be created or changed and that might lead to a clash. those are typically very hard to find.

          so far, clearing the whole db has proven to work well. but i'm open to hearing alternatives if you think they are better.

          • 2. Re: Major refactoring of the jbpm3 core test suite
            thomas.diesler

            > can you explain how you make sure that each test starts from a clean DB ?

            The system exits on tearDown() when there are records left in the DB. A test that does this is invalid and not allowed to enter the test suite.

            > turned out to be very impractical and tedious

            Not for new tests, but cleaning up all those tests that didn't care what they leave behind was tedious indeed.

            > those are typically very hard to find

            not if the invalid test is immediately rejected

            > so far, clearing the whole db has proven to work well

            Really? If so, there should be successful hudson runs that show that.


            • 3. Re: Major refactoring of the jbpm3 core test suite
              kukeltje

              I made some tests for some more complex jBPM usage and had to adapt them to what Thomas did (great job). Just deleting each generated processDefinition seems to delete all related processInstances, logs etc... so it was as simple as putting a try/finally around my test and indeed keeping track of what processDefinitions I deployed.

              If we adapt all tests to deploy via a deployProcessDefinitions method of the AbstractDBTestCase, it can keep track of all of them and automatically undeploy/delete at teardown. That would get rid of all try/catches again.

              The only thing is that the System.exit stops the tests, but Eclipse thinks JUnit is still running... Initially (when I updated from SVN, I was confused.... The FIXME can have a little more explanation, but now I know this, it does not realy bother me

              • 4. Re: Major refactoring of the jbpm3 core test suite
                tom.baeyens

                deleting deployed definitions and checking if no records are left seems more error prone to me then just deleting all records in the db.

                but what you suggest works and that is the most important part. it's your call.

                i don't think the same solution will hold in jbpm 4 as i'm not sure that we'll have the same behaviour for deletion of process definitions. i think it might be wise to have two separate methods: deleteAllExecutions and deleteProcessDefinition. Then deleteProcessDefinition could throw an exception if process executions are still around. but that decision is for later.