7 Replies Latest reply on Sep 22, 2012 12:56 PM by bmajsak

    Strange Behavior with @ShouldMatchDataSet in APE

    btsaunde

      I am developing some tests using APE and have come across some strange behavior with the @ShouldMatchDataSet annotation. The problem I am having is when I run my tests, and I pass it the Dataset that I expect to see after the test it passes, but it will also pass if I give it an Empty Dataset. If I give it an incorect Dataset it will properly fail.

       

      For instance, if I seed the test with this Dataset:

       

      *Seed Dataset*

      {code}

      <dataset>

           <user id="1" firstName="Bryan" lastName="Saunders"/>

            <user id="2" firstName="Joe" lastName="Bob"/>

      </dataset>

      {code}

      and in my test I delete User #2, I expect to see this Dataset

       

      *Correct Result Dataset*

      {code}

      <dataset>

           <user id="1" firstName="Bryan" lastName="Saunders"/>

      </dataset>

      {code}

      If I pass it that Dataset, the test will pass, and if I pass it either of the following dataset's it will fail.

       

      *Bad Result Dataset 1*

      {code}

      <dataset>

           <user id="1" firstName="Bryan" lastName="Bob"/>

      </dataset>

      {code}

       

      *Bad Result Dataset 2*

      {code}

      <dataset>

           <user id="1" firstName="Bryan" lastName="Saunders"/>

           <user id="2" firstName="Joe" lastName="Bob"/>

      </dataset>

      {code}

      However, if I pass it this dataset,

       

      *Empty Dataset*

      {code}

      <dataset/>

      {code}

      The test will happily pass. Even though only one row was deleted, and there is still data in the database. From what I can tell from my testing and from analyzing the data dumps from the tests, it is using the following process to check the tabls

       

      {code}

           if( FileRowCount == TableRowCount ){

                if( FileRowData == TableRowData ){

                     Pass Test

                }else{

                     Fail Test

                }

           }else if( FileRowCount < TableRowCount ){

                Pass Test

           }else{

                Fail Test

           }

      {code}

      I have checked the source code for DbUnit 2.4.8 and it appears to not work this way, and should be failing whenever the number of rows in the file and the database do not match, so I have no idea what is going on here. Has anyone else had this problem, or seen this behavior?

       

      I am running Arqullian 1.0.1.Final and Alpha5 of APE and have tried it in both AS 7.1 and EAP6 on RHEL 6.2 using an H2 in-memory Database.

        • 1. Re: Strange Behavior with @ShouldMatchDataSet in APE
          bmajsak

          Thanks Bryan for such a detailed report and apologies for making your life harder

           

          Would you mind to open issue in jira and simply link to this thread?

           

          I will take care of the rest.

           

          Cheers,

          Bartosz

          • 2. Re: Strange Behavior with @ShouldMatchDataSet in APE
            btsaunde

            I created JIRA Issue https://issues.jboss.org/browse/ARQ-1094.

             

            Thanks for taking a look at this for me. APE may still have some quirks and be in the Alpha stages, but I would say even with its issues it still makes my life easier :-)

            • 3. Re: Strange Behavior with @ShouldMatchDataSet in APE
              bmajsak

              Glad to hear that. It's still in its' early days but thanks to real life use cases like yours I can make it better I will keep you posted.

              • 4. Re: Strange Behavior with @ShouldMatchDataSet in APE
                bmajsak

                Ok, so here's the thing. @ShouldMatchDataSet is comparing only those tables which are defined in provided datasets. Others, which could be non-empty in the database are simply skipped. Therefore in this corner case of empty <dataset/> nothing is compared, which is obviously wrong. For the time being I will simply handle empty data sets as assumption that whole db should be empty.

                 

                But it got me thinking.

                 

                1. Would you prefer having @ShouldBeEmpty annotation which will simply check emptiness of the whole database (and optionally take table names to verify instead).

                2. Do you think strict comparison would be also handy for @ShouldMatchDataSet - meaning it will compare the whole DB content including tables which are not specified in the provided datasets?

                • 5. Re: Strange Behavior with @ShouldMatchDataSet in APE
                  btsaunde

                  I think having @ShouldBeEmpty annotation would certainly be useful. We have several tests where we only populate the database with one or two rows, that are then deleted in the test, so the resulting database should be 100% empty.

                   

                  As for the strict matching on @ShouldMatchDataSet, that would certainly be Handy. There are some scenarios where I want to test that the database is 100% Identical (with the exception of excluded columns like generated timestamps) to what is in my Dataset.

                  • 6. Re: Strange Behavior with @ShouldMatchDataSet in APE
                    bmajsak

                    The problem with passing <dataset /> is that sometimes you could have some "utility" tables such as hibernate_sequences which you would need to exclude somehow... if in @ShouldMatchDataSet then it's getting a bit awkward construct

                    • 7. Re: Strange Behavior with @ShouldMatchDataSet in APE
                      bmajsak

                      As for the current improvement, I pushed fix to the upstream repo. There is also additional DBUnit configuration property added where you can specify tables which you don't want to verify for emptiness - see here

                      Obviously when it will be released with Alpha 6 documentation will be updated on our Confluence page

                       

                      Thanks for the feedback!

                       

                      Cheers,

                      Bartosz