14 Replies Latest reply on Aug 21, 2007 7:17 AM by porcherg

    db test infrastructure

    tom.baeyens

      i want to make an inventory in this topic of the requirements and set up of the database tests in pvm. fire away !

        • 1. Re: db test infrastructure
          tom.baeyens

          here's an initial brain dump of my ideas related to db tests:

          * upon first usage of the db, the db should be created.

          * between each test, the database should be cleaned. recreation of the db often takes too long. therefor, in jbpm i first removed the constraints, then deleted all records in all the tables one by one and then recreate the constraints. i didn't really find a better approach then that although that is a bit clumsy as hibernate tools doesn't really support this scenario in a convenient way.

          * it would be great of after the complete test suite, the database was dropped. but i don't know how to do that. especially if you take into account that this should work when running test suites, but also with running individual tests. suggestions are welcome :)

          * i did an initial check with JPA and i didn't find a convenient way to specify an EntityManager programmatically. you can do it with resource META-INF/persistence.xml, but i would like it better if all the configuration was located in the central wiring based configuration file. in that case we can provide more convenience in the xml schema for configuring the persistence service.

          • 2. Re: db test infrastructure
            csouillard

             

            "tom.baeyens@jboss.com" wrote:
            here's an initial brain dump of my ideas related to db tests:

            * upon first usage of the db, the db should be created.

            OK
            "tom.baeyens@jboss.com" wrote:

            * between each test, the database should be cleaned. recreation of the db often takes too long. therefor, in jbpm i first removed the constraints, then deleted all records in all the tables one by one and then recreate the constraints. i didn't really find a better approach then that although that is a bit clumsy as hibernate tools doesn't really support this scenario in a convenient way.

            We don't have a better solution yet... DB recreation will take too much time with serverside databases installation i.e. Oracle but probably not if using a lightweight DB i.e. Hypersonic

            "tom.baeyens@jboss.com" wrote:

            * it would be great of after the complete test suite, the database was dropped. but i don't know how to do that. especially if you take into account that this should work when running test suites, but also with running individual tests. suggestions are welcome :)

            We can have a look at that point... Maybe there is a way to know inside a unit test if it is launched inside a test suite...
            "tom.baeyens@jboss.com" wrote:

            * i did an initial check with JPA and i didn't find a convenient way to specify an EntityManager programmatically. you can do it with resource META-INF/persistence.xml, but i would like it better if all the configuration was located in the central wiring based configuration file. in that case we can provide more convenience in the xml schema for configuring the persistence service.

            We can have only one configuration file (with a specific schema for pvm). Then by using an interface for persistence service we can either programmatically create necessary objects for Hibertnate either generate a compliant JPA configuration file ?
            Same way for the pvm persistence service :one icommon nterface and maybe two different implementations : Hibernate and JPA. This interface will contain high level methods : persist, remove, load, update objects...

            • 3. Re: db test infrastructure
              tom.baeyens

               

              "csouillard" wrote:
              "tom.baeyens@jboss.com" wrote:

              * it would be great of after the complete test suite, the database was dropped. but i don't know how to do that. especially if you take into account that this should work when running test suites, but also with running individual tests. suggestions are welcome :)

              We can have a look at that point... Maybe there is a way to know inside a unit test if it is launched inside a test suite...


              or maybe there is a way in java to add a hook to closing of the JVM. I know that clover must do a trick like that. But i never saw how to do it.

              • 4. Re: db test infrastructure
                tom.baeyens

                 

                "csouillard" wrote:
                "tom.baeyens@jboss.com" wrote:

                * i did an initial check with JPA and i didn't find a convenient way to specify an EntityManager programmatically. you can do it with resource META-INF/persistence.xml, but i would like it better if all the configuration was located in the central wiring based configuration file. in that case we can provide more convenience in the xml schema for configuring the persistence service.

                We can have only one configuration file (with a specific schema for pvm). Then by using an interface for persistence service we can either programmatically create necessary objects for Hibertnate either generate a compliant JPA configuration file ?
                Same way for the pvm persistence service :one icommon nterface and maybe two different implementations : Hibernate and JPA. This interface will contain high level methods : persist, remove, load, update objects...


                Generating the persistence.xml is already clumsy. Then you would have to mess with classloaders to make sure that the JPA implementation will actually find that file as resource "META-INF/persistence.xml"

                And all that effort is only worth if we really test with at least 1 other JPA implementation.

                Don't get me wrong. I want to stick to JPA as much as we can in any case. But I want to take a shortcut with hibernate to get ease of configuration for the pvm and to speed up the development process.

                Afaict now, the most reasonable alternative is to ditch the idea of 1 single central configuration and just use the META-INF/persistence.xml. But in that case, we can't offer configuration convenience like e.g.

                <peristence...>
                 <pvm-classes />
                 <bpel-classes />
                 <jpdl-classes />
                 <!-- <xpdl-classes /> -->
                </persistence>


                and people will have to mess with the individual classes in the long list in persistence.xml

                I think the best approach is to go for hibernate now, but use the JPA API's. So that later, we can still create a META-INF/persistence.xml and make the JPA implementation pluggable at that time. This way, we can postpone the moment from which we have to maintain 2 or more files with the same content. I can tell from experience that that is a nightmare.

                • 5. Re: db test infrastructure
                  tom.baeyens

                  * we also need to establish a naming scheme to translate from classnames and fieldnames to table and columnnames.

                  * naming remarks in general:
                  - max 18 chars long (some db has got a limit of 18 chars, i believe it was DB2)
                  - end with an underscore : i have had numerous field names translated into reserved words for a specific database until I systematically added an underscore to every column name.

                  * let's not forget to add indexes. ideally we should come up with a strategy.

                  • 6. Re: db test infrastructure
                    tom.baeyens

                    To what extend will we be using annotations and to what extend will we use XML configuration to override/complete the annotations.

                    This is important as different languages might want to skip persistence of certain fields. We must make sure that tweaking persistence is easily doable.

                    Best way to start i think is to just persist a single Process object. And then discuss how it's done.

                    • 7. Re: db test infrastructure
                      csouillard

                      or maybe there is a way in java to add a hook to closing of the JVM. I know that clover must do a trick like that. But i never saw how to do it.

                      • 8. Re: db test infrastructure
                        tom.baeyens

                        charles, could you comment on the ShutdownHook you've added ? i don't see yet how it solves the problem.

                        • 9. Re: db test infrastructure
                          csouillard

                          I just createsd a new Class : ShutdownHook which is a thread. I added this class as a shutdownhook of the jvm in JbpmTestCase (static constructor). Each time the jvm is closed, this thread is started... We just need to fill in the run method to delete database.

                          • 10. Re: db test infrastructure
                            tom.baeyens

                            ah... i just also found the JbpmTestCase code

                            good work ! that is the solution I was looking for

                            • 11. Re: db test infrastructure

                               

                              "tom.baeyens@jboss.com" wrote:

                              Afaict now, the most reasonable alternative is to ditch the idea of 1 single central configuration and just use the META-INF/persistence.xml. But in that case, we can't offer configuration convenience like e.g.

                              <peristence...>
                               <pvm-classes />
                               <bpel-classes />
                               <jpdl-classes />
                               <!-- <xpdl-classes /> -->
                              </persistence>


                              and people will have to mess with the individual classes in the long list in persistence.xml


                              I don't see that as a big constraint and the advantage is that we leverage a standard xml file.

                              "tom.baeyens@jboss.com" wrote:

                              I think the best approach is to go for hibernate now, but use the JPA API's. So that later, we can still create a META-INF/persistence.xml and make the JPA implementation pluggable at that time. This way, we can postpone the moment from which we have to maintain 2 or more files with the same content. I can tell from experience that that is a nightmare.


                              Our feeling is that if we go first directly on hibernate, we will never support JPA.
                              Again, I would suggest to use the persistence.xml file rather to set up our own configuration mechanism for persistence. I see constraints less important than advantages.

                              "tom.baeyens@jboss.com" wrote:

                              And all that effort is only worth if we really test with at least 1 other JPA implementation


                              Even if we only support 1 JPA implementation first (hybernate) I think its useful because we are following the standard way to persist in Java. However I agree we should also check that others JPA implementation are working. We could easily integrate that in the persistence test suite configuration (let's say taking openJPA and Hibernate).

                              "tom.baeyens@jboss.com" wrote:

                              To what extend will we be using annotations and to what extend will we use XML configuration to override/complete the annotations.

                              This is important as different languages might want to skip persistence of certain fields. We must make sure that tweaking persistence is easily doable.

                              Best way to start i think is to just persist a single Process object. And then discuss how it's done.


                              Good point. We should definetely differentiate between common mappings vs languages ones.
                              We should agree so in attributes to persist as common mappings as well as to choose the way to define those mappings:

                              - Common mappings as annotations and process languages ones as XML
                              or
                              - Both as XML

                              regards,
                              Miguel Valdes


                              • 12. Re: db test infrastructure
                                tom.baeyens

                                 

                                Our feeling is that if we go first directly on hibernate, we will never support JPA.
                                Again, I would suggest to use the persistence.xml file rather to set up our own configuration mechanism for persistence. I see constraints less important than advantages.


                                apart from the separate META-INF/persistence.xml configuration, there are a couple of other issues.

                                * for persisting variables, we need to be able to ask the persistence service: "do you know how to store this type in the db?". JPA doesn't support that yet.

                                * for the test suite, we also need DB schema generation. JPA doesn't support that either.

                                * scanning. jpa includes automatic scanning for classes that are persistable (marked with annotations) and then including them in the persistence automatically. i think we don't want this automatic things since we will offer more persistable classes then we want to use in the default configuration. it might be turned off, but i'm not sure.

                                as far as i can estimate those issues, it should be possible to work around it. we can extract that in an interface, but all of this complexity is only needed if you actually will have implementations for it.

                                i think that you can only make good abstractions if you actually match them with the needs of the different use cases. if you make an abstraction up front, it never turns out to have the pluggability you need when you actually start using it. that is why i insist that we set up another impl from the beginning if we go that route. otherwise there's no point in building the extra pluggability layer in the hopes it will work when we need it.

                                i seem to remember that there were more issues or difficulties with JPA, but can't recal more then the ones given here. i'll add them when they pass my mind again.

                                • 13. Re: db test infrastructure
                                  tom.baeyens

                                  here's a thing that i don't know how to do it with JPA:

                                  install a listener on loading of ExecutionImpl's so that the current environment can be injected automatically.

                                  • 14. Re: db test infrastructure
                                    porcherg

                                    there is a postLoad annotation to specify a callback method called when the object is loaded.
                                    JPA defines Entitylisteners that can be used to observe this kind of event.

                                    Maybe that can be a solution ?