Testsuite -refactoring to make test easier to run

[maeste]

Hi all,

I'm working on our test suite. I'm trying to figure out not only how to achieve a better coverage, but also how to make current and future tests easier to read, write and run. Currently a lot (almost all) of our unit tests are deploying something on an embedded version of ironjacamar standalone server and then verify the deployment is correct and or stress some behavior of deployment environment.

And it's fine, since it is stressing an almost real environment. The problem there is this test need to deploy a rar file, and this rar have to be packaged before. We are hitting this goal with a dedicated ant task called prepare-test that is getting right classes and xmls and package all the rar needed for a group of tests.

The problem with this approach is that we always need to run them with ant, making much more difficult to run them inside an IDE and/or in a debug mode. Moreover, the opportunity for potential new contributor to sit in front of or code and play with a particular feature would be much easier if they can just take a unit tests and run it in debug mode and go step-by-step through our code.

Anyway to make this long story short, I'd like to purpose a refactoring of our test suite to make it possible like what I've done for this single test:

https://github.com/maeste/IronJacamar/commit/902423fdd0f6976a0c1d1d386d3b4a1a04b8b8aa

Of course some minor refinement should be done (like rename the test method calling it something like "shouldRegisterJndiNameAsSpecifiedByIronjacamarXml()"....much easier to understand for new contributors what is the purpose of the test), but you this commit should give you an idea. Please take a look and let me know your feedack

best regards

S.

[jesper.pedersen]

Anyway to make this long story short

Too late

The patch looks good - it should be a lot clearer how to add new test cases.

However, there is one major area that needs to be looked at - the EmbeddedJCA class - as it isn't optimal for both our users but also for us in our test suite.

It needs to be rewritten as a factory/interface pair, so the internals are more hidden from the user -- and the test suite only methods needs to go away. I'll take a look at this after the holiday as discussed on IRC.'

For now just add a (boolean, ClassLoader) constructor in order to continue.

Thanks for looking into this

[maeste]

Jesper Pedersen wrote:

However, there is one major area that needs to be looked at - the EmbeddedJCA class - as it isn't optimal for both our users but also for us in our test suite.

Oki I've opened a jira about test cases refactoring. I'll wait to start it because I think we should make another step and make sure that embeddedJCA could be used in our too test using @RunWith(Arquillian.class).

IOW if we are going to support Arquillian wouldbe cool to use it also into our test suite.

S.

[maeste]

About this refactoring, I think we should also verify AS6 test suite and port all tests (or verify the already done porting in most cases) refactoring them to use embedded/arquillian approach.

best regards

S.

[jesper.pedersen]

I'll wait to start it because I think we should make another step and  make sure that embeddedJCA could be used in our too test using  @RunWith(Arquillian.class).

What do you mean by this ?

There is an example of using Arquillian in our test suite already (./embedded/src/test/java/org/jboss/jca/embedded/unit/ArquillianTestCase.java). More advanced ways of using Arquillian have been filed against Arquillian -- Aslak promised me that those would be in the Beta release.

[jesper.pedersen]

Yes, and only port those that make sense - all the stress tests needs to go to the performance branch instead.

[maeste]

Jesper Pedersen wrote:

I'll wait to start it because I think we should make another step and  make sure that embeddedJCA could be used in our too test using  @RunWith(Arquillian.class).

What do you mean by this ?

There is an example of using Arquillian in our test suite already (./embedded/src/test/java/org/jboss/jca/embedded/unit/ArquillianTestCase.java). More advanced ways of using Arquillian have been filed against Arquillian -- Aslak promised me that those would be in the Beta release.

I know, but I think it suffers same classloading problems of plain use of embedded for which I've patched. I'd like to have an Arquillian tests without needings of prepare rar package before.

bye

S.

[jesper.pedersen]

I know, but I think it suffers same classloading problems of plain use of embedded for which I've patched.

The official Arquillian integration are meant for our end-users which will load the container configuration through the embedded container itself.

That being said there may be a need to extend this integration in our test suite in order to specify overrides - such as the configuration location and so on. This is however specific to the test suite itself and won't be shipped in the distribution. If you have requirements just posts them - this will end up in ironjacamar-embedded-arquillian-testsuite.jar.

I'd like to have an Arquillian tests without needings of prepare rar package before.

Say what ?!? Arquillian deployments are done using ShrinkWrap. What do you mean ?

[maeste]

Jesper Pedersen wrote:

Say what ?!? Arquillian deployments are done using ShrinkWrap. What do you mean ?

Yep right, I've written without double check the problem I had running it iside eclipse...the problem is on kernel startup in embedded that need dependency from embedded binaries. IOW what I'm doing here:

https://github.com/maeste/IronJacamar/blob/testRefactoring/deployers/src/test/java/org/jboss/jca/test/deployers/spec/AbstractDeployerTestCase.java#L62

would be nice to have in Arquillian test case too. Or maybe something better than that, but the key for me is the ability for users/contributors to run the test as Junit test inside eclipse without any special config/dependency definition for test runner

[jesper.pedersen]

the problem is on kernel startup in embedded that need dependency from embedded binaries

From what I can tell from the method you need something like:

EmbeddedJCA embedded = new EmbeddedJCA(false);

// For all configuration files
embedded.deploy(myConfigurationFile);

to override the configuration used by the container, correct ?

would be nice to have in Arquillian test case too.

That is why we need a test suite extension of the embedded container -- to externalize the configuration paths.

but the key for me is the ability for users/contributors to run the test  as Junit test inside eclipse without any special config/dependency  definition for test runner

Yes, the default setup should allow just to use the container/configuration on the class path.