Selective component activation for unit tests

[tfennelly]

We're looking at this JIRA, which is/might-be about adding the ability to selectivly activate specific components during unit test execution.

Keith came across this on his travels to Japan/Aus.  Basically... you have an application which has an associated switchyard.xml configuration.  All is rosie when the app is deployed to the full AS container, but writing tests for the application can get hairy because the test is always trying to activate everything, even when it might not need to, or (more importantly), is unable to (because it doesn't have everything it needs around it).

We need a mechanism in the test framework that allows the developer to indicate what components should/shouldn't be deployed as part of the test.

Options that have been talked about:

1. Tests need to have trimmed down configs, only containing the components that need to be deployed.  Advantage of this is clarity in the tests ... wysiwyt ( )... disadvantage is obviously the repitition/duplication of configs in the test and the potential maintenance issues that go with that.
2. Use the defined MixIns to decide which components are activated.  Advantage of this is that it might just work and the developer doesn't have to do any additional work.  Disadvantage... it assumes there's always a 1:1 relationship between the components you'll want/need to activate and the MixIns you'll be using in your test.
3. Explicitly declare the Activator types to be executed as part of the test deployment.  Advantage of this is that it's explicit.  Disadvantage is that it's dragging the developer down into layers of SwitchYard that they prob don't need to be working at (wtf is an activator and which ones should I configure?).
4. Specify XPath expressions to identify the fragments of the config to be included/excluded in the deployment process (i.e. activated).  Advantage is that it's explicit and you're not duplicating configs (as in #1) - you're pointing out the fragments of the apps config that you want activated in the test.  Disadvantage... might not be feasible because of how configs are often generated at build time (by scanners)... the developer can't see the config model they're going to be using and so prob can't specify the XPaths.

The only ones I can really see working reliably atm are #1 and #3.  All options have their disadvantages, but at least (imo) the disadvantages of #1 and #3 are predictable and workable.  I think options #2 and #4 are a bit hokey and would just lead to a myriad of other issues.

Any other options/opinions people can think of?

[rcernich]

Hey Tom,

I'm sure I'm missing some details, but I kind of like the idea behind 3, but using some sort of annotation mechanism, possibly similar to the way mixins are configured.  Also, instead of calling them "activators" why not call them "components" or "modules."  At that point, the user would be defining what their runtime looks like (i.e. they might not know what an activator is, but they do know what the SOAP component is).  Individual component types should define their own test-activator-mixin-thingy-annotation, ensuring one exists for all components.

I think it would also be helpful if this could be defined on a test suite, to be inherited by all tests in a suite.

The default behavior, should no configuration be specified, should be to include everything (i.e. preserve the existing behavior).

Best,

Rob

[tfennelly]

Thanks Rob.

Yeah... I think we need to use "friendy" names for the components we want included/excluded in/from the test deployment.  Maybe something like...

@SwitchYardTestCaseConfig( ... 
          exclude= {"binding.soap", "implementation.bpm", "service name='ProcessOrder'", "component name='ProcessOrder'"}
)

Might leave the test suite angle for now

[tfennelly]

I'm thinking that the least intrusive way of implementing this is by manipulating the switchyard model that's being deployed by the test i.e. by removing the parts that are to be excluded.  That way, we don't need to dick about with the activators at all... they can just execute as normal... they'll just have more/less components to activate.

I think this will work, but the manipulation of the model (removing stuff from it) may actually invalidate it.  I think this is a somewhat accademic point, but wondering did anyone see it as a real issue ?

[kcbabo]

We pass a list of activators to the deployment.  Can't we just compare that list to the list of what's passed in the annotation?  The list of activator types is currently "flat" in the sense that bindings and implementations have the same namespace, so something like this would be sufficient:

@SwitchYardTestCaseConfig( ...
     include = {"soap", "bpm"}

I also like the idea of supporting exclude as you have above, so this could be used too:

@SwitchYardTestCaseConfig( ...
    exclude = {"soap"}

That would deliver all the functionality I wanted when I initially hit this problem and it should be pretty clean from an implementation perspective.  If users feel that using the activator type is a bit funky, then we can gather that feedback and adjust based on their ideas/expectations.

[tfennelly]

We can do that if you prefer.  So you want all activators to have an annotation that defines its alias ("soap", "bpm" etc) and then add/remove that activator if its alias included/excluded?

[tfennelly]

I should also ask... this means you can't selectively turn off deployment of a specific soap binding, or a specific bpm process (or whatever), while other soap/bpm/etc components are activated... it's an all or nothing thing you want?

[kcbabo]

Activators don't need an annotation, they already are registered with a specific type which is a string literal.  So all we need to do is take the list of strings that passed into the test config annotation and compare that to the list of activator types we find via discovery.  As an example, you could could iterate through the list of what's passed in, then call canActivate() on each Activator that's been discovered - depending on whether the list is exclude or include, you can remove or keep the activator in the list if canActivate() returns true.

Only other thing we might want to do is add a helper enum to switchyard-test which contains a list of "well known" activator types for components that we have written.  This would only be a list of strings, so there wouldn't be a compile-time dependency on any component classes.

[kcbabo]

Tom Fennelly wrote:

I should also ask... this means you can't selectively turn off deployment of a specific soap binding, or a specific bpm process (or whatever), while other soap/bpm/etc components are activated... it's an all or nothing thing you want?

Yep, all or nothing.

[tfennelly]

okidoki... consider it done