Checking Exceptions in tests

[beve]

I'm using SwitchYardTestCase and invoking services using newInvoker. I like to use JUnits @Rule feature to verify the exception thrown and also the exception message. When using the SwitchYardTestCase and JUnit's ExpectedException it works, but I can only verify that the exception thrown is of type InvocationFaultException. What I'd really like is to check that a Service implementation threw the exception that I expect.

I've implemented an example of what I'd like to be able to do. If other find this useful perhaps we could move the SwitchYardExpectedException class to a different location, or perhaps bake it into the SwitchYardTestCase class itself.  

Any thoughts on this? 

/Dan

[tfennelly]

Hey Dan.

Tbh... I found the @Rule + code inside the method a bit confusing.  I had to look at it for a bit before I groked what was happening.

So with the @Rule you'd have a class level property annotated with @Rule and then some code inside the method:

@Test
public void shouldThrowRuntimeExceptionFromCamelRoute() throws Exception {
    thrown.expect(CustomException.class);
    thrown.expectMessage("dummy exception");
    newInvoker("OrderService").operation("getTitleForItem").sendInOut("10");
}

Would the intent be clearer if it was all done on a test method annotation ala (i.e. no @Rule preropty + no code inside the method)...

@Test
@ExpectedException(type = CustomException.class, message = "dummy exception")
public void shouldThrowRuntimeExceptionFromCamelRoute() throws Exception {
    newInvoker("OrderService").operation("getTitleForItem").sendInOut("10");
}

[beve]

Would the intent be clearer if it was all done on a test method annotation ala (i.e. no @Rule preropty + no code inside the method)...

I was also a bit put off when I first saw the @Rule syntax for expected exceptions so I'd prefer to use your suggestion as I think that would be cleaner.

[kcbabo]

I'm definitely interested in both easier and stricter testing of exception activity.  It would be great to have this when we resolve SWITCHYARD-78, which is long overdue.

In terms of the @Rule thing, I also found it a bit confusing.  Tom's suggestion is definitely easier to understand.  Haven't really thought it through all the way, but I would lean toward that. 

One thing we need to be careful about here is string equality comparison for exception messages.  This is super brittle when messages are not resolved outside of constants and also fails when localization is introduced.