Warp extension for testing RESTful services

[jmnarloch]

Arquillian already allows for deploying JAX-RS services and running them in the client mode like in the provided example:

{code}
@RunWith(Arquillian.class)
public class StockServiceResourceTestCase {
    @Deployment
    @OverProtocol("Servlet 3.0")
    public static Archive createTestArchive() {
        return ShrinkWrap.create(WebArchive.class)
                .addClasses(StockApplication.class, Stock.class, StockService.class, StockServiceResource.class)
                .addAsWebInfResource("web.xml");
    }
    @ArquillianResource
    private URL contextPath;
    private StockService stockService;
    @BeforeClass
    public static void setUpClass() {
        RegisterBuiltin.register(ResteasyProviderFactory.getInstance());
    }
    @Before
    public void setUp() {
        stockService = ProxyFactory.create(StockService.class, contextPath.toString());
    }
    @Test
    @RunAsClient
    public void testGetStock() {
        Stock stock = createStock();
        ClientResponse response = (ClientResponse) stockService.createStock(stock);
        response.releaseConnection();
        Stock result = stockService.getStock(1L);
        assertEquals("Stock has invalid name.", stock.getName(), result.getName());
        assertEquals("Stock has invalid code.", stock.getCode(), result.getCode());
        assertEquals("Stock has invalid value.", stock.getValue(), result.getValue());
    }
}
{code}

But if we could introduce here Arquillian Warp and it's ability to bind the tests to server lifecycle. We could call a service and run the server assertion directly in REST service execution context.

So what would be Arquillian without another extension. I've started developing Warp extension particular for testing JAX-RS services. The good news is that I had already created a proof of concept that integrates with RESTEasy.

The fallowing test case ilustrates the possible usage:

{code}
    @Test
    @RunAsClient
    public void testWarpGetStock() {
        final Stock stock = createStock();
        ClientResponse response = (ClientResponse) stockService.createStock(stock);
        response.releaseConnection();
        Warp.execute(new ClientAction() {
            @Override
            public void action() {
                Stock result = stockService.getStock(1L);
                assertEquals("Stock has invalid name.", stock.getName(), result.getName());
                assertEquals("Stock has invalid code.", stock.getCode(), result.getCode());
                assertEquals("Stock has invalid value.", stock.getValue(), result.getValue());
            }
        }).verify(new ServerAssertion() {
            private static final long serialVersionUID = 1L;
            @ArquillianResource
            private RestContext restContext;
            @AfterServlet
            public void testGetStock() {
                assertEquals(HttpMethod.GET, restContext.getRequest().getMethod());
                assertEquals(200, restContext.getResponse().getStatusCode());
                assertEquals("application/xml", restContext.getResponse().getContentType());
                assertNotNull(restContext.getResponse().getEntity());
            }
        });
    }
{code}

The ServerAssertion is being run in the web server, where it allows to verify the response before it would be returned to the client. The RestContext is completly artificial entity which stores the current execution context.

There is no generic way to intercept JAX-RS service execution context. This is one of the new features introduced in JAX-RS 2.0, but the standard will need to first become widely supported. The idea for now is to provide suport for concreate implementations.

The target frameworks would be:

• RESTEasy
• Jersey
• CXF

The project is currently hosted here: https://github.com/jmnarloch/arquillian-extension-warp-rest

Anyone would like to join and develop one of the functionalities?

[lfryc]

Hey Jakub,

great job and congratulations for bringing second Warp's extension alive!

I'm looking forward for integrating REST extension with JavaScript testing extension and Android extension.

[blabno]

This is awsome!

I'd love to contribute, but I'm already working on RichFaces sandbox, Arquillian JRebel Extension, Arquillian mock-contexts-extension, revu-idea-plugin and some others. So I'm a bit overloaded with open source projects

But I'm keeping 2 thumbs up for this.

Lukas, what is this combination of rest and JS be? Sounds intriguing.

[jmnarloch]

Hi Bernard.

You are more then welcome to join in. I will be working in not far future on trying to prototype integration with Jersey and CXF.

Even without strict integration with Javascript extensions it will be still possible to use Arquillian Drone and WebDriver for running tests. I' going to post an example use case soon.

It might be more interesting then the simple test case from the previouse post.

[lfryc]

I can imagine native REST clients could be used as well.

But integration with JavaScript test could look like following:

test.js:

test(function() {
    Warp.execute(function() {
        $.ajax({
             url: "/rest/people",
             data: { 'sort': 'name', 'limit': 5 }),
              success: function(data) {
                     updateTable(data);
         };
      }).verify("table-update");
}

WarpedFromJavaScript.java:

@Named("table-update")
public class WarpedFromJavaScript extends ServerAssertion {

     @ArquillianResource
     RestContext context;

     public void observe(@Observe TableReadEvent tableReadEvent) {   // CDI integration
           // verify
     }
}

It would be especially useful when you will need to verify what's happenning on server during REST request,

verify side-effects, or prepare failure scenario without mocking.

[jmnarloch]

The previouse example was fairly simple. Now let's examine a more usefull use case. Let's say that we have Ajax rich front-end with back-end JAX-RS service. We could simply exercise our javascript code, but still we would end up with unit tests. Here is an example how we can write a integration test for our front-end, that access directly the server.

The backend service allows to perform CRUD operations on stocks. The service contract is pretty much simple:

{code}
@Path("/stocks")
@Consumes({MediaType.APPLICATION_JSON, MediaType.APPLICATION_XML})
@Produces({MediaType.APPLICATION_JSON, MediaType.APPLICATION_XML})
public interface StockService {
    @POST
    Response createStock(Stock stock);
    @Path("/{id}")
    @PUT
    void updateStock(@PathParam("id") long id, Stock stock);
    @Path("/{id}")
    @GET
    Stock getStock(@PathParam("id") long id);
    @GET
    List<Stock> getStocks(@QueryParam("startIndex") int startIndex);
    @Path("/{id}")
    @DELETE
    Response deleteStock(@PathParam("id") long id);
}
{code}

On "client" side we will use jquery for retriving data from the service:

{code}
             function onStockClick() {
                var stockId = $(this).attr('stockId');
                $.get('rest/stocks/' + stockId, function(data) {
                    // TODO do something with the stock
                });
            }
            function displayStocks(data) {
                var table = $('<table />').append($('<thead />')
                        .append($('<tr />')
                            .append($('<td />').text('Name'))
                            .append($('<td />').text('Code'))
                            .append($('<td />').text('Value'))
                        )
                    );
                $.each(data, function(ind, val) {
                    var row = $('<tr />');
                    $('<td />').append($('<a />', {class: 'stockLink'} ).text(val.name).attr('href', '#').attr('stockId', val.id).click(onStockClick)).appendTo(row);
                    $('<td />').text(val.code).appendTo(row);
                    $('<td />').text(val.value).appendTo(row);
                    table.append(row);
                });
                table.appendTo('#mainContent');
            }
            function getStocks(callback) {
                if(typeof callback != 'undefined') {
                    $.get('rest/stocks', callback);
                }
            }
            $('document').ready(function() {
                getStocks(displayStocks);
            });
{code}

Final thing is to write the test. In this example we will integrate Arquillian Drone and web driver for navigating to the web page.

{code}
    @Test
    @RunAsClient
    public void testAjaxGetStocks() {
        Warp.filter(new GetStocksFilter()).execute(new ClientAction() {
            @Override
            public void action() {
                browser.navigate().to(contextPath + "restclient.jsp");
            }
        }).verify(new ServerAssertion() {
            private static final long serialVersionUID = 1L;
            @ArquillianResource
            private RestContext restContext;
            @AfterServlet
            public void testGetStocks() {
                assertThat(restContext.getRequest().getMethod()).isEqualTo(HttpMethod.GET);
                assertThat(restContext.getResponse().getStatusCode()).isEqualTo(Response.Status.OK.getStatusCode());
                assertThat(restContext.getResponse().getContentType()).isEqualTo("application/json");
                List list = (List) restContext.getResponse().getEntity();
                assertThat(list.size()).isEqualTo(1);
            }
        });
    }
{code}

The test only navigates to the JSP file and loads it through selenium. The request executed made on document ready is being intercepted through Warp and the internal server state is being verified by the server assertion.

Similary we can interact with UI, for example by clicking a link which grabs from the service the details on the given Stock.

{code}
    @Test
    @RunAsClient
    public void testAjaxGetStock() {
        browser.navigate().to(contextPath + "restclient.jsp");
        Warp.filter(new GetStockFilter()).execute(new ClientAction() {
            @Override
            public void action() {
                browser.findElement(By.className("stockLink")).click();
            }
        }).verify(new ServerAssertion() {
            private static final long serialVersionUID = 1L;
            @ArquillianResource
            private RestContext restContext;
            @AfterServlet
            public void testGetStock() {
                assertThat(restContext.getRequest().getMethod()).isEqualTo(HttpMethod.GET);
                assertThat(restContext.getResponse().getStatusCode()).isEqualTo(Response.Status.OK.getStatusCode());
                assertThat(restContext.getResponse().getContentType()).isEqualTo("application/json");
                Stock stock = (Stock) restContext.getResponse().getEntity();
                assertThat(stock.getId()).isEqualTo(1L);
                assertThat(stock.getName()).isEqualTo("Acme");
                assertThat(stock.getCode()).isEqualTo("ACM");
            }
        });
    }
{code}

I hope that this ilustrates better how the Warp extension can be used, while it still remains fairly an simple example, but it gives the idea of potential usage.

[meetoblivion]

In this example, what is GetStockFilter ?

[jmnarloch]

The GetStocksFilter and GetStockFilter are used to choose the exact request that will be enriched by warp. Why the first one is mandatory becouse of the additional resources being loaded on page load like scripts, css etc. the second test quite likely would run without any filter needed.

You can find the test case here https://github.com/jmnarloch/arquillian-extension-warp-rest/blob/master/ftest/ftest-resteasy/src/test/java/org/jboss/arquillian/quickstart/resteasy/service/rs/StockServiceAjaxTestCase.java

[lfryc]

Hey Jakub, great job!

Could we define some generic filter (in REST extension) which would cover all REST requests, so user would need to use it just once per test, using annotation?

@RunWith(Arquillian.class)
@WarpTest @Filter(value=RESTRequestFilter.class, config= {"foo: bar", "some: value"})
public class MyTest {}

[jmnarloch]

Lukáš Fryč wrote:

Hey Jakub, great job!

Could we define some generic filter (in REST extension) which would cover all REST requests, so user would need to use it just once per test, using annotation?

@RunWith(Arquillian.class)
@WarpTest @Filter(value=RESTRequestFilter.class, config= {"foo: bar", "some: value"})
public class MyTest {}

I was thinking on maybe creating builder pattern in core Warp for creating the filters. Something like:

{code}
Warp.requestFilter().method("GET").uri(contains("something")).filter(new MyCustomFilter()).build();
{code}

It would allow on easy chaining the filters and reuse them.

{code}
WarpRequestFilterBuilder builder =  Warp.requestFilter().uri(contains("/rest_service"));
{code}

And afterwards use that in sequential invocations:

{code}
filter1 = builder.uri(endsWith("/path1")).build();
filter2 = builder.uri(endsWith("/path2")).build();
{code}

[lfryc]

Sounds great.

import static org.jboss.arquillian.warp.filter.HttpFilters.*;

HttpFilterBuilder restFilter = uri(contains("/rest/service"));

Warp.defaultFilter(restFilter);

Warp
     .execute(...)
     .filter(method(GET).uri(endsWith("/path1")))
     .verify(...);

Could you please create a feature request for filter builder API?

Feel free to start on hack on it - underlying Filter<HttpRequest> interface might still get small modifications, but in general it won't change much.

The current branch which will be soon merged into master: https://github.com/lfryc/arquillian-extension-warp/tree/ARQ-967

For a sake of genericity I would provide Filter<HttpRequest> which could be used for implementing general purpose filters - e.g. JSF filter.

Those filters could read application configuration in order to determine how to filter requests correctly or they can be configured explicitly.

[lfryc]

I have also plan to introduce server-side filters as well - those filters will take advantage of using server state to determine if request should be filtered.

Obviously, assertions would need to make sure the don't have side-effects before it is decided request should be filtered or not.

Warp can log warnings for such a methods.

[dan.j.allen]

This is thrilling stuff guys! You are definitely making the case for why the testing world should be paying attention to the activity in Arquillian (as if we didn't already give them enough reasons )

Jakub, since you are covering all the functionality of pure client-side REST testing, do you think you could also incorporate the ideas from the REST extension that Aslak had hacked up during Devoxx 2011 (a year ago)? (or decide what we want to do with it, because it's been idle for awhile).

https://github.com/arquillian/arquillian-extension-rest/blob/master/src/test/java/org/jboss/arquillian/extension/rest/RestClientTestCase.java

The basic idea of that extension is to hide the client code for boilerplate requests, possibly mixing them w/ parameterized tests to ensure the endpoints work with a sampling of values.

I learned from Aslak last week that the client proxies aren't going to make it in JAX-RS 2.0. That gives me reason to propose creating an abstraction layer so we can support the major implementations transparently. We definitely want to get any references to RESTEasy out of the test for the average test developer...keeps the test portable and clear.

Lukas, I know you've been asking about ideas for the next iteration of Warp. Whenever I see the ClientAction, I want to change it in my mind to be ClientInvoker#invoke(). For some reason, that seems more natural to me. I think it's because "action" is so strongly associated with server-side web frameworks that I keep thinking that is server-side code at first (even though it clearly says client). (It so happens that RESTEasy uses ClientInvoker for it's client wrapper too). WDYT?

Btw, JavaScript testing FTW! Testing was huge this year at Devoxx, and lots and lots of it was JavaScript testing. Everyone is asking, "How do I automated JavaScript testing in the browser?" and "How do I test client and server interactions in a TDD way?" Keep on the hunt, because this is big stuff.

[lfryc]

Lukas, I know you've been asking about ideas for the next iteration of Warp. Whenever I see the ClientAction, I want to change it in my mind to be ClientInvoker#invoke(). For some reason, that seems more natural to me. I think it's because "action" is so strongly associated with server-side web frameworks that I keep thinking that is server-side code at first (even though it clearly says client). (It so happens that RESTEasy uses ClientInvoker for it's client wrapper too). WDYT?

I would carry Warp methods naming discussion further in a specific topic: fluent API for request execution

[jmnarloch]

Dan Allen wrote:

This is thrilling stuff guys! You are definitely making the case for why the testing world should be paying attention to the activity in Arquillian (as if we didn't already give them enough reasons )

Jakub, since you are covering all the functionality of pure client-side REST testing, do you think you could also incorporate the ideas from the REST extension that Aslak had hacked up during Devoxx 2011 (a year ago)? (or decide what we want to do with it, because it's been idle for awhile).

https://github.com/arquillian/arquillian-extension-rest/blob/master/src/test/java/org/jboss/arquillian/extension/rest/RestClientTestCase.java

The basic idea of that extension is to hide the client code for boilerplate requests, possibly mixing them w/ parameterized tests to ensure the endpoints work with a sampling of values.

Can you just satisfy my curiosity. I think that such aproach can be great for GET/DELETE requests, but have you figure out what to do with POST and PUT requests? Would it be possible to marshall an entity and set it to the service in such declarative approach?