Feedback from PAX 2011
I presented an Arquillian workshop and attended a few talks at the Project Automation Experience in Dec 2011. I'd like to share some of the feedback I collected from that event.
First, there is absolutely no question that developers are facing serious challenges with integration/functional/acceptance testing. One developer told me that his company just gave up on tests because it was just too much of a headache. Most of the attendees in the Arquillian workshop agreed that anything but unit tests are just cryptic, voodoo machine setups.
...so it's nice to know that these are real problems...something we've picked up from talks, but in a workshop it's cool to have a conversation with people.
As I began to present Arquillian, the first two questions that came up were:
- isn't the test archive going to be inconsistent with the application archive generated by the build tool?
- does it support Spring? (a question that always comes up with a NFJS audience)
To the first question, I think it's an indicator that we need to put out more clear information about the value of microdeployments and to clarify that they don't preclude using the application archive for end-to-end acceptance testing. Basically, people are looking for both guidance and assurance. Hopefully the Arquilian guides can cover this subject. If you have advice, please feel free to add it to a guide or write up a new guide.
To the second question, it's partially answered by the Seam Spring module, which let's CDI drive Spring, hence making Spring immediately available in Arquillian. However, we *really*, *really*, *really* need to get going on the native Spring archive processor and test enricher (done as an extension). We could grab tons of users by making ourselves relevant to Spring developers. I consider this a high priority for the project. Perhaps a good GSoC idea. In the meantime, we should put out a short blog on using the Seam Spring module as a means to test Spring with Arquillian. Any takers?
One attendee asked what was going on with the JSFUnit extension. I see that we finally have some movement on that again (see https://community.jboss.org/wiki/200FinalRoadmapPlanningCommunityMeeting2March2012)
One of the guys in the class is developing with Websphere and was asking about the adapter. He said he would be willing to help test it and give feedback (see the wiki page for building the WAS container adapter).
The audience remained very engaged the whole time, I think. They most appreciated the recent extensions:
- DBunit extension
- RESTEasy extension (still living in the Arquillian Showcase)
- Drone (of course, always blows people away)
- integration-style mockito (mock objects that are produced as CDI beans, still living in the Arquillian Showcase)
- Spock testrunner
I wasn't able to get to:
- jacoco
- Arquillian Extensions plugin for Forge (dang!)
- hacking on a spring enricher
It would be pretty cool if during the talk or workshop, we could setup Jenkins and get the tests running in the background...that will really drive home a point. We could use OpenShift to avoid having to configure Jenkins manually during the talk.
The other thing I'm picking up on is that we need to start establishing some resources for testing best practices (hence the JBoss Testing SIG). I'm hearing at the conference a lot about the testing pyramid. We should be advising how people should use Arquillian in their shops.
I received some additional feedback the following day.
First, when we show the CDI example, we need to pause and clearly introduce that CDI is dependency injection and that it's the standard in the Java EE platform. We can then draw on a loose association w/ Spring for those purely Spring developers. Apparently we lose some people in this example and subsequent ones since we use CDI frequently.
Tying into a point I made earlier, we need to explain better how Arquillian fits in with QA, esp existing QA operations. From the beginning we have appealed to developers, so we need to get inside the head of QA. (Thankfully, we have lots of insight from our own QE team, though getting into some other testing conferences may gain us even more perspective).
We also need to get into the JavaScript game. There was a lot of focus on it at RWX / PAX, both client and serverside. One person asked if phantom (browser controller) and qunit could be used together. The speaker was like "that would be a good idea". Yeah, hello, Arquillian
Here are some of the frameworks that came up in talks:
- JsTestDriver
- Geb
- PhantomJS
- Testling
- Jasmine
- Cucumber
- Mocha
- Watir
Some of these tools are JavaScript, some of them Ruby, all of them worth exploring for ideas.
I also saw examples of the following testing techniques & tools:
- Auto-flag slow tests, either for quality or for categorization
- Test categories by regex (or an idea I had to use Drools)
- Growl / libnotify support for failed tests
People love Arquillian, no doubt, but there are many more people still on the fence. It's going to be a fun challenge to being them over, and watch Arquillian grow and mature as a result.
