Testing inter-version interoperability
(discussions related to http://jira.jboss.com/jira/browse/JBCACHE-419, specifically the testing of compatibility, as opposed to specific compatibility issues).
Testing for serialVersionUID compatibility is one part of compatibility testing, but isn't sufficient, as it won't detect API changes, state transfer compatibility, Externalizable implementation changes, etc. To do that we need to actually have caches from different versions interoperate. Fortunately, interoperability testing of this kind of thing is fairly limited in scope, as we just need to check that put(), remove(), prepare(), rollback(), commit() and state transfer work properly. This list may grow as we add more methods to the distributed API, but I believe that's it right now.
What I'm thinking of doing is:
1) Create a tests/interop source tree.
2) Add a couple classes with main methods -- say Main and Shutdown. Main takes an arg with a URL to a cache config file. It launches a thread that configures a cache based on the url, then opens a socket and listens on it. Shutdown's main opens a connection to the socket and sends a signal telling Main to terminate.
3) Create ant macros that can work with the above. What actual JBossCache version gets started depends on the classpath passed to the macro.
4) Create an ant task that can get the jars from a previous JBossCache dist from a repository and store them in, say /output/interop. These jars can then be used to create a classpath for #3 above. Initially, I'll probably just do this manually; hopefully I can enlist someone from QA to work out the download-from-repository stuff.
5) Testing would be as follows, in this illustration we're testing interop with 1.2.3.
a) call the Main target, launching a JBossCache instance that's using 1.2.3.
b) Start a junit process, using the current cache version in the classpath.
c) Test creates a cache, which will discover the other cache created in a)
d) Test does a bunch of calls to exercise the API.
e) Test creates a second cache, which will request a state transfer from the cache created in a). This tests state transfer. Once state is received, do assertions to ensure that the state is as expected, thus verifying the calls from d).
f) ant calls the Shutdown target
g) Repeat a-f, except switch the cache versions.
h) repeat a-g, except switch the config files to use REPL_SYNC.
