I'm going to ask and answer my own question because this took me a while to deal with and it might help someone else.

When upgrading from 4.0.3SP1 to 4.0.4GA, and when using Oracle instead of HSQL, it looks like the example oracle-jdbc3-service.xml is no longer supplied, and the 4.0.3SP1 version no longer works. So you need to use instead the oracle-jdbc2-service.xml but don't deploy it in deploy/jms but rather in deploy-haslington/jms.

Then don't delete hssqldb-state-service.xml, because in 4.0.4GA it is now needed, but you may want to rename it to be oracle-state-service.xml.

Also on a different note, jbossall-client.jar no longer contains the servlet stuff, but you have to add javax.servlet.jar to your development environment.