this is possible. JBoss security is configured through a file "login-config.xml", where you declare the login modules for security domains.
There is an option for each of those configs whether the login module has to succeed or whether another module is used as fallback when it fails.
You might start with the JBoss security FAQ: http://www.jboss.org/community/wiki/SecurityFAQ
Sorry for my short reply, I don't have much more time now, but I am willing to help another day.
OK, having some more time ;-).
I think I misunderstood your question: you are still at the web level of security configuration, but I was much deeper in the security config ;-).
I think for the web layer, this is not possible. The only way I can imagine is that you call the authentication yourself using this: http://www.jboss.org/community/wiki/WebAuthentication
If the certificate login attempt fails, you might perform a user/password login.
Or you could try to find out how to extend JBoss to use multiple auth methods.
For the JBoss specific config of the multiple login modules, here is a helpful link: http://www.redhat.com/docs/en-US/JBoss_Enterprise_Application_Platform/4.2.0.cp07/html/Server_Configuration_Guide/Security_on_JBoss-Defining_Security_Domains.html
See the sample for an application-policy named "todo".
Hope this helps