P.S. We use form-based login without JAAS and would not like to add a login module if possible. We just want certain extensions to be inaccessible via direct HTTP request.
I know this doesn't address the problem you're having WRT security, but perhaps there's another way to achieve what you want.
You could write a very simple servlet whose service method simply sends a 404 back to the response. Then map *.vax to that servlet. This will make it look like those files don't exist via a direct HTTP request.
Using a security constraint, the response would be something like a 401 or 403. It could also cause the browser to be redirected to your login page.
From the Servlet 2.4 spec, section SRV.12.8:
An authorization constraint that
names no roles indicates that access to the constrained requests must not be
permitted under any circumstances.
So, an empty authorization constraint element should get you what you want.
I must admit I'm not certain why your "fake role" didn't work, but in any case the authorization constraint with no roles is the standard way to do it.
Thanks for replies. Turns out I cannot eliminate every URL access to those resources so I will have to use a bona fide role. Since we use our own home brewed authentication/authorization, a servlet filter should do the trick.