Look at ClientLoginModule.java, which can be found in the jboss src archive, for some ideas on how to solve your problem. The anwser lies in the SecurityAssociation method calls.
if your webapp runs in the same vm, you must provide 2 security domain entries in login-config.xml for your application (or even more, if you need)
the first one is for your application and the second one is for your client, remember the webapp (servlet/jsp) now is your client. for the client side you must use the JBOSS ClientLoginModule and for your App you use your own JAAS or JBOSS loginmodules. i use it in this way and of course it works