OK - I've spent an entire day working on this problem. I think I know what's going on. We're on an older version of jBoss (2.2.2), so it may have been corrected in more recent versions.
Custom login modules need to provide two main methodes - login and getRoleSets. What is happening is that getRoleSets gets called even if login fails(!), and the roles that are returned are cached. For the next login module in series, the login may succeed, but the roles for this successful login are not retrieved becuase jBoss already has cached roles from the previous login attempt.
The workaround that I am using is that whenever getRoleSets is called I do a callback on the userid and password and re-authenticate. If the authentication fails, I throw a LoginException rather than pass back a list of roles which will be cached and may conflict with a future successful login.
The sample login module (UsersRolesLoginModule) exhibits this problem. I haven't tested the others that come with jBoss.