I've had the same problem in JBoss-5.1.0.GA. I was trying to run CAS (the jasig common authentication server) on JBoss with it using my existing prc4 application policy to authenticate users. However, it was failing with a null pointer exception in the logs. Luckily, after a while (several hours, I'm afraid) we tried using the jmx-console policy instead and this worked. A bit more experimentation showed that all the login modules worked except the DatabaseServerLoginModule.
I wrote a class that extended DatabaseServerLoginModule and had a lot more logging in it. This confirmed that the lookup of the datasource was returning null. I then got the class to print out the names and bound values in the java:/ namespace and found that prc4 was there and was bound correctly to a nmon-null value of the right class. However, I still couldn't retrieve it.
I was about to write code that would create a new db connection of the ds was null when I came across this post. I followed Prabhat Jha's advice abd checked if cas.war had any jboss libraries in it and found jboss-common-core-2.2.2.GA.jar. When I removed this it all started working. I think I hadn't found this thread when searching as I was exlicitly mentioning JBoss 5. I'm hoping that this post of mine will help other lost souls to find the solution.
Many thanks and much kudos to Prabhat Jha!!