It's the JVM that caches DNS entries forever by default - which is an annoying thing for it to do.
You can change the settings in two ways:
1) Set "sun.net.inetaddr.ttl" system property
e.g. -Dsun.net.inetaddr.ttl=30 in JAVA_OPTS when starting JBoss to limit the caching to 30 seconds
2) Set "networkaddress.cache.ttl" security property (I assume in a policy file) - this does the same thing and isn't a sun specific property so it isn't liable to change.
There's some documentation on this here: http://java.sun.com/j2se/1.4.2/docs/guide/net/properties.html