It sounds like you're doing all the correct steps to enable webapp response times. I'm guessing some sort of error is occurring on the Agent side that's causing it to fail to read the response time log files. Can you look in your agent.log file and see if you see any errors that look relevant?
Thanks Ian! Good advice - here is what I found:
2012-03-02 12:03:33,684 ERROR [ResourceContainer.invoker.daemon-756] (org.rhq.plugins.jbossas5.WebApplicationContextComponent)- The 'responseTime' metric is enabled for WAR resource 'jboss.web:host=localhost,path=/seam-dvdstore,type=Manager', but no value is defined for the 'responseTimeLogFile' connection property.
I have it working now. For anyone who needs to know this someday, I missed a step (that I don't see documented anywhere). On the resource page of your Web Application Context, go to Inventory->Connection Settings and set the "Response Time Log File" property to the full path of your *_rt.log file.
It works great now - thanks again!