We started stabilization and performance directed works right after 3.2.0 release (we had user voting about the directions and this way won).. So many components was optimized during this time. And I could highly recommend you to try 3.3.1 GA in order to check this. Also we designed http://www.jboss.org/community/docs/DOC-13329 dependencies plugin in order to reduce resource dependencies to just needed ones.
Next that should be mentioned is 4.x release which already has in plans semantic layouts, JS dependencies clearing and so on. Partially such works will be done in 3.3.x branch also.(http://www.jboss.org/index.html?module=bb&op=viewtopic&p=4234141#4234141)
Browse among our wiki pages in order to get more info about work done and planned ;) And for sure we appreciate further feedback about RichFaces improvements.