Yes, in your web.xml you need to define an error page.... look at the web.xml spec... but basically something like this
which should come just after your welcome page declarations if I am not mistaken.. but again... read the spec.
Then you need to look at the jsp error page spec, because generally you want the<%@ page isErrorPage="true" %> tag in there. This gives you access to all the error information.
Now THE TRICK that nobody tells you... IE will throw away any error page that is smaller than some magic number. So I always pad my error pages with a couple kbytes of spaces.