Yes, this is fairly straightforward.
You need to make 2 entries for each of your servlets (compiled from the JSP) in the web.xml file and to ensure that the servlet source files are compiled and placed under the meta-inf/classes directory (as for any other classes your web app uses).
to register the servlet class with a servlet name.
and then to map the servlet name to a url pattern
This can be a bit tedious if you have a large number of pages, so you you'll probably be better of autogenerating this part of the web.xml in you builds scripts.
Alternatively, someone else might have a simpler strategy.
Maybe there is some flag you can set that would ensure all JSPs are precompiled as the war file is deployed?
There are a number of measures you can take to ensure pre-compilation when the web app is deployed (it still takes time though for the first compilation).
You can add a load-on-startup for your JSP.
For Jetty, you can prevent recompilation of JSPs between JBoss bounces via WEB-INF/jetty-web.xml on a per web-app instance:
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE Configure PUBLIC "-//Mort Bay Consulting//DTD Configure 1.2//EN" "http://jetty.mortbay.org/configure_1_2.dtd">
The scratchdir is not known by JBoss so will not be cleaned up on a JBoss bounce. I have not tried this but the Jetty team provided this info.