It depends - if they are not in your WAR or context then you probably need to use absolute references. If they are in your context, use relative references. It is generally better to use relative references as it gives you flexibility to change context at deployment without breaking your web application.
So if your JSP is at the top level directory for the WAR and the CSS is a level below at css/myapp.css you would refer to it from your JSP as:
<link rel="stylesheet" href="css/myapp.css" type="text/css">
jonlee - thank you for the reply. I will post what I come up with.