Found out that if I change the encoding in the browser to Cp1252 instead of UTF-8 then the page displays ok.
So, why do I end up with a UTF-8 page that the browser thinks is Cp1252 ?!?
Found that if I change the character encoding in my template to "iso-8859-1" the faulty error page shows up correct in all situations.
To fix the problem I will create a separate template for the 404 page using iso-8859-1 and let the other pages have UTF-8.
Curious if anyone has any ideas about the cause of the problem.