JBoss 3.0.0/Tomcat 4.0.2 not showing ISO8859-1 accented char

[pazu]

Hello, folks.

This time a got a problem wih JBoss 3.0.0/Tomcat 4.0.2 showing accented (ISO8859-1) characters. Worse: it stops writing any string at the first accented chacter.

I have a JSP page that fetches data from a database and then shows it on the browser. (BTW, I'm using Struts to ease my work). Everytime I write a value from the database that contains an accented chacter, the generated page shows only the string up to the last non-accented character. For example, SERVIÇO would be shown only as SERVI.

Attached goes the JSP page so you can take a close look. BTW, this is only a test page, so no complaints about the lack of structure, directly calling EJB's and the like. The final JSP page will probably use a SLSB and/or custom tags.

[pazu]

I miss the old mailing list. People were *so* more helpful than here...

[wimflam]

This sort of problem tends to occur when you have a code page mismatch somewhere.

Java runs in UTF8, databases have their own defaults (USACII7 on Oracle), and web-browsers tend to guess what code-page to use.

I have had similar problems to this many times over the last few years. Here is a list of things to check:

1. That your JDBC driver and database are set to use the same code page. Oracle (fat) drivers use the NLS_LANG settings to determine what code page to decode with. In Postgres the default is LATIN1, but you can override this in the JDBC URL.

2. If the the drivers and the database match correctly, the Java should happily convert your data into UTF8. You could test this by adding some log messages in your code.

3. If the String that java puts in your log messages is correct, then the problem must exist in how the web-browser is detecting the code page.
Check that any charset headers, or meta tags you are sending back contain the correct character set.
If you are not setting the character set anywhere, then this is probably your problem.

[crypto]

What version of struts are you using.
I had the same problem with struts 1.0,
it was fixed afterwards by one of the daily builds
and should have been rolled into version 1.0.2
So , if you are not yet using Struts 1.0.2, switch to the most recent release as that could solve your problem.

crypto

[pazu]

I'm not sure about which struts version I'm using -- I'll be updating to the latest nightly builds anyway. I'll be needing indexed tags.

My DB is PostgreSQL 7.0.x. All my databases are using UNICODE encoding. How do I check which encoding the driver is using and how can I specify it as a specific value?

[wimflam]

I bet this is your problem.
The driver uses LATIN1 by default. This means that you have almost certainly got a mismatch.

According to the Postgres JDBC driver docs, you should be able to set the encoding in JDBC url, like this:

jdbc:postgresql:mydatabase?charSet=UNICODE

Unfortunately I have to report that I have not been able to get this to work.
Additionally if it does work, it probably only works in recent JDBC drivers.

I had exactly the same problem as you. My solution was to recreate the database with LATIN1 encoding, as this is the default used by the driver.
As you are only worried about ISO-8859-1 characters, this should work for you also.

I would also recommend upgrading you database to Postgres 7.1 or 7.2 as they added significant improvements to 7.1.

If you get it to work with UNICODE, I'd be interested to hear how, as I too would like to use UNICODE.

[wimflam]

I think I just answered my own riddle.

The charSet property should be set to "utf-8" not "UNICODE" as I indicate above.

This seems to solve the problem.