JDBC internationalization/localization problem Unicode/UTF-8

[Wolfgang]

I have a servlet that reads data from a mySQL database and writes them
to a Web page. (I'm using the latest mySQL 4.1.0-alpha, supporting
Unicode)
The JDBC connection string I use calls for UTF-8 encoding (which is
what the data in the database is), like

jdbc:mysql://host/database?useUnicode=true&characterEncoding=UTF-8

This works fine for some (special) characters, like the German
ä (\u00E4)
ö (\u00F6) and
ü (\u00FC)

but not for others, like

ß (\u00DF)

(this is the funny German "sharp S")

Can you think of an explanation for this problem, why does the UTF-8
decoding not work for ß (\u00DF) when it works ok for other
special characters?

Thanks,
Wolfgang

 

[Roedy Green]

Can you think of an explanation for this problem, why does the UTF-8
decoding not work for ß (\u00DF) when it works ok for other
special characters?

Find out WHERE it does not work.

Does the character go into the database and come out unmolested?

If so it is not MySQL's fault.

Did you explicitly tell MySQL to use unicode for that field?

If it is doing some translation to a national set, that translate
table could be defective.

Does the character fail to display properly in some font?

In that case the font could be at fault or the translate tables for
it.

 

[John C. Bollinger]

I have a servlet that reads data from a mySQL database and writes them
to a Web page. (I'm using the latest mySQL 4.1.0-alpha, supporting
Unicode)
The JDBC connection string I use calls for UTF-8 encoding (which is
what the data in the database is), like

jdbc:mysql://host/database?useUnicode=true&characterEncoding=UTF-8

I have found this approach to work fine with MySQL 3.23, but I have not
tested it thoroughly enough to determine whether there are any
characters on which it fails.

This works fine for some (special) characters, like the German
ä (\u00E4)
ö (\u00F6) and
ü (\u00FC)

but not for others, like

ß (\u00DF)

(this is the funny German "sharp S")

Can you think of an explanation for this problem, why does the UTF-8
decoding not work for ß (\u00DF) when it works ok for other
special characters?

Have you verified that the DB actually contains the characters you think
it does? You can check by specifying a single-byte character encoding
(ISO-8859-1 ought to work) reading the offending data, and looking at
the integer values of the resulting characters. Decode the encoded
characters by hand or with a separate tool, and compare to the Unicode
character tables.

You may want to check that the server is configured to use UTF-8. IIRC,
there is an encoding setting both in the server and in the JDBC driver;
you have described your apparently correct use of the latter, but your
trouble could be related to the former. Note: changing this setting in
your server may corrupt character data in the databases it manages.
Proceed with caution.


John Bollinger
(e-mail address removed)

 

[Wolfgang]

PROBLEM FIXED (though I'm not sure exaxtly how, read on).
Thanks for the good responses so far.

What did I do? I installed and tried mySQL 4.1.2-alpha on XP, that's
all. (My previous results came from mySQL 4.0.16-alpha on Solaris)

This made the problem go away, without doing anything else (except for
restarting the server after loading the data, which I coudn't do on
Solaris as I'm not the sysadmin there).

What all this made me realize, if nothing else, is how much gory
detail there is in internationalization/localization that I know
almost nothing about so far. I have a lot to learn here...

Thanks,

Wolfgang

 

[venukjs]

Hey i am facing problem to set the encoding UTF-8 for JDBC.

I am using the this URL,


String URL = "jdbc:mysql://localhost:3306/nttest?useUnicode=true&characterEncoding=UTF-8";



Thanks and Regards
Venu J