strange behaviour of str()

[Juho Vuori]

Hello,

I'm wondering about the following behaviour of str() with strings
containing non-ASCII characters:

str(u'foo') returns 'foo' as expected.

str('lää') returns 'lää' as expected.

str(u'lää') raises UnicodeEncodeError

Is this behaviour sane? Possibly, but not documented at all. Somehow
you'd expect str() to never fail. I'm hesitating about sending a bug
report about a third party application, which fails because it is
relaying on this.

Cheers,
Juho Vuori

 

[Fredrik Lundh]

str(u'lää') raises UnicodeEncodeError

Is this behaviour sane? Possibly, but not documented at all.

str() on a Unicode string attempts to convert the string to an 8-bit
string using Python's default encoding, which is ASCII. "ä" is not
an ASCII character.

if this problem appears in a 3rd party program, that program has
not been properly internationalized.

Somehow you'd expect str() to never fail.

except for id() and type(), virtually all builtins can fail. If you want
to convert something to a string no matter what it contains, repr() is
a better choice. If you want to convert Unicode strings to a given
byte encoding, you have to use the encode method.

</F>

 

[Uwe Schmitt]

=20

Hello,
=20
I'm wondering about the following behaviour of str() with strings=20
containing non-ASCII characters:
=20
str(u'foo') returns 'foo' as expected.
=20
str('l=E4=E4') returns 'l=E4=E4' as expected.
=20
str(u'l=E4=E4') raises UnicodeEncodeError
=20

This does not work, because you need an encoder to convert
unicode to str. str() does not know a priori which encoder
to use. There are many ways to encode a unicode string
to a classic byte-stream based string.

you have to procede as follows:
=E4=E4=E4

try "utf-8" and "utf-16" instead of "latin-1"

Greetings, Uwe.