Encoding troubles

[Xaver Hinterhuber]

Hello pythonistas,

I program a class which stores the source code for an output page in a
string.
At request time it compiles it, executes it and returns the result.
I now have upgraded the class from python 2.1 to python 2.3.
So I have to do some encoding work I previously didn't have to do.
If I execute the appended code, then it raises me an error stating:

Error Type: UnicodeDecodeError
Error Value: 'ascii' codec can't decode byte 0xc2 in position 0: ordinal not
in range(128)

What is wrong?

The code is the following:

class Report:
"""This class is generating reports as pdf-files with the source code
provided in the variable content"""
# default values for test
content = """
Story = ['ÄÖÜäöüß?'] #german umlauts for test purposes
"""
def compileContent(self):
"""Compiles the content of the pdf-pages"""
content = self.content
# Here the error occurs
content = content.decode('iso8859-15')
codeString = HTML(content, globals())
codeString = codeString(self._getContext())
codeString = codeString.replace('\r\n', '\n')
codeString = codeString.split('\n')
if codeString[-1].strip() == '\n':
del codeString[-1]
# Code als Funktion kompilieren
codeString.append('return Story')
codeString = '\n\t'.join(codeString)
codeString += '\n'
codeString = 'def f():\n\t' + codeString
codeObject = compile(codeString, 'codeObject', 'exec')
return codeObject

def __call__(self):
dict={}
dict.update(globals())
dict.update(locals())
dict.update(kw)
codeObject = self.compileContent()
exec codeObject in dict
Story = dict['f']() # Ausführen der Funktion
return Story

 

[Peter Otten]

At request time it compiles it, executes it and returns the result.
I now have upgraded the class from python 2.1 to python 2.3.
So I have to do some encoding work I previously didn't have to do.
If I execute the appended code, then it raises me an error stating:

Error Type: UnicodeDecodeError
Error Value: 'ascii' codec can't decode byte 0xc2 in position 0: ordinal
not in range(128)

What is wrong?

The code is the following:

class Report:
"""This class is generating reports as pdf-files with the source code
provided in the variable content"""
# default values for test
content = """
Story = ['ÄÖÜäöüß?'] #german umlauts for test purposes
"""
def compileContent(self):
"""Compiles the content of the pdf-pages"""
content = self.content
# Here the error occurs
content = content.decode('iso8859-15')

Or here?

codeString = HTML(content, globals())

What does HTML() do? Are there any non-unicode strings with non-ascii
characters that you try to concatenate with content? E. g.:

'\xe4\xf6\xfc\xe4\xf6\xfc'

But

Traceback (most recent call last):
File "<stdin>", line 1, in ?
UnicodeDecodeError: 'ascii' codec can't decode byte 0xe4 in position 0:
ordinal not in range(128)

The solution is to make sure that either all strings are unicode or all
strings are non-unicode (hopefully sharing the same encoding).

Peter

 

[Xaver Hinterhuber]

Hi Peter,

[snip]

What does HTML() do? Are there any non-unicode strings with non-ascii
characters that you try to concatenate with content? E. g.:

HTML is a zope class which handles dtml-markup.

Traceback (most recent call last):
File "<stdin>", line 1, in ?
UnicodeDecodeError: 'ascii' codec can't decode byte 0xe4 in position 0:
ordinal not in range(128)

The solution is to make sure that either all strings are unicode or all
strings are non-unicode (hopefully sharing the same encoding).

This was the problem.

Thank you very much for your help.