Problems with joining Unicode strings

[Ulysse]

Hello,

I have problems with joining strings.

My program get web page fragments, then joins them into one single web
page. I have error when I try to join these fregments :
"UnicodeDecodeError: 'ascii' codec can't decode byte 0xe9 in position
208: ordinal not in range(128)"

Here is my code :

# Parcours des RSS
f = open('soupresult.html', 'w')
d = feedparser.parse(rss_url)
separator = '<hr>'
data = ''
refresh_time = 2*60*60 #Secondes
resume_news = []

# Parcours des RSS
if len(d['items']) > 0:
now = datetime.datetime.now()
for item in d['items']:
item_date = item.date_parsed
print item_date
item_date2 = datetime.datetime(item_date[0], item_date[1],
item_date[2], item_date[3], item_date[4], item_date[5])
age_item = now - item_date2
age_item = age_item.days*24*3600 + age_item.seconds

if age_item < refresh_time:
url = item['link']
print item.title
try:
req = urllib2.Request(url)
browser = urllib2.urlopen(req)
data = browser.read()
clean_item = data
resume_news.append(item.title)
resume_news.append(clean_item)

except urllib2.URLError, e:
print e.code
f.write(u''.join(resume_news))

 

[John Machin]

Hello,

I have problems with joining strings.

My program get web page fragments, then joins them into one single web
page. I have error when I try to join these fregments :
"UnicodeDecodeError: 'ascii' codec can't decode byte 0xe9 in position
208: ordinal not in range(128)"

Here is my code :

[snip]

Contrary to your message subject, you are *not* joining "Unicode
strings". Your resume_news contains at least one str (8-bit string)
objects [probably all of them are str!]. You need to decode each str
object into a unicode object, using whatever encoding is appropriate
to that str object. When you do
u''.join(sequence_including_str_objects), Python attempts to decode
each str object using the default 'ascii' encoding ... this of course
fails if there is a non-ASCII character in the str object.

This may help: www.amk.ca/python/howto/unicode

Cheers,
John