Encodign issue in Python 3.3.1 (once again)

[Íßêïò Ãêñ33ê]

This is the code that although correct becaus it works with englisg(standARD ASCII letters) it wont with Greek:

if( log ):
name = log
# print specific client header info
cur.execute('''SELECT hits, money FROM clients WHERE name = %s''', (name,) )
data = cur.fetchone()
=======================

The following is the live output of: tail -F /usr/local/apache/logs/error_log &


[Sun May 26 19:24:04 2013] [error] [client 46.12.46.11] Error in sys.excepthook:, referer: http://superhost.gr/cgi-bin/pelatologio.py
[Sun May 26 19:24:04 2013] [error] [client 46.12.46.11] UnicodeEncodeError: 'ascii' codec can't encode characters in position 2050-2056: ordinal not in range(128), referer: http://superhost.gr/cgi-bin/pelatologio.py
[Sun May 26 19:24:04 2013] [error] [client 46.12.46.11] , referer: http://superhost.gr/cgi-bin/pelatologio.py
[Sun May 26 19:24:04 2013] [error] [client 46.12.46.11] Original exception was:, referer: http://superhost.gr/cgi-bin/pelatologio.py
[Sun May 26 19:24:04 2013] [error] [client 46.12.46.11] Traceback (most recent call last):, referer: http://superhost.gr/cgi-bin/pelatologio.py
[Sun May 26 19:24:04 2013] [error] [client 46.12.46.11] File "pelatologio.py", line 122, in <module>, referer: http://superhost.gr/cgi-bin/pelatologio.py
[Sun May 26 19:24:04 2013] [error] [client 46.12.46.11] cur.execute('''SELECT hits, money FROM clients WHERE name = %s''', (name,) ), referer: http://superhost.gr/cgi-bin/pelatologio.py
[Sun May 26 19:24:04 2013] [error] [client 46.12.46.11] File "/opt/python3/lib/python3.3/site-packages/pymysql/cursors.py", line 108, in execute, referer: http://superhost.gr/cgi-bin/pelatologio.py
[Sun May 26 19:24:04 2013] [error] [client 46.12.46.11] query = query.encode(charset), referer: http://superhost.gr/cgi-bin/pelatologio.py
[Sun May 26 19:24:04 2013] [error] [client 46.12.46.11] UnicodeEncodeError: 'latin-1' codec can't encode characters in position 46-52: ordinal not in range(256), referer: http://superhost.gr/cgi-bin/pelatologio.py


I can udnerstand that this is an encoding issue but i dont knwo how to fix this.
please help.

The search works only for english letters as expected but my cleints have Greek name and surnames.

 

[Íßêïò Ãêñ33ê]

Any idea how to correct this encoding issue?

 

[Terry Jan Reedy]

This is the code that although correct becaus it works with englisg(standARD ASCII letters) it wont with Greek:

if( log ):
name = log
# print specific client header info
cur.execute('''SELECT hits, money FROM clients WHERE name = %s''', (name,) )
data = cur.fetchone()
=======================

The following is the live output of: tail -F /usr/local/apache/logs/error_log &


[Sun May 26 19:24:04 2013] [error] [client 46.12.46.11] Error in sys.excepthook:, referer: http://superhost.gr/cgi-bin/pelatologio.py
[Sun May 26 19:24:04 2013] [error] [client 46.12.46.11] UnicodeEncodeError: 'ascii' codec can't encode characters in position 2050-2056: ordinal not in range(128), referer: http://superhost.gr/cgi-bin/pelatologio.py
[Sun May 26 19:24:04 2013] [error] [client 46.12.46.11] , referer: http://superhost.gr/cgi-bin/pelatologio.py
[Sun May 26 19:24:04 2013] [error] [client 46.12.46.11] Original exception was:, referer: http://superhost.gr/cgi-bin/pelatologio.py
[Sun May 26 19:24:04 2013] [error] [client 46.12.46.11] Traceback (most recent call last):, referer: http://superhost.gr/cgi-bin/pelatologio.py
[Sun May 26 19:24:04 2013] [error] [client 46.12.46.11] File "pelatologio.py", line 122, in <module>, referer: http://superhost.gr/cgi-bin/pelatologio.py
[Sun May 26 19:24:04 2013] [error] [client 46.12.46.11] cur.execute('''SELECT hits, money FROM clients WHERE name = %s''', (name,) ), referer: http://superhost.gr/cgi-bin/pelatologio.py

[Sun May 26 19:24:04 2013] [error] [client 46.12.46.11] File "/opt/python3/lib/python3.3/site-packages/pymysql/cursors.py", line 108, in execute, referer: http://superhost.gr/cgi-bin/pelatologio.py
[Sun May 26 19:24:04 2013] [error] [client 46.12.46.11] query = query.encode(charset), referer: http://superhost.gr/cgi-bin/pelatologio.py
[Sun May 26 19:24:04 2013] [error] [client 46.12.46.11] UnicodeEncodeError: 'latin-1' codec can't encode characters in position 46-52: ordinal not in range(256), referer: http://superhost.gr/cgi-bin/pelatologio.py

This appears to be an issue with your mysql database and how you set it
up.. It is using charset='latin-1' whereas you need it to consistently
use charset='utf8' both for storing strings and for retrieval. Check the
mysql manual or ask on a mysql list for how to do that. I have no idea
so don't ask me, or send mail to my emall address.

 

[Íßêïò Ãêñ33ê]

No thi is not a mysql issue becaus ei have this line above for storing and retrieval form database.

con = pymysql.connect( db = 'metrites', host = 'localhost', user = 'me', passwd = 'somepass', init_command='SET NAMES UTF8' )

 

[Chris Angelico]

No thi is not a mysql issue becaus ei have this line above for storing and retrieval form database.

con = pymysql.connect( db = 'metrites', host = 'localhost', user = 'me', passwd = 'somepass', init_command='SET NAMES UTF8' )

Research MySQL and UTF-8, then see how your command is actually
flawed. Then research ESR and asking questions the smart way, and see
how your post is also flawed.

ChrisA

 

[Mark Lawrence]

No thi is not a mysql issue becaus ei have this line above for storing and retrieval form database.

con = pymysql.connect( db = 'metrites', host = 'localhost', user = 'me', passwd = 'somepass', init_command='SET NAMES UTF8' )

No wonder the Greek economy is so screwed up.

 

[Carlos Nepomuceno]

----------------------------------------

To: (e-mail address removed)
From: (e-mail address removed) [...]
No wonder the Greek economy is so screwed up.

--
If you're using GoogleCrap™ please read this
http://wiki.python.org/moin/GoogleGroupsPython.

Mark Lawrence

LOL LOL LOL LOL LOL LOL HAHAHAHAHA!!!!

 

[Chris Angelico]

No wonder the Greek economy is so screwed up.

Do you think the Greek economy would be improved by Python coming with
PostgreSQL bindings? That'd push a lot of people onto a better
database engine than MySQL.

ChrisA

 

[Mark Lawrence]

Do you think the Greek economy would be improved by Python coming with
PostgreSQL bindings? That'd push a lot of people onto a better
database engine than MySQL.

ChrisA

Even the best economies would be improved by Python coming with
PostgreSQL bindings, but if and only if the user knew how to use them

 

[Íßêïò Ãêñ33ê]

But iu have it set up for 'utf-8' as seen in this statement.

con = pymysql.connect( db = 'metrites', host = 'localhost', user =
'me', passwd = 'somepass', charset='utf-8', init_command='SET NAMES UTF8' )

Yoiu mean i shoudl chnag eit to greek isoo= (iso-8859-7)

but then i store english names and greek names as well, so it has to be a utf-8.

 

[Íßêïò Ãêñ33ê]

I cant solve this plz help!

 

[Michael Torrie]

But iu have it set up for 'utf-8' as seen in this statement.

con = pymysql.connect( db = 'metrites', host = 'localhost', user =
'me', passwd = 'somepass', charset='utf-8', init_command='SET NAMES UTF8' )

That might not help... see below.

Yoiu mean i shoudl chnag eit to greek isoo= (iso-8859-7)
but then i store english names and greek names as well, so it has to be a utf-8.

Right. UTF-8 is the encoding you want.

I think your problem is that when the MySQL database was created, it was
not created using the UTF-8 mode, but rather latin-1 (despite your
init_command which has nothing to do with database creation). You'll
want to recreate the database and make double sure you've selected the
right character set and collation:

http://dev.mysql.com/doc/refman/5.0/en/charset-applications.html

At least that should be your first step in debugging this problem.

If the database is indeed utf-8, then you can move on to debugging why
the python code is choosing latin-1 to encode your query string.

That's my two-cents for today.

 

[Mark Lawrence]

I cant solve this plz help!

Sure, all you need do is get your cheque book out. Also have you ever
heard the expression "patience is a virtue"?

 

[Íßêïò Ãêñ33ê]

Ôç ÄåõôÝñá, 27 ÌáÀïõ 2013 5:45:25 ì.ì. UTC+3, ï ÷ñÞóôçò Mark Lawrence Ýãñáøå:

Sure, all you need do is get your cheque book out. Also have you ever
heard the expression "patience is a virtue"?

Well, if i'am gonna pay someone and i will at some point because i want my script to be also re-written using a web framework(Django) it will certainly be someone that can actually provide a so;lution instead of aksing for cheque books.

 

[Chris Angelico]

Ôç ÄåõôÝñá, 27 ÌáÀïõ 2013 5:45:25 ì.ì. UTC+3, ï ÷ñÞóôçò Mark Lawrence Ýãñáøå:


Well, if i'am gonna pay someone and i will at some point because i want my script to be also re-written using a web framework(Django) it will certainly be someone that can actually provide a so;lution instead of aksing for cheque books.

So you want to pay someone who won't ask for money?

It may be sordid, but you're going to have to discuss money at some
point if you're serious about paying someone.

Oh, and you may want to hire a typist, too. At the moment, your posts
make you appear not to care about the job.

ChrisA

 

[Íßêïò Ãêñ33ê]

Ôç ÄåõôÝñá, 27 ÌáÀïõ 2013 6:52:32 ì.ì. UTC+3, ï ÷ñÞóôçò Chris Angelico Ýãñáøå:

So you want to pay someone who won't ask for money?
It may be sordid, but you're going to have to discuss money at some
point if you're serious about paying someone.
Oh, and you may want to hire a typist, too. At the moment, your posts
make you appear not to care about the job.

I always make typos when iam typing fast.
And yes at some point i will pay a person from this groups to alter my scripts to utilize web frameworks.
To answer your question yes, i will be paying a person from this group thathave been providing me solutions to previous questions without laughing atmy expense and without requiring money frome to do so.
I have 2-3 people in mind.
I have managed to turn ALL my python 2.6 scripts to python 3.3.1 except this last problem i have to deal with.

http://superhost.gr/?page=pelatologio.py works partiall but only displaysinfo when i click on person names in English, not with Greek names.

If someone want to give me a hand with this, please do.

 

[Íßêïò Ãêñ33ê]

I have checked the database through phpMyAdmin and it is indeed UTF-8.

I have no idea why python 3.3.1 chooses to work with latin-iso only....

 

[Chris Angelico]

Ôç ÄåõôÝñá, 27 ÌáÀïõ 2013 6:52:32 ì.ì. UTC+3, ï ÷ñÞóôçò Chris Angelico Ýãñáøå:


I always make typos when iam typing fast.

Yes, but then you don't go back and correct them before hitting Send,
which is what makes you appear to care very little. Typos in real-time
chat are understandable (though even on MUDs, most people tend to
broadcast less typos than your posts have, because even there there's
time to edit before hitting Enter), but with email, it's very easy to
spend 60 seconds glancing over your post before you send it. You're
using gmail, so chances are all you need to do is use a web browser
with a spellchecker and you'll catch all of the typos; I use Google
Chrome, and it has the now-prevalent red squiggly lines under all
suspected errors. (It's not perfect, of course; quite a few
correctly-spelled words are mistakenly flagged. But it's still pretty
effective at catching typos.)

You're effectively asking people to put in a few minutes' work,
sometimes quite a few minutes, to help you. Is it too much to hope
that you'll spend one more minute on your posts?

This is something that's utterly trivial, yet a window to your mind.
It's like boarding an airliner and finding coffee stains on the
flip-down trays - the plane flies just fine with those stains, yet the
impression they give is that the crew don't care about engine
maintenance either.

ChrisA

 

[Íßêïò Ãêñ33ê]

Ôç Ôñßôç, 28 ÌáÀïõ 2013 1:18:06 ð.ì. UTC+3, ï÷ñÞóôçò Chris Angelico Ýãñáøå:

You're effectively asking people to put in a few minutes' work,
sometimes quite a few minutes, to help you. Is it too much to hope
that you'll spend one more minute on your posts?

No it is not, you are right, i should be more careful when typing and spendsome time to correct before i send my responses. I promise i will not overlook this as well as the blank line google imposes.

As fo my initial question any comments-remarks?

 

[rusi]

Ôç Ôñßôç, 28 ÌáÀïõ 2013 1:18:06 ð.ì. UTC+3, ï ÷ñÞóôçò Chris Angelico Ããñáøå:


No it is not, you are right, i should be more careful when typing and spend some time to correct before i send my responses. I promise i will not overlook this as well as the blank line google imposes.

As fo my initial question any comments-remarks?

You say you should be more careful to correct your typing and
'promise' to not overlook this.
Your next line has a 'fo' instead of a 'for'