popen and unicode

[Stuart McGraw]

When I run (Python 2.3.3) this script on a MS Windows 2000 machine:

import os
print "1st popen..."
f = os.popen ("echo " + u"\u0054\u0045\u0053\u0054.txt")
print f.read ()
print "2nd popen..."
f = os.popen ("echo " + u"\u5927\u7530\u8061\u7f8e.txt")
print f.read ()

I get this:

1st popen...
TEST.txt

2nd popen...
Traceback (most recent call last):
File "test.py", line 6, in ?
f = os.popen ("echo " + u"\u5927\u7530\u8061\u7f8e.txt")
UnicodeEncodeError: 'ascii' codec can't encode characters in position 5-8: ordinal not in range(128)

Does os.popen() work with unicode strings containing characters outside
the ASCII range? (I think I answered my own question, above.)

How can I call popen to process non-ascii filenames on a Windows box?
Is there anything else I could use (e.g. spawn) that will work with unicode
filnames?

 

[Neil Hodgson]

Stuart McGraw:

Does os.popen() work with unicode strings containing characters outside
the ASCII range? (I think I answered my own question, above.)

There is no Unicode support in the current implementation of popen. This
could be added in a similar way to the Unicode file name support but it is
more work than I currently have time for. If you are interested in fixing
this, the code is in dist/src/Modules/posixmodule.c _PyPopenCreateProcess.

How can I call popen to process non-ascii filenames on a Windows box?
Is there anything else I could use (e.g. spawn) that will work with unicode
filnames?

You can use ctypes to call the underlying platform APIs.

Neil

 

[Stuart McGraw]

Thanks for the info, Neil.

I tried using spawnv too, with slightly different but still
unsuccessful results. With spawn, the spawned program
is receiving unicode characters, but non-ascii range
characters have been converted to unicode "?" before
the spawned program receives them. .

I looked into calling the Windows API using Ctypes but how
to convert a C *File pointer returned by Windows' popen()
to something that Python could use, was not at all clear to
me. I guess it would be fairly simple with a C extension, but
that is a bit beyond me at this point.

So, to my surprise, I conclude that it is not currently feasible
to work with the process stuff and unicode file names on a
Windows machine with Python. I would still love to find out
I am wrong though. And I think this info should be in the
documentation. It would have saved me a couple day's
work.