os.popen3 with windows; help?

[Russell E. Owen]

I'm trying to launch an application from Python 2.3 on Windows. The
application is "ds9" (an image viewer), and is installed in C:\Program
Files\ds9\ds9

On unix I just do:
os.popen3("ds9")
and close the returned files and all is good. (I'm not trying to
communicate with the program via popen3 and so had been using
os.spawnlp, but that doesn't exist on Windows.)

On Windows os.popen3("ds9") does nothing, and the stderr returned from
os.popen3 has a message saying the program is unknown.

So I tried being specific:
os.popen3("C:\\Program Files\\ds9\\ds9")
This also fails and the program that is not found is "C:\Program",
suggesting that the space in "Program Files" is causing the problem. The
following failed in exactly the same way:
os.popen3("C:\\Program\ Files\\ds9\\ds9")

Any suggestions?

-- Russell

P.S. I installed the app using its Windows binary installer, which
unpacks a few files in C:\\Program files\. Windows doesn't seem to know
the program exists; it's not in the task bar, for example. But one can
double-click it to run it.

 

[Sylvain Defresne]

So I tried being specific:

os.popen3("C:\\Program Files\\ds9\\ds9")
This also fails and the program that is not found is "C:\Program",
suggesting that the space in "Program Files" is causing the problem. The
following failed in exactly the same way:
os.popen3("C:\\Program\ Files\\ds9\\ds9")

Any suggestions?

I don't have a windows to test, but does the following works (that is
protecting the space from the shell by enclosing the path in double
quote) ?

os.popen3('"C:\\Program Files\\ds9\\ds9.exe"')

 

[Sean Blakey]

I don't have a windows to test, but does the following works (that is
protecting the space from the shell by enclosing the path in double
quote) ?

os.popen3('"C:\\Program Files\\ds9\\ds9.exe"')

--
Sean Blakey
Saint of Mild Amusement, Evil Genius, Big Geek
Python/Java/C++/C(Unix/Windows/Palm/Web) developer
email: (e-mail address removed)
quine = ['print "quine =",quine,"; exec(quine[0])"'] ; exec(quine[0])

 

[Lad]

I'm trying to launch an application from Python 2.3 on Windows. The
application is "ds9" (an image viewer), and is installed in C:\Program
Files\ds9\ds9

On unix I just do:
os.popen3("ds9")
and close the returned files and all is good. (I'm not trying to
communicate with the program via popen3 and so had been using
os.spawnlp, but that doesn't exist on Windows.)

On Windows os.popen3("ds9") does nothing, and the stderr returned from
os.popen3 has a message saying the program is unknown.

So I tried being specific:
os.popen3("C:\\Program Files\\ds9\\ds9")
This also fails and the program that is not found is "C:\Program",
suggesting that the space in "Program Files" is causing the problem. The
following failed in exactly the same way:
os.popen3("C:\\Program\ Files\\ds9\\ds9")

Any suggestions?

-- Russell

P.S. I installed the app using its Windows binary installer, which
unpacks a few files in C:\\Program files\. Windows doesn't seem to know
the program exists; it's not in the task bar, for example. But one can
double-click it to run it.

I think the problem is in long name folder: Program Files. So, try

Progra~1 instead of Program Files

 

[Patrick L. Nolan]

I think the problem is in long name folder: Program Files. So, try

Progra~1 instead of Program Files

An automatic way to accomplish this is

import win32api
shortpath = win32api.GetShortPathName(path)
os.popen3(shortpath)

Also, I have found by bitter experience that ds9 is unable to open any
file whose path includes a space. Put your data files somewhere else.

 

[Tony C]

I'm trying to launch an application from Python 2.3 on Windows. The
application is "ds9" (an image viewer), and is installed in C:\Program
Files\ds9\ds9

On Windows os.popen3("ds9") does nothing, and the stderr returned from

This won't work unless the installer for DS9 or someone manually put
the directory where DS9 is installed, into the path.


for long paths try using raw strings as in
os.popen3(r"C:\My Long Dir\My Long Program\MyProg.exe")

 

[Michele Petrazzo]

> So I tried being specific:
os.popen3("C:\\Program Files\\ds9\\ds9")
This also fails and the program that is not found is "C:\Program",
suggesting that the space in "Program Files" is causing the problem. The
following failed in exactly the same way:
os.popen3("C:\\Program\ Files\\ds9\\ds9")

Any suggestions?

The problem is that there are spaces into the path. Try to user
os.popen3('"C:\\Program\ Files\\ds9\\ds9"').
The better choice is:

path_complete = os.path.join('c:\\', 'Program Files', 'ds', 'ds9')
os.popen3('"'+ path_complete +'"' )

It work for me.

Michele

 

[Russell E. Owen]

An automatic way to accomplish this is

import win32api
shortpath = win32api.GetShortPathName(path)
os.popen3(shortpath)

Also, I have found by bitter experience that ds9 is unable to open any
file whose path includes a space. Put your data files somewhere else.

Thanks to everyone for their replies.

Lad and Patrick's suggestions both worked perfectly. (Simply trying to
protect the path with an extra set of quotes, as suggested by one kind
non-Windows-savvy soul, did not work)..

I also tested the bug mentioned by Patrick. I put a fits image file into
a directory whose name contained a space and tried to open that file in
ds9. it worked fine. I hope this means that the problem with spaces in
names has been thoroughly fixed.

Tip for ds9 and xpa users (xpa is a communication protocol used by ds9):
I learned from a ds9 developer that a proper installation of ds9 and
should have the xpa files in the same directory as ds9. This is NOT how
the binary ds9 and xpa installers do it, but it is easy to fix after the
installation. Unless this is done, ds9 cannot register itself with xpa,
making communication needlessly messy.


One more question:
- I noticed that when I use os.popen3 to open ds9, i have to keep the
returned reference to stdout around as long as ds9 itself is open or
Python hangs. I can't close it. I can't lose the reference and allow it
to be garbage-collected. Any suggestions for avoiding this? (At least it
doesn't prevent Python from shutting down when requested.)

I had been using os.spawn... to launch ds9, but spawn is not supported
on Windows, alas. I communicate with ds9 via xpa, so the pipes returned
by popen3 are of no interest.

-- Russell

 

[Russell E. Owen]

One more question:
- I noticed that when I use os.popen3 to open ds9, i have to keep the
returned reference to stdout around as long as ds9 itself is open or
Python hangs. I can't close it. I can't lose the reference and allow it
to be garbage-collected. Any suggestions for avoiding this? (At least it
doesn't prevent Python from shutting down when requested.)

I had been using os.spawn... to launch ds9, but spawn is not supported
on Windows, alas. I communicate with ds9 via xpa, so the pipes returned
by popen3 are of no interest.

Sorry, that was a stupid question. I had been using spawnlp and hadn't
noticed the little footnote in the docs about the spawn variants that
aren't supported under Windows. All I have to do is switch to a version
of spawn that IS cross-platform.

So. Case closed. Thanks again for everybody's help!

-- Russell

 

[Tony C]

The problem is that there are spaces into the path. Try to user
os.popen3('"C:\\Program\ Files\\ds9\\ds9"').
The better choice is:

path_complete = os.path.join('c:\\', 'Program Files', 'ds', 'ds9')
os.popen3('"'+ path_complete +'"' )

It work for me.

Michele

Try using raw strings, as I had posted in my reply.
Then you don't have to mess with joining strings, or using double
backslashes, or qoutes within quotes.

 

[Russell E. Owen]

Try using raw strings, as I had posted in my reply.
Then you don't have to mess with joining strings, or using double
backslashes, or qoutes within quotes.

I find this fails the same way as not using r"..." and doubling the
backslashes. Windows thinks the path to the executable stops at the
space. (I tried os.popen3(r"C:\Program Files\ds9\ds9") and the less
likely variant os.popen3(r"C:\Program\ Files\ds9\ds9"))

More info for anyone still following this thread:
The new subprocess module has promise here as a replacement for
os.popen3, since it allows a separate executable argument. Unfortunately
my tests indicate that executable and commands-as-strings (instead of
lists) aren't compatible; I've reported this as bug 1056441
<http://sourceforge.net/tracker/index.php?func=detail&aid=1056441&group_i
d=5470&atid=105470> for anyone who wants details.

I could recode to use commands-as-lists. This works, but is a big
headache for interactive use (one of the main targets for this module).

Also, for ds9 or xpa users:
- On Windows: ds9 will not connect to xpa unless the current directory
includes xpans or xpans is already running. Yecch. I ended up using
os.chdir before spawning ds9, then changing back.

- spaces in file names ARE bad news (as somebody warned me). ds9 can
open files with spaces in their names using the File menu, but not using
xpa. If the file name has a space you are totally out of luck. If the
path has a space (but not the file name) then you can try various tricks:
- quote the path in '"..."' (single quotes on the outside; no other form
of double quoting I tried worked). Fails on Windows.
- backslash-escape the spaces. Fails on Windows, of course.
- chdir to the file's directory first, then open, then chdir back. But
you can't be in two places at once. If you use chdir to get around the
os.popen3 problem of the executable having a space in its path name then
you can't also use it to get around the other problem.

Sigh.

-- Russell

 

[Anthony]

I find this fails the same way as not using r"..." and doubling the
backslashes. Windows thinks the path to the executable stops at the
space. (I tried os.popen3(r"C:\Program Files\ds9\ds9") and the less
likely variant os.popen3(r"C:\Program\ Files\ds9\ds9"))

Have you tried a wildcard? ie. something like:

os.popen3(r"C:\Program*\ds9\ds9"))

Bit of a hack though

Anthony

 

[Josiah Carlson]

- quote the path in '"..."' (single quotes on the outside; no other form
of double quoting I tried worked). Fails on Windows.

Which windows? It works for me on both 2k and XP (though I'm not
running ds9)<open file '"C:\Program Files\WinImage\winimage.exe"', mode 'w' at 0x007E7AE0>,
<open file '"C:\Program Files\WinImage\winimage.exe"', mode 'r' at 0x007EC2E0>,
<open file '"C:\Program Files\WinImage\winimage.exe"', mode 'r' at 0x007EC360>)


- Josiah

 

[Russell E. Owen]

Which windows? It works for me on both 2k and XP (though I'm not
running ds9):
(<open file '"C:\Program Files\WinImage\winimage.exe"', mode 'w' at
0x007E7AE0>,
<open file '"C:\Program Files\WinImage\winimage.exe"', mode 'r' at
0x007EC2E0>,
<open file '"C:\Program Files\WinImage\winimage.exe"', mode 'r' at
0x007EC360>)

You are right!

Putting double quotes around the path to the executable works so long as
there are no command line arguments. Unfortunately, as soon as you add
extra words, os.popen3 acts as if the double quotes are missing and can
no longer find the executable.

This works:
os.popen3('"C:\\Program Files\\ds9\\ds9"')

This fails on Windows -- cannot launch C:\Program -- but I tried
equivalent code on unix and it works fine (not that one normally needs
it on unix; executables are usually found automatically):
os.popen3('"C:\\Program Files\\ds9\\ds9" -title foo -port 0')

I see this same bug trying to use the subprocess module in Python 2.4b1
with my Windows Python 2.3 (see PR 105470:
<http://sourceforge.net/tracker/index.php?func=detail&aid=1057048&group_i
d=5470&atid=105470>).

My current solution is to temporarily change to the executable's
directory, for example:
currDir = os.getcwd()
try:
os.chdir(_XPADir)
fds = os.popen3("xpaget ds9 file myfile.fits")
finally:
os.chdir(currDir)

*****

Regarding my statement you quoted:
- quote the path in '"..."' (single quotes on the outside; no other form
of double quoting I tried worked). Fails on Windows.

That was in reference to using xpa to send file names to ds9. The file
name is sent as a command line argument, so it's a different issue than
discussed above. For those who care, it turns out that "{...}" is the
recommended way to send such paths to ds9. That's fine, but it also
turns out that ds9 on Windows requires the directory separator to be /,
not \!

For any ds9/xpa users out there, I posted a summary what I learned:
"Tricks for Using xpa to Command ds9":
<http://www.astro.washington.edu/rowen/ds9andxpa.html>. I'll keep this
updated if I learn anything new. I also hope to shorten it if some of
the ds9 bugs and quirks are fixed. I do not discuss the details of
os.popen3 and such, just xpa/ds9 issues.

Thank you all again for all your help. I hope to have a cross-platform
version of RO.DS9 out in the next week or so.

-- Russell