Threading and Windows.

[Jorge Godoy]

Hi!


I must have been searching in the wrong places or with the wrong
keywords but I couldn't find out how to implement something such as
what is done by the threading module on Windows (95, 98 and XP are the
ones used by my clients). My preference is for something available in
the standard library if possible.

I have a simple task where I want my GUI to be able to open several
applications simultaneously. Using the threading module everything
works perfectly on Linux but the program won't run on Windows.


Any hints on what should I do?

The working code is really simple and is all what I need on both
platforms (pseudo-code follows):


import threading
(...)

class MyApplication(...):
(...)
def someMethod(...):
(...)
def temporaryFunction():
os.spawnv('P_NOWAIT', 'another.py', ['another.py', parameters])
thread = threading.Thread(target = temporaryFunction)
thread.start()


def anotherMethod(...):
(...)
def temporaryFunction():
os.spawnv('P_NOWAIT', 'yetanother.py', ['yetanother.py', parameters])
thread = threading.Thread(target = temporaryFunction)
thread.start()


and so on.

(Yes, I could factor 'temporaryFunction' and reduce 5 lines of code in
my application, but I still don't know what will be necessary to do to
make it work on Windows, so I left it as is and factor it later.)


Any hints on how to accomplish that in a portable way? Or what
additional code will I have to use to make it work in Windows?


Thanks in advance,

 

[Dennis Lee Bieber]

Jorge Godoy fed this fish to the penguins on Monday 29 September 2003
05:00 am:

I have a simple task where I want my GUI to be able to open several
applications simultaneously. Using the threading module everything
works perfectly on Linux but the program won't run on Windows.

Forgive me if this has some obvious facet that I'm missing -- I'm not
experienced in os.spawnv* family.

However, the documentation I have for os.spawnv() suggests that this
call alone will create a separate/new process. And you are calling it
with a no-wait argument... So why do you even need the threading
overhead?

def temporaryFunction():
os.spawnv('P_NOWAIT', 'another.py', ['another.py',
parameters])

Hmmm, if "another.py" is the target, you may be encountering the
possibility that Windows doesn't know how to start a .py file from the
command line. Windows (especially the W9x line) doesn't, to my
knowledge, honor #! lines in script files.

--

 

[Peter Abel]

Hi!


I must have been searching in the wrong places or with the wrong
keywords but I couldn't find out how to implement something such as
what is done by the threading module on Windows (95, 98 and XP are the
ones used by my clients). My preference is for something available in
the standard library if possible.

I have a simple task where I want my GUI to be able to open several
applications simultaneously. Using the threading module everything
works perfectly on Linux but the program won't run on Windows.


Any hints on what should I do?

The working code is really simple and is all what I need on both
platforms (pseudo-code follows):


import threading
(...)

class MyApplication(...):
(...)
def someMethod(...):
(...)
def temporaryFunction():
os.spawnv('P_NOWAIT', 'another.py', ['another.py', parameters])
thread = threading.Thread(target = temporaryFunction)
thread.start()


def anotherMethod(...):
(...)
def temporaryFunction():
os.spawnv('P_NOWAIT', 'yetanother.py', ['yetanother.py', parameters])
thread = threading.Thread(target = temporaryFunction)
thread.start()


and so on.

(Yes, I could factor 'temporaryFunction' and reduce 5 lines of code in
my application, but I still don't know what will be necessary to do to
make it work on Windows, so I left it as is and factor it later.)


Any hints on how to accomplish that in a portable way? Or what
additional code will I have to use to make it work in Windows?


Thanks in advance,

http://www.python.org/doc/current/lib/os-process.html says
....
....
....
spawnl(mode, path, ...)
spawnle(mode, path, ..., env)
spawnlp(mode, file, ...)
spawnlpe(mode, file, ..., env)
spawnv(mode, path, args)
spawnve(mode, path, args, env)
spawnvp(mode, file, args)
spawnvpe(mode, file, args, env)
....
....
....
Availability: Unix, Windows. spawnlp(), spawnlpe(), spawnvp() and
spawnvpe() are not available on Windows. New in version 1.6.
....
....
....
Regards
Peter

 

[Jorge Godoy]

http://www.python.org/doc/current/lib/os-process.html says
...
...
...
spawnl(mode, path, ...)
spawnle(mode, path, ..., env)
spawnlp(mode, file, ...)
spawnlpe(mode, file, ..., env)
spawnv(mode, path, args)
spawnve(mode, path, args, env)
spawnvp(mode, file, args)
spawnvpe(mode, file, args, env)
...
...
...
Availability: Unix, Windows. spawnlp(), spawnlpe(), spawnvp() and
spawnvpe() are not available on Windows. New in version 1.6.

So I should be safe since I'm using spawnv (and not spawnvp or
spawnvpe).


Thanks anyway.


See you,

 

[Jorge Godoy]

Forgive me if this has some obvious facet that I'm missing -- I'm not
experienced in os.spawnv* family.

Neither am I. In fact, this program was my first use of it.

However, the documentation I have for os.spawnv() suggests
that this call alone will create a separate/new process. And you are
calling it with a no-wait argument... So why do you even need the
threading overhead?

This isn't the behaviour I found here on a Linux box. The caller
process got 'stuck' and only worked again when the called proccess
ended.

With threads I got it working all the time.

Hmmm, if "another.py" is the target, you may be encountering the
possibility that Windows doesn't know how to start a .py file from the
command line. Windows (especially the W9x line) doesn't, to my
knowledge, honor #! lines in script files.

Using the thread module I could get it to work on Windows and then,
yes. I got this problem.

Sorry for my lameness on Windows but this isn't my platform of
choice. I'll add some conditional and check the platform...

On the other hand, isn't there a way to associate programs with
extensions on Windows? Wouldn't this solve the problem with the call I
made? Or it only works in certain circunstances and (by Murphy's law)
this is not one of those circunstances?


Thanks for your help,

 

[Jordan Krushen]

So I should be safe since I'm using spawnv (and not spawnvp or
spawnvpe).

For running a file based on its extension, you could try os.startfile().

J.

 

[Dennis Lee Bieber]

Jorge Godoy fed this fish to the penguins on Monday 29 September 2003
07:31 pm:

This isn't the behaviour I found here on a Linux box. The caller
process got 'stuck' and only worked again when the called proccess
ended.

Strange...

On the other hand, isn't there a way to associate programs with
extensions on Windows? Wouldn't this solve the problem with the call I
made? Or it only works in certain circunstances and (by Murphy's law)
this is not one of those circunstances?

I think NT has the ability to go by extension, but not W9x.

You might have to explicitly invoke the python executable, and pass
/it/ the .py file as the first argument

--

 

[Jorge Godoy]

For running a file based on its extension, you could try os.startfile().

Thank you.

It will probably solve my problem on Windows.

Unfortunately, it isn't a portable way since this command doesn't even
exist on my Linux (Python 2.2.2). I will have to use the conditional I
said on my previous message.


Thanks again,

 

[Jorge Godoy]

Jorge Godoy fed this fish to the penguins on Monday 29 September 2003
07:31 pm:

Strange...

I thought the same thing and even thought I was using os.spawnv() in a
wrong way. Even with the 'P_NOWAIT' flag it still hangs.

I'm using:

Python 2.2.2 (#1, Mar 6 2003, 13:36:19)
[GCC 3.2.2] on linux-i386


I haven't investigated os.spawnv() alone in Windows to see if this
also happens there. It will be funny if by this very first time the
problem is on the Linux box instead of the Windows box (since I'm more
used to unices, I'm more familiar with their quirks than with
Windows', specially with all those 'flavours' of Windows...).

I think NT has the ability to go by extension, but not W9x.

Windows XP can't go by extension either.

You might have to explicitly invoke the python executable, and pass
/it/ the .py file as the first argument

I see. If it doesn't work using os.startfile() as suggested on another
message I'll probably stick with this approach.


Thanks!

 

[Miki Tebeka]

Hello Jorge,

I have a simple task where I want my GUI to be able to open several
applications simultaneously. Using the threading module everything
works perfectly on Linux but the program won't run on Windows.

What do you mean by "won't run"? Does it crash? ...

Any hints on what should I do?
The working code is really simple and is all what I need on both
platforms (pseudo-code follows):
[...]

Just guessing:
1. The spawnl of a .py don't work. You need to start the python interpreter
on the .py file
2. The GUI toolkit you're using is causing the problem. To check this try
lauch threads without any gui and see if it works.

HTH.
Miki

 

[Jorge Godoy]

Hello Jorge,

Hi Miki.

What do you mean by "won't run"? Does it crash? ...

No. It just hangs. I have to kill it. If it crashed I could try
investigating the reason but the reason of the hangup is a little
harder to find out.

Any hints on what should I do?
The working code is really simple and is all what I need on both
platforms (pseudo-code follows):
[...]

Just guessing:
1. The spawnl of a .py don't work. You need to start the python interpreter
on the .py file

This is what I found out using the 'thread' module. I'll change the
code so that it checks the OS and take the action based on that
answer, but it really annoyed me using threads there.

BTW, the 'threading' module didn't work whie the 'thread' did. I dunno
the inner details of implementation, but it might give a hint on the
problem... (Just guessing)

2. The GUI toolkit you're using is causing the problem. To check this try
lauch threads without any gui and see if it works.

I't worked with one example I found on the net while trying to find an
answer, but it hang with another. The one that hung was just making 4
threads and measuring the time each of them 'sleep()'ed and summing it
all on the main thread. It worked perfectly here (Linux), but not
there (Windows XP).

HTH.
Miki

Thanks Miki.

 

[Jorge Godoy]

Jorge Godoy fed this fish to the penguins on Monday 29 September 2003
07:31 pm:

Strange...

OK... I tracked it down and found the 'P_NOWAIT' didn't work. I
replaced it with '1' (no quotes) and everything went fine on bot OSs.

The error message on Windows helped finding it out (it said that an
integer was expected but I got nothing on Linux).

You might have to explicitly invoke the python executable, and pass
/it/ the .py file as the first argument

It ended up like this:

os.spawnv(1, 'c:/python22/pythonw.exe', ['c:/python22/pythonw.exe', 'another.py', parameter])


This avoided the need for threads on this program, thanks.


Just to make the solution complete, only the 'thread' module worked on
Windows. I couldn't get the 'threading' module to work there.



Thanks for all the help.



See you,

 

[Dennis Lee Bieber]

Jorge Godoy fed this fish to the penguins on Tuesday 30 September 2003
04:08 am:

I thought the same thing and even thought I was using os.spawnv() in a
wrong way. Even with the 'P_NOWAIT' flag it still hangs.

I may have found the problem (surprised the experts didn't catch it).

From your initial post:
os.spawnv('P_NOWAIT', 'another.py', ['another.py', parameters])

You specified P_NOWAIT as a STRING. It should be a constant /in/ the os
module.

The following seem to work on Mandrake 8.2... (Danger, I don't know if
Knode loses tabs on send, or just on read -- the loop structure should
be simple though).


Python 2.2 (#1, Nov 5 2002, 15:43:24)
[GCC 2.96 20000731 (Mandrake Linux 8.2 2.96-0.76mdk)] on linux-i386

t.py ================
#! /usr/bin/python

import os
import time

plist = []

for i in range(3):
print "starting", i
plist.append(os.spawnv(os.P_NOWAIT, "./t1.py", ("t1.py", str(i))))

for i in range(20):
print time.asctime()
time.sleep(2.000)

for p in plist:
print "Waiting @", time.asctime()
endit = os.wait()
print "\t", endit, time.asctime()


===============

Note the use of os.P_NOWAIT rather than your 'P_NOWAIT'


t1.py =========
#! /usr/bin/python

import time
import sys


fid = "Scratch%s.txt" % sys.argv[1]

fout = file(fid, "w")

for i in range(50):
ln = "Iteration: %s\tTime: %s\n" % (i, time.asctime())
fout.write(ln)
time.sleep(1.000)

fout.close()

==============


t1.py was CHMOD u+x. Output from t.py is:


[wulfraed@beastie wulfraed]$ python t.py
starting 0
starting 1
starting 2
Tue Sep 30 08:11:50 2003
Tue Sep 30 08:11:52 2003
Tue Sep 30 08:11:54 2003
Tue Sep 30 08:11:56 2003
Tue Sep 30 08:11:58 2003
Tue Sep 30 08:12:00 2003
Tue Sep 30 08:12:02 2003
Tue Sep 30 08:12:04 2003
Tue Sep 30 08:12:06 2003
Tue Sep 30 08:12:08 2003
Tue Sep 30 08:12:10 2003
Tue Sep 30 08:12:12 2003
Tue Sep 30 08:12:14 2003
Tue Sep 30 08:12:16 2003
Tue Sep 30 08:12:18 2003
Tue Sep 30 08:12:20 2003
Tue Sep 30 08:12:22 2003
Tue Sep 30 08:12:24 2003
Tue Sep 30 08:12:26 2003
Tue Sep 30 08:12:28 2003
Waiting @ Tue Sep 30 08:12:30 2003
(4139, 0) Tue Sep 30 08:12:40 2003
Waiting @ Tue Sep 30 08:12:40 2003
(4138, 0) Tue Sep 30 08:12:40 2003
Waiting @ Tue Sep 30 08:12:40 2003
(4140, 0) Tue Sep 30 08:12:40 2003

and Scratch0.txt, Scratch1.txt, Scratch2.txt have overlapping output --
in truth, since time.asctime() only records whole seconds, the three
files are identical.

I see. If it doesn't work using os.startfile() as suggested on another
message I'll probably stick with this approach.

If you want portability, this may be the way to go -- I just changed
the spawnv() call above from "./t1.py" to "python t1.py" and it still
ran on Linux. I'd presume it would also run on Windows (I'm not going
to reboot just to prove that)


NO THREADS NEEDED.

--

 

[Jorge Godoy]

Jorge Godoy fed this fish to the penguins on Tuesday 30 September 2003
04:08 am:

I thought the same thing and even thought I was using os.spawnv() in a
wrong way. Even with the 'P_NOWAIT' flag it still hangs.

I may have found the problem (surprised the experts didn't catch it).

From your initial post:
os.spawnv('P_NOWAIT', 'another.py', ['another.py', parameters])

You specified P_NOWAIT as a STRING. It should be a constant /in/ the os
module.

The most interesting thing is that I get an error in Windows but I
don't get such an error in Linux.

This is the output on an interactive session running on Linux:

import os
os.spawnv(os.P_NOWAIT, 'ls', ['ls', '*.py']) 790
os.spawnv('os.P_NOWAIT', 'ls', ['ls', '*.py']) 127
os.spawnv('P_NOWAIT', 'ls', ['ls', '*.py']) 127
os.spawnv(os.P_WAIT, 'ls', ['ls', '*.py']) 127
os.spawnv(os.P_NOWAIT, 'ls', ['ls', '*.py']) 795

Now, on the Windows box:

import os
os.spawnv(os.P_NOWAIT, 'c:/python22/python.exe', ['c:/python22/python.exe']) 112
os.spawnv('P_NOWAIT', 'c:/python22/python.exe', ['c:/python22/python.exe'])

Traceback (most recent call last):
File "<pyshell#6>", line 1, in ?
os.spawnv('P_NOWAIT', 'c:/python22/python.exe', ['c:/python22/python.exe'])
TypeError: an integer is required

And this is how I found out where was the problem.

Note the use of os.P_NOWAIT rather than your 'P_NOWAIT'

It indeed work. I'll replace the '1's with it on the code.

If you want portability, this may be the way to go -- I just changed
the spawnv() call above from "./t1.py" to "python t1.py" and it still
ran on Linux. I'd presume it would also run on Windows (I'm not going
to reboot just to prove that)

Just if it is in the PATH, I suppose. I don't believe Windows scans
through all directories if it can't find the executable.

NO THREADS NEEDED.

Solving the problem with os.P_NOWAIT is better than using threads. ;-)

The problem was my misinterpretation of the docs (or maybe they don't
make it explicit that I must use the 'os.' prefix...):

----------------------------------------------------------------------
Help on function spawnv:

spawnv(mode, file, args)
spawnv(mode, file, args) -> integer

Execute file with arguments from args in a subprocess.
If mode == P_NOWAIT return the pid of the process.
If mode == P_WAIT return the process's exit code if it exits normally;
otherwise return -SIG, where SIG is the signal that killed it.

 

[Dennis Lee Bieber]

Jorge Godoy fed this fish to the penguins on Tuesday 30 September 2003
09:01 am:

The most interesting thing is that I get an error in Windows but I
don't get such an error in Linux.

spawnv() is probably a direct match with a unix/linux system call, so
the arguments are just being passed on -- and what linux does when it
gets the address of a string where it expected an integer constant is
unknown to me (probably sees the address /as/ the integer, and since
P_WAIT is 0, the runtime is treating any non-zero to be the equivalent
of P_NOWAIT)

On Windows, spawnv() probably has to be translated into the W32 system
call, CreateProcess(), which means parsing the arguments so they can be
translated into the proper values for Windows -- and probably has to
parse the mode, so it can explicitly do the WAIT as a separate
operation.

IE, Windows has something like:

pid = CreateProcess(....)
if mode == P_WAIT
Wait(pid)

and the error message is trapped at the Python/C interface.

--

 

[Brian Kelley]

If you are using wxPython you might want to check out this little package:

http://staffa.wi.mit.edu/people/kelley/

look for LongRunningTasks at the bottom.

-brian

 

[Jorge Godoy]

Jorge Godoy fed this fish to the penguins on Tuesday 30 September 2003
09:01 am:

spawnv() is probably a direct match with a unix/linux system call, so
the arguments are just being passed on -- and what linux does when it
gets the address of a string where it expected an integer constant is
unknown to me (probably sees the address /as/ the integer, and since
P_WAIT is 0, the runtime is treating any non-zero to be the equivalent
of P_NOWAIT)

The behaviour of treating it as nonzero would be expected. What
happens is the opposite of that as I described before: the program
hangs as if it received P_WAIT.


I think that they check explicitly for P_NOWAIT and treat anything
different as zero (P_WAIT).


The principle of least surprise was violated, IMVHO as a newbie on
that area.


See you,

 

[Jorge Godoy]

If you are using wxPython you might want to check out this little package:

http://staffa.wi.mit.edu/people/kelley/

look for LongRunningTasks at the bottom.

-brian

I didn't knew this address but I got something like that from
http://wiki.wxpython.org/index.cgi/LongRunningTasks

It didn't solve my problem but thanks for this new pointer.

In my case I really needed something that ran a new proccess.

 

[Dennis Lee Bieber]

Dennis Lee Bieber fed this fish to the penguins on Tuesday 30 September
2003 10:17 am:

spawnv() is probably a direct match with a unix/linux system

Okay, I took some time at work to look at the source for os.py... My
assumptions are totally backwards...

From my email to myself:

Had a reversal... spawnv() does not exit on Linux, which uses
combination of (see os.py for actual code):

pid = fork()
if not pid: #new process
execv() #replace with desired program
else:
if mode == P_NOWAIT:
return pid
while 1: # why the loop?
status = waitpid(pid, 0)
#tests for status return


Since 'P_NOWAIT' is not equal to os.P_NOWAIT, Linux defaults to
performing the wait operation.

It is on Windows where a low-level _spawnv() exists,
int _spawnv( int mode, const char *cmdname, const char *const *argv );
and the error is raised when Python tries to convert the string
'P_NOWAIT' into the integer mode for the direct call.

--

 

[Bengt Richter]

OK... I tracked it down and found the 'P_NOWAIT' didn't work. I
replaced it with '1' (no quotes) and everything went fine on bot OSs.

What is os.P_NOWAIT on your system? It looks like '1' (no quotes) on my system:
1

The error message on Windows helped finding it out (it said that an
integer was expected but I got nothing on Linux).

You might have to explicitly invoke the python executable, and pass
/it/ the .py file as the first argument

It ended up like this:

os.spawnv(1, 'c:/python22/pythonw.exe', ['c:/python22/pythonw.exe', 'another.py', parameter])

So one would think

os.spawnv(os.P_NOWAIT, 'c:/python22/pythonw.exe',
['c:/python22/pythonw.exe', 'another.py', parameter])

would work, except I note that os.P_NOWAIT is not defined in Python 1.5.2 on Linux (slackware).
Gotta upgrade one of these days... My nt python is 2.3 though, but I don't suppose that's changed
since the 2.2 you are apparently running on windows??

Regards,
Bengt Richter