print in multithreaded environement ???

[vincent delft]

What will be the result if I have several threads that print to
different buffers ?

I have several threads, each one execute different python code.
Just before the "exec" of this small python code, I redirect the
output to a "buffer".
Each thread has his own buffer.
What will be the result when they print at the same time ?

Does sys.stdout is a seperate instance for each thread ?
Does sys.stdout is shared between all of them ?

Does any one has experience with that ?

Thanks.

 

[Josiah Carlson]

Does sys.stdout is a seperate instance for each thread ?

Does sys.stdout is shared between all of them ?

sys.stdout, sys.stdin and sys.stderr are all shared.

Does any one has experience with that ?

Yes. As soon as a thread uses up its timeslice, it is paused. Sending
a lot of information to sys.stdout, or doing any non-trivial computation
is a good way of using up a timeslice...as is sleeping for an
arbitrarily small amount of time, even 0.

Run the following code to see how it behaves.

import time
from threading import Thread
def funct(i):
for j in xrange(50):
print i,
time.sleep(0)

for i in xrange(50): Thread(target=funct, args=(i,)).start()


- Josiah

 

[vincent_delft]

Thanks Josiah for the answer.

but in my case each thread will have his own buffer and stdout will be
redirect to it.

Following your example, I think that what I'm talking about should be like
this :
import time
from threading import Thread
import sys
def funct(i):
fid=open('/tmp/tt%s.tt' % i,'w')
sys.stdout=fid
for j in xrange(50):
print i,j,
time.sleep(0)
fid.close()
for i in xrange(50): Thread(target=funct, args=(i,)).start()

If I run it, I've the result I expected: counter from 0 to 49 in each file.
Thus sys.stdout is not a shared object between each thread. It's his
redirection to the console which is common.
Thus, problems occurs when we link it to a shared output (like a console or
single output file).

Am I correct ?

 

[Bob Ippolito]

Thanks Josiah for the answer.
but in my case each thread will have his own buffer and stdout will be
redirect to it.

Following your example, I think that what I'm talking about should be like
this :
import time
from threading import Thread
import sys
def funct(i):
fid=open('/tmp/tt%s.tt' % i,'w')
sys.stdout=fid
for j in xrange(50):
print i,j,
time.sleep(0)
fid.close()
for i in xrange(50): Thread(target=funct, args=(i,)).start()

If I run it, I've the result I expected: counter from 0 to 49 in each file.
Thus sys.stdout is not a shared object between each thread. It's his
redirection to the console which is common. Thus, problems occurs when
we link it to a shared output (like a console or
single output file).

Am I correct ?

What you actually want to try and do is:

print >>fid, i, j,

Using a global output for multiple threads at the same time is an
absolutely awful idea..

dont_try_this_at_home = """
it is possible to make sys.stdout a special kind of object that writes
to different streams depending on the thread id, or at least I would
assume so because I have done it for stackless tasklets. However, the
*only* reason I did it that way because what I was writing was an
multi-interpreter GUI environment, so the print statements and
sys.stdout were something I did not have any control over.

In either case, I learned a lesson trying to do this:

Python (short of PyPy) is not capable of doing in-process
"sub-interpreters" for arbitrary code in any reliable way.

If I were to hack at it again, I would rip that out and emulate IDLE.
It spawns a new process for each interpreter and communicates with it
over some kind of file descriptor (socket, pipe, etc.). This is more
scalable anyway, because Python has the dreaded GIL.
"""

My general rule is to avoid threads whenever possible, because in many
cases they aren't going to be faster anyway (unless their primary
function is to call a C function that releases the GIL, like maybe
communicating with a blocking database library) and that it is hard to
debug threaded programs that aren't working quite right. If you don't
like writing asynchronous code explicitly, look into Stackless Python

-bob

 

[Aahz]

What will be the result if I have several threads that print to
different buffers ?

What do you mean by "buffer"? Different file handles?

I have several threads, each one execute different python code.
Just before the "exec" of this small python code, I redirect the
output to a "buffer".
Each thread has his own buffer.
What will be the result when they print at the same time ?

If the buffer is a simple string that you're printing, the order in
which buffers are output is non-determinate (absent explicit ordering
through the use of Queue.Queue or equivalent), but the entire string
will be output as a single chunk.