Python Threads + MPI

[Barry Rountree]

Hello,

I'd like to use a Python MPI wrapper (pypar, pyMPI, or ScientificPython) but
I can't get threading to work. I'd like individual threads responsible for
MPI_Send and MPI_Recv. Things tend to either work perfectly or hang,
depending on ordering and sleep times.

My best guess is that the signal waking up the C-level MPI_Recv gets snagged
by another thread, which promptly ignores it. (This theory is based
entirely on a single sentence culled from the thread module documentation:
"Threads interact strangely with interrupts".

I've also tried surrounding the C call to MPI_Recv with
Py_BEGIN_ALLOW_THREADS and Py_END_ALLOW_THREADS, to no avail.

I guess this boils down to: can I put C blocking socket calls in their own
Python threads and expect them to work? If so, how?

Thanks much,

Barry Rountree

Sample code:

#!/opt/install/bin/mpipython

import Scientific.MPI as MPI
import time
import threading as T

class Sender(T.Thread):
def run(self):
count = 0
while True:
time.sleep(0.2)
print "Starting send of", count
MPI.world.send( "x"+str(count)+"x", 0, count )
print "Finished send of", count
count+=1
break;
class Recver(T.Thread):
def run(self):
count = 0
while True:
time.sleep(1.0)
print "Starting recv"
x = MPI.world.receiveString()
print "Received", x
count += 1

r = Recver()
s = Sender()


r.start()
s.start()