Trapping the segfault of a subprocess.Popen

[Pierre GM]

All,

I need to run a third-party binary from a python script and retrieve
its output (and its error messages). I use something likeThat works fine, except that the third-party binary in question
doesn't behave very nicely and tend to segfaults without returning any
error. In that case, `process.communicate` hangs for ever.

I thought about calling a `threading.Timer` that would call
`process.terminate` if `process.wait` doesn't return after a given
time... But it's not really a solution: the process in question can
sometimes take a long time to run, and I wouldn't want to kill a
process still running.
I also thought about polling every x s and stopping when the result of
a subprocess.Popen(["ps","-p",str(initialprocess.pid)],
stdout=subprocess.PIPE) becomes only the header line, but my script
needs to run on Windows as well (and no ps over there)...

Any suggestion welcome,
Thx in advance
P.

 

[Nobody]

I need to run a third-party binary from a python script and retrieve
its output (and its error messages). I use something like
That works fine, except that the third-party binary in question doesn't
behave very nicely and tend to segfaults without returning any error. In
that case, `process.communicate` hangs for ever.

Odd. The .communicate method should return once both stdout and stderr
report EOF and the process is no longer running. Whether it terminates
normally or on a signal makes no difference.

The only thing I can think of which would cause the situation which you
describe is if the child process spawns a child of its own before
terminating. In that case, stdout/stderr won't report EOF until any
processes which inherited them have terminated.

I thought about calling a `threading.Timer` that would call
`process.terminate` if `process.wait` doesn't return after a given
time... But it's not really a solution: the process in question can
sometimes take a long time to run, and I wouldn't want to kill a
process still running.
I also thought about polling every x s and stopping when the result of
a subprocess.Popen(["ps","-p",str(initialprocess.pid)],
stdout=subprocess.PIPE) becomes only the header line, but my script
needs to run on Windows as well (and no ps over there)...

It should suffice to call .poll() on the process. In case that doesn't
work, the usual alternative would be to send signal 0 to the process (this
will fail with ESRCH if the process doesn't exist and do nothing
otherwise), e.g.:

import os
import errno

def exists(process):
try:
os.kill(process.pid, 0)
except OSError, e:
if e.errno == errno.ESRCH:
return False
raise
return True

You might need to take a different approach for Windows, but the above is
preferable to trying to parse "ps" output. Note that this only tells you
if /some/ process exists with the given PID, not whether the original
process exists; that information can only be obtained from the Popen
object.

 

[Terry Reedy]

I am not sure this will help you now, but....
Victor Stinner has added a new module to Python 3.3 that tries to catch
segfaults and other fatal signals and produce a traceback before Python
disappears.

 

[Pierre GM]

Odd. The .communicate method should return once both stdout and stderr
report EOF and the process is no longer running. Whether it terminates
normally or on a signal makes no difference.

The only thing I can think of which would cause the situation which you
describe is if the child process spawns a child of its own before
terminating. In that case, stdout/stderr won't report EOF until any
processes which inherited them have terminated.

I think you nailed it. Running the incriminating command line in a
terminal doesn't return to the prompt. In fact, a ps shows that the
process is sleeping in the foreground. Guess I should change the
subject of this thread...

I thought about calling a `threading.Timer` that would call
`process.terminate` if `process.wait` doesn't return after a given
time... But it's not really a solution: the process in question can
sometimes take a long time to run, and I wouldn't want to kill a
process still running.
I also thought about polling every x s and stopping when the result of
a subprocess.Popen(["ps","-p",str(initialprocess.pid)],
stdout=subprocess.PIPE) becomes only the header line, but my script
needs to run on Windows as well (and no ps over there)...

It should suffice to call .poll() on the process. In case that doesn't
work, the usual alternative would be to send signal 0 to the process (this
will fail with ESRCH if the process doesn't exist and do nothing
otherwise), e.g.:

import os
import errno

def exists(process):
    try:
        os.kill(process.pid, 0)
    except OSError, e:
        if e.errno == errno.ESRCH:
            return False
        raise
    return True

OK, gonna try that, thx.

 

[Pierre GM]

I am not sure this will help you now, but....
Victor Stinner has added a new module to Python 3.3 that tries to catch
segfaults and other fatal signals and produce a traceback before Python
disappears.

Unfortunately, I'm limited to Python 2.5.x for this project. But good
to know, thanks...