Script to capture stderr of subprocess

[jslowery]

We have a lot of curses-based console applications running on linux. I
would like to write a wrapper script that notifies us if the
application terminates unexpectedly. With my first, obviously naive
attempt, the subprocess dies instantly. STDIN and STDOUT will need to
connect to the terminal of course, since these are complex keyboard-
based applications. STDERR and the return code is what I would like to
capture. Actually, if it was possible, it would be nice to capture all
the bytes going between stdin and stdout in a file as well for
debugging purposes.

Could someone point me in the right direction here? I have a feeling
I"m missing something fundamental about how the Unix processes and
file descriptors work.

import os
import subprocess
import sys

cmd = ['/usr/local/bin/runcobol'] + sys.argv[1:]
proc = subprocess.Popen(cmd, stderr=subprocess.PIPE)
proc.communicate()

if proc.returncode:
f = file('/tmp/boom.txt', 'w')
f.write(" ".join(cmd) + " returned unexpectedly.\n")
f.write(proc.stderr.read(-1))
f.close()
sys.exit(proc.returncode)

 

[Arnaud Delobelle]

We have a lot of curses-based console applications running on linux. I
would like to write a wrapper script that notifies us if the
application terminates unexpectedly. With my first, obviously naive
attempt, the subprocess dies instantly. STDIN and STDOUT will need to
connect to the terminal of course, since these are complex keyboard-
based applications. STDERR and the return code is what I would like to
capture. Actually, if it was possible, it would be nice to capture all
the bytes going between stdin and stdout in a file as well for
debugging purposes.

Could someone point me in the right direction here? I have a feeling
I"m missing something fundamental about how the Unix processes and
file descriptors work.

import os
import subprocess
import sys

cmd = ['/usr/local/bin/runcobol'] + sys.argv[1:]
proc = subprocess.Popen(cmd, stderr=subprocess.PIPE)
proc.communicate()

If you want to capture what was sent to stderr, (which you probably do,
as you opened the process with stderr=subprocess.PIPE) you need to do
this:

stdoutdata, stderrdata = proc.communicate()

if proc.returncode:
f = file('/tmp/boom.txt', 'w')
f.write(" ".join(cmd) + " returned unexpectedly.\n")
f.write(proc.stderr.read(-1))

Now use this instead:

f.write(stderrdata)

f.close()
sys.exit(proc.returncode)

Now it should work. After you use proc.communicate(), proc.stderr has
been read and there is nothing left on it! Its content has been put in
the data returned by proc.communicate as shown above.

HTH

 

[Nobody]

Actually, if it was possible, it would be nice to capture all
the bytes going between stdin and stdout in a file as well for
debugging purposes.

If the child process expects to be running on a terminal, you would need
to use a pseudo-terminal (pty). See the "pty" module (although the
documentation assumes that you already understand how Unix ptys work).

Redirecting stdin/stdout to pipes tends to interfere with interactive
behaviour for a whole variety of reasons. Not least of which being that
many programs specifically use something like isatty(fileno(stdin)) to
determine whether they are being run interactively.

cmd = ['/usr/local/bin/runcobol'] + sys.argv[1:]
proc = subprocess.Popen(cmd, stderr=subprocess.PIPE)
proc.communicate()

if proc.returncode:
f = file('/tmp/boom.txt', 'w')
f.write(" ".join(cmd) + " returned unexpectedly.\n")
f.write(proc.stderr.read(-1))
f.close()
sys.exit(proc.returncode)

The .communicate() method waits for the process to terminate. Once that
has happened, you can't call .read() on proc.stderr.

The .communicate method returns a tuple, where the first item is whatever
was written to stdout (provided that stdout was redirected to a pipe), and
the second item is whatever was written to stderr (provided that stderr
was redirected to a pipe).

So you should use e.g.:

errors = proc.communicate()[1]