Suppress output to stdout/stderr in InteractiveInterpreter

[Dave W.]

I've subclassed InteractiveInterpreter in an attempt to make it
friendlier for use in a remote console for a C++ app. What I really
wanted to do was wrap/adapt the base class's runsource() method to
return a 2-tuple (more, result) where 'more' is a bool indicating
whether a complete command was recognized, and 'result' is a string
representing the results of evaluating the last chunk (which may be
empty, or may contain an error message).

The only fool-proof way I found to do this was to redirect stdout/
stderr
during calls to the base class's runsource() method. If I only
provide
custom sys.displayhook and sys.excepthook replacements, then any print
statements the user enters go to the host app's stdout:

instead of being captured and echoed back to the remote console.
If I omit the print, it does what I expect:

But I'd really like to:

1) Get rid of this slight difference to the normal python REPL
2) Not have to redirect stdout/stderr (I'm afraid other [C++]
threads may compete with python for these streams.)

I thought I could get away with import print_function from __future__
(see example code below), but my re-pointed print function never gets
called.

I think I'm just naive about how exec() works. Is it hard-wired to
stdout or something? *Should* this even work?

Any insight much appreciated!

- Dave Wolfe

----------

from __future__ import print_function
....
class HookContext(object):

def __init__(self, printhook, displayhook, excepthook):
self.printhook = printhook
self.displayhook = displayhook
self.excepthook = excepthook

def __enter__(self):
print = self.printhook
sys.displayhook = self.displayhook
sys.excepthook = self.excepthook

def __exit__(self, exc_type, exc_value, traceback):
print = __builtins__.print
sys.displayhook = sys.__displayhook__
sys.excepthook = sys.__excepthook__


class Interpreter(InteractiveInterpreter):

[ ... lots of stuff omitted ...]

def do_runsource(self, source):
self.output.reset()
self.output.truncate()

with HookContext(self.printhook
self.displayhook, self.excepthook):
try:
more = InteractiveInterpreter.runsource(self, source)
result = self.output.getvalue()
except (EOFError, OverflowError, SyntaxError):
pass

return more, result

 

[Jerry Hill]

I thought I could get away with import print_function from __future__
(see example code below), but my re-pointed print function never gets
called.
-snip-

   def __enter__(self):
       print = self.printhook

That redefines the print function local to __enter__. You need to
change the global value of print.

from __future__ import print_function
from contextlib import contextmanager

def printhook(obj, *args, **kwargs):
__builtins__.print('['+obj+']', *args, **kwargs)

@contextmanager
def local_printhook():
print = printhook
yield
print = __builtins__.print

@contextmanager
def global_printhook():
global print
print = printhook
yield
print = __builtins__.print


with local_printhook():
print("Test 1")

with global_printhook():
print("Test 2")
Test 1
[Test 2]

 

[Dave W.]

I thought I could get away with importing print_function

That redefines the print function local to __enter__. You need to
change the global value of print.

Thanks for the clue. I had actually tried using 'global print' in
the __enter__ function, but it broke my tests even worse so I
figured I must not need it. :-} After your reply, I went back and
discovered that everything was actually working fine with 'global'
in place---except my automated tests break. So I just have a little
mystery to solve regarding how execution using nose differs from
running the app manually.

from contextlib import contextmanager

Hey, thanks for that gem! I didn't know about this clever little
module---really useful!

 

[Dave W.]

After your reply, I went back and

discovered that everything was actually working fine with 'global'
in place <snip>

Oops, I lied. It still doesn't work. It doesn't seem possible (or
at least not easy) to *globally* override the built-in print
function. The best I've managed to do is override it for the
current module.

Think I'll start a new post with the subject: "Globally override
built-in print function?"

 

[Robert Kern]

Oops, I lied. It still doesn't work. It doesn't seem possible (or
at least not easy) to *globally* override the built-in print
function. The best I've managed to do is override it for the
current module.

Think I'll start a new post with the subject: "Globally override
built-in print function?"

Don't bother. Replacing sys.stdout is the right thing to do. It won't interfere
with the C++ streams but it will take care of some potential Python output that
does not go through the print() function.

--
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless enigma
that is made terrible by our own mad attempt to interpret it as though it had
an underlying truth."
-- Umberto Eco

 

[Dave W.]

Think I'll start a new post with the subject: "Globally override

Don't bother. Replacing sys.stdout is the right thing to do. It
won't interfere with the C++ streams...

-snip-

I'm not so certain. Won't the C++ host app share the same
stdin/stdout/stderr file descriptors with the embedded Python
interpreter? So there's at least the *potential* for a background
C++ thread dedicated to processing Python commands (which redirect
stdout) to interfere with, say, the main thread's log statements to
stdout. Seems like I could wind up with log statements in my
interpreter results.

As it happens, the background thread for processing incoming
commands from the remote REPL currently just queues those commands
to be executed in the main thread, so there's no contention. But
this makes some 'interesting' things possible using the remote
console:

c:\pt\celoverus\scripts>remote_console.py 192.168.1.10
Python 2.6.2 [IG REPL]
Because this is executed in the main app thread's context, it hangs
the host app forever. Kind of a perfect DoS attack. Not that
security is a big deal for this particular app. I'm more worried
about not being able to perform debugging tasks like this:
.... for obj in engine.get_objects():
.... if obj.collides_with(engine.get_object(OWNSHIP)
.... print("'{0}' hit the ownship".format(obj.name))
.... time.sleep(5.0)

Currently, this would hang the app forever, too. Moving the
execution of remote Python commands into a low-priority background
thread would allow the host app to keep running in this case.

Anyway, that's why I've got this mild obsession with finding a way
to capture output from the interpreter *without* redirecting
stdout...

 

[Robert Kern]

-snip-

I'm not so certain. Won't the C++ host app share the same
stdin/stdout/stderr file descriptors with the embedded Python
interpreter? So there's at least the *potential* for a background
C++ thread dedicated to processing Python commands (which redirect
stdout) to interfere with, say, the main thread's log statements to
stdout. Seems like I could wind up with log statements in my
interpreter results.

No. Changing the object that the name sys.stdout refers to does not swap out the
underlying file descriptors.

Anyway, that's why I've got this mild obsession with finding a way
to capture output from the interpreter *without* redirecting
stdout...

Not possible. Many things write directly to sys.stdout/sys.stderr.

--
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless enigma
that is made terrible by our own mad attempt to interpret it as though it had
an underlying truth."
-- Umberto Eco

 

[Dave W.]

Don't bother. Replacing sys.stdout is the right thing to do. It

No. Changing the object that the name sys.stdout refers to does
not swap out the underlying file descriptors.

Whoa--that's a bombshell. I guess I shouldn't have assumed that
this was the case, but it seemed so 'obvious'. Glad I was wrong,
since it makes like much, much easier. Thank you!

[xref "Globally override built-in print function?"]