python shell silently ignores termios.tcsetattr()

[kj]

This post is a continuation of an earlier thread called

annoying CL echo in interactive python / ipython


I found some more clues to the problem, although no solution yet.
First, I found a post from 2009.05.09 that describes exactly the
same situation I've observed (although it got no responses):

http://groups.google.com/group/comp.emacs/browse_thread/thread/72973892717d0bfa

I tried to fix the problem by applying the equivalent of "stty
-echo" within a python interactive session, but discovered that
this setting is immediately (and silently) overwritten. The
following interaction illustrates what I mean. (I've added my
comments, preceded by ###.)

First I start the barest possible instance of Emacs (only the bare
mininum of environment settings, and loading no configurations from
an ~/.emacs file):

% env -i HOME=$HOME DISPLAY=$DISPLAY TERM=$TERM /opt/local/bin/emacs -Q

Within Emacs, the first command I execute is M-x shell, to bring
up an Emacs shell. What follows is the interaction within this shell:

sh-3.2$ stty -a ### first I get the initial settings for the terminal
speed 9600 baud; 0 rows; 0 columns;
lflags: icanon isig iexten -echo echoe -echok echoke -echonl echoctl
-echoprt -altwerase -noflsh -tostop -flusho -pendin -nokerninfo
-extproc
iflags: -istrip icrnl -inlcr -igncr ixon -ixoff ixany imaxbel -iutf8
-ignbrk brkint -inpck -ignpar -parmrk
oflags: opost -onlcr -oxtabs -onocr -onlret
cflags: cread cs8 -parenb -parodd hupcl -clocal -cstopb -crtscts -dsrflow
-dtrflow -mdmbuf
cchars: discard = ^O; dsusp = ^Y; eof = ^D; eol = <undef>;
eol2 = <undef>; erase = <undef>; intr = ^C; kill = <undef>;
lnext = ^V; min = 1; quit = ^\; reprint = ^R; start = ^Q;
status = ^T; stop = ^S; susp = ^Z; time = 0; werase = ^W;
### note the echo setting under lflags and the onlcr setting under
### oflags; also note that the stty -a command above was not echoed
sh-3.2$ python ### next I start an interactive python session
Python 2.6.1 (r261:67515, Feb 11 2010, 00:51:29)
[GCC 4.2.1 (Apple Inc. build 5646)] on darwin
Type "help", "copyright", "credits" or "license" for more information.

import termios import termios ### note the echoing
old = termios.tcgetattr(1) old = termios.tcgetattr(1)
new = termios.tcgetattr(1) new = termios.tcgetattr(1)
new[3] = new[3] & ~termios.ECHO & ~termios.ONLCR new[3] = new[3] & ~termios.ECHO
old[3], new[3]

old[3], new[3]
(536872395, 536872385)

(termios.tcsetattr(1, termios.TCSANOW, new), termios.tcgetattr(1)[3], old[3], new[3])

(termios.tcsetattr(1, termios.TCSANOW, new), termios.tcgetattr(1)[3], old[3], new[3])
(None, 536872385, 536872395, 536872385)
### The output above shows that the setting attempted through
### tcsetattr "took" momentarily...termios.tcgetattr(1) == old
True
### ...but by this point it has already been reset back to its original value.

termios.tcgetattr(1)[3], old[3], new[3]

termios.tcgetattr(1)[3], old[3], new[3]
(536872395, 536872395, 536872385)sh-3.2$ stty -a ### after quitting python, the echo and onlcr settings have been reversed
stty -a
speed 9600 baud; 0 rows; 0 columns;
lflags: icanon isig iexten echo echoe -echok echoke -echonl echoctl
-echoprt -altwerase -noflsh -tostop -flusho -pendin -nokerninfo
-extproc
iflags: -istrip icrnl -inlcr -igncr ixon -ixoff ixany imaxbel -iutf8
-ignbrk brkint -inpck -ignpar -parmrk
oflags: opost onlcr -oxtabs -onocr -onlret
cflags: cread cs8 -parenb -parodd hupcl -clocal -cstopb -crtscts -dsrflow
-dtrflow -mdmbuf
cchars: discard = ^O; dsusp = ^Y; eof = ^D; eol = <undef>;
eol2 = <undef>; erase = ^?; intr = ^C; kill = ^U; lnext = ^V;
min = 1; quit = ^\; reprint = ^R; start = ^Q; status = ^T;
stop = ^S; susp = ^Z; time = 0; werase = ^W;
sh-3.2$


Does anyone understand what's going on? Is there any way to prevent
python from resetting the settings made with tcgetattr? (I realize
that this is relatively low-level stuff, so it is unlikely that
there's a clean solution; I'm hoping, however, that there may be
a way to fool python into doing the right thing; after all, this
strange behavior only happens under the Emacs shell; I don't observe
it under, e.g., Terminal or xterm.)

TIA!

~kj

 

[Lawrence D'Oliveiro]

I tried to fix the problem by applying the equivalent of "stty
-echo" within a python interactive session, but discovered that
this setting is immediately (and silently) overwritten.

That seems reasonable behaviour; the command loop is resetting the terminal
to a reasonable state to allow it to read the next command. But the command
you execute can do anything it likes in-between. What’s wrong with that?

 

[kj]

That seems reasonable behaviour; the command loop is resetting the terminal
to a reasonable state to allow it to read the next command. But the command
you execute can do anything it likes in-between. What’s wrong with that?

What's wrong with it is that what python thinks is a "reasonable
state" is actually wrong in this case (it differs from the default
setting established by the Emacs shell). I had hoped that there
was a way to tell python what to regard as a "reasonable state".
I gather that there is no way to do this?

~kj

 

[Nobody]

I tried to fix the problem by applying the equivalent of "stty -echo"
within a python interactive session, but discovered that this setting is
immediately (and silently) overwritten.

FWIW, I don't see this behaviour with Python 2.6.5 on Linux. If I change
the tty settings from within an interactive Python session, they stay
changed. The only reference to tcsetattr() I can find in the Python source
code is in the termios module.

I suspect that it may be related to the "readline" module. My Python is
built with readline support (I normally use XEmacs' comint-based modes
rather than e.g. xterm, and readline either doesn't help or gets in the
way). Also, the implementation of the readline module is platform-specific:

<http://docs.python.org/library/readline.html>

The readline module defines a number of functions to facilitate
completion and reading/writing of history files from the Python
interpreter. This module can be used directly or via the rlcompleter
module. Settings made using this module affect the behaviour of both the
interpreter's interactive prompt and the prompts offered by the
raw_input() and input() built-in functions.

Note: On MacOS X the readline module can be implemented using the libedit
library instead of GNU readline.

 

[Lawrence D'Oliveiro]

What's wrong with it is that what python thinks is a "reasonable
state" is actually wrong in this case (it differs from the default
setting established by the Emacs shell).

I personally wouldn’t try to run one program that wants to do its
own interactive terminal control from within another such program;
that’s just asking for trouble.

I wrote an Emacs function called make-command-buffer (available
as part of <http://github.com/ldo/emacs-prefs>) which feeds a selected
part of the buffer’s contents to a predefined command (which can be
an interpreter like python, bash, perl, bc or whatever you want), and
inserts any returned output into the same buffer. That way you keep all
the necessary editing and history management within Emacs.