Python 2.2 code continues running before list comprehension is completed?

[Chris P.]

Hi. I've made a program that logs onto a telnet server, enters a
command, and then creates a list of useful information out of the
information that is dumped to the screen as a result of the command.
Here's a generic version of the code in question:

#####

# Prior code opens telnet connection "tn" and logs in.
tn.read_until('> ')
tn.write('THE COMMAND IS HERE\n')
dump = tn.read_very_eager()
dump_lines = dump.split('\n')
dump_info = [x for x in dump_lines if (VARIOUS CONDITIONS ON x)]

# string1 and string2 are strings that are known to
# always appear in the dump.
A = dump_info.index('string1\r')
B = dump_info.index('string2\r')
C = A - B - 1

#####

The thing is, I can step through this code in the debugger and it
works fine. When run from the command line, however, I get the
following error in response to my dump_info.index lines:

ValueError: list.index(x): x not in list

So I tried putting (for x in dump_info: print x) right after I make
the list comprehension and, when I run that, it shows that the list is
incomplete... so it SEEMS like Python begins to create the list
comprehension and then continues to execute commands before the list
comprehension is finished!

Any help with a solution or workaround will be greatly appreciated,

- Chris

 

[Heiko Wundram]

Am Montag, 19. Juli 2004 16:25 schrieb Chris P.:

Any help with a solution or workaround will be greatly appreciated,

IIRC, Python 2.2 didn't support list comprehensions (feature new as of 2.3),
so you'll have to work around that by creating some form of simple for loop
which does what you want, esp. considering that VARIOUS CONDITIONS ON x seems
to be a lot more than just a simple check (which will make the code a
helluvalot cleaner if spelled out).

Heiko.

 

[Sion Arrowsmith]

# Prior code opens telnet connection "tn" and logs in.
tn.read_until('> ')
tn.write('THE COMMAND IS HERE\n')
dump = tn.read_very_eager()
dump_lines = dump.split('\n')
dump_info = [x for x in dump_lines if (VARIOUS CONDITIONS ON x)]
[ ... ]
So I tried putting (for x in dump_info: print x) right after I make
the list comprehension and, when I run that, it shows that the list is
incomplete... so it SEEMS like Python begins to create the list
comprehension and then continues to execute commands before the list
comprehension is finished!

Did you try putting a print in *before* the list comprehension
(for x in dump_lines: print x) to make sure you're getting all
the lines you think you're getting? Documentation says
read_very_eager() "Read everything that's possible without
blocking in I/O" -- my first thought is that your client
running outside the debugger is reading faster than the server
can feed it data, it blocks part way through the execution of
the command, and read_very_eager() returns without the complete
output.

 

[Peter Otten]

IIRC, Python 2.2 didn't support list comprehensions (feature new as of
2.3), so you'll have to work around that by creating some form of simple
for loop which does what you want, esp. considering that VARIOUS
CONDITIONS ON x seems to be a lot more than just a simple check (which
will make the code a helluvalot cleaner if spelled out).

Python 2.2.1 (#1, Sep 10 2002, 17:49:17)
[GCC 3.2] on linux2
Type "help", "copyright", "credits" or "license" for more information.

[c for c in "you are wrong"][-5:] ['w', 'r', 'o', 'n', 'g']

(but only with regard to list comprehensions)

Peter

 

[Peter Otten]

(but only with regard to list comprehensions)

....not being supported by 2.2

 

[Peter Otten]

Hi. I've made a program that logs onto a telnet server, enters a
command, and then creates a list of useful information out of the
information that is dumped to the screen as a result of the command.
Here's a generic version of the code in question:

#####

# Prior code opens telnet connection "tn" and logs in.
tn.read_until('> ')
tn.write('THE COMMAND IS HERE\n')
dump = tn.read_very_eager()

You could try tn.read_all() instead (just a guess).

assert "string1\r" in dump # would do no harm
assert ""string2\r" in dump

dump_lines = dump.split('\n')
dump_info = [x for x in dump_lines if (VARIOUS CONDITIONS ON x)]

# string1 and string2 are strings that are known to
# always appear in the dump.

Bold claim

A = dump_info.index('string1\r')
B = dump_info.index('string2\r')
C = A - B - 1

#####

The thing is, I can step through this code in the debugger and it
works fine. When run from the command line, however, I get the
following error in response to my dump_info.index lines:

ValueError: list.index(x): x not in list

So I tried putting (for x in dump_info: print x) right after I make
the list comprehension and, when I run that, it shows that the list is
incomplete... so it SEEMS like Python begins to create the list
comprehension and then continues to execute commands before the list
comprehension is finished!

I'm pretty sure that the list comprehension is finished. Look out for
something else.

Peter

 

[Heiko Wundram]

Am Montag, 19. Juli 2004 18:31 schrieb Peter Otten:

Python 2.2.1 (#1, Sep 10 2002, 17:49:17)
[GCC 3.2] on linux2
Type "help", "copyright", "credits" or "license" for more information.

[c for c in "you are wrong"][-5:]

['w', 'r', 'o', 'n', 'g']

Humm... Okay, I don't have an installation of 2.2 around anymore, so I
couldn't check before I posted... Forgive my faux pas.

Heiko.

 

[Chris P.]

Hey. Just thought I'd give an update that I figured out how to make
my script work. The original code was:

tn.read_until('> ')
tn.write('THE COMMAND IS HERE\n')
dump = tn.read_very_eager()
dump_lines = dump.split('\n')
dump_info = [x for x in dump_lines if (VARIOUS CONDITIONS ON x)]

Sion had suggested that the client is reading faster than the server,
and I might want to try tn.read_all() instead of tn.read_very_eager().
That actually didn't work for me - read_all() documentation says:
"Read all data until EOF; block until connection closed" and that's a
problem... it makes my system hang, I'm guessing because the command I
send doesn't close the connection directly after it runs.

SO, I should have just looked to myself for inspriation: replacing the
tn.read_very_eager() with tn.read_until('> ') ensures that everything
up to the next command prompt is read in.

- Chris