Does fileinput.input() read STDIN all at once?

[Adam Funk]

I'm using this sort of standard thing:

for line in fileinput.input():
do_stuff(line)

and wondering whether it reads until it hits an EOF and then passes
lines (one at a time) into the variable line. This appears to be the
behaviour when it's reading STDIN interactively (i.e. from the
keyboard).

As a test, I tried this:

for line in fileinput.input():
print '**', line

and found that it would print nothing until I hit Ctl-D, then print
all the lines, then wait for another Ctl-D, and so on (until I pressed
Ctl-D twice in succession to end the loop).

Is it possible to configure this to pass each line of input into line
as it comes?

Thanks,
Adam

 

[Jonathan Gardner]

I'm using this sort of standard thing:

for line in fileinput.input():
do_stuff(line)

and wondering whether it reads until it hits an EOF and then passes
lines (one at a time) into the variable line. This appears to be the
behaviour when it's reading STDIN interactively (i.e. from the
keyboard).

As a test, I tried this:

for line in fileinput.input():
print '**', line

and found that it would print nothing until I hit Ctl-D, then print
all the lines, then wait for another Ctl-D, and so on (until I pressed
Ctl-D twice in succession to end the loop).

There is probably a 1024 byte buffer. Try filling it up and see if you
get something before you hit CTRL-D.

Otherwise, the writers have to flush the buffer if they want a line to
be sent before the buffer fills up. C++ endl would do this, I believe,
in addition to printing '\n'.

Note that terminals (what you get if you are typing from the command
line) are terminals and behave quite differently than open files.

Is it possible to configure this to pass each line of input into line
as it comes?

It's up to the writer to flush the buffers. There's not much you can
do, so just learn to live with it.

It sounds like you want to write some kind of interactive program for
the terminal. Do yourself a favor and use curses or go with a full-
blown GUI.

 

[Adam Funk]

There is probably a 1024 byte buffer. Try filling it up and see if you
get something before you hit CTRL-D.

Thanks; I'm looking into the buffering question.

It sounds like you want to write some kind of interactive program for
the terminal. Do yourself a favor and use curses or go with a full-
blown GUI.

No, I'm really interested in reading the input from files!

This just came up because I was trying to test the program by giving
it no filename arguments and typing sample input.

 

[Tim Roberts]

I'm using this sort of standard thing:

for line in fileinput.input():
do_stuff(line)

and wondering whether it reads until it hits an EOF and then passes
lines (one at a time) into the variable line. This appears to be the
behaviour when it's reading STDIN interactively (i.e. from the
keyboard).

As a test, I tried this:

for line in fileinput.input():
print '**', line

and found that it would print nothing until I hit Ctl-D, then print
all the lines, then wait for another Ctl-D, and so on (until I pressed
Ctl-D twice in succession to end the loop).

Note that you are doing a lot more than just reading from a file here. This
is calling a function in a module. Fortunately, you have the source to
look at.

As I read the fileinput.py module, this will eventually call
FileInput.next(), which eventually calls FileInput.readline(), which
eventually calls stdin.readlines(_bufsize). The default buffer size is
8,192 bytes.

 

[Adam Funk]

Note that you are doing a lot more than just reading from a file here. This
is calling a function in a module. Fortunately, you have the source to
look at.

As I read the fileinput.py module, this will eventually call
FileInput.next(), which eventually calls FileInput.readline(), which
eventually calls stdin.readlines(_bufsize). The default buffer size is
8,192 bytes.

OK, I think I've got this figured out!

I'll keep using fileinput.input() for the normal case (reading from
files named as arguments) and use sys.stdin.readline() for testing
(reading from the keyboard, when no filenames are given).

Thanks to you and Jonathan for your advice.