handling STDIN line by line in Gtk

[Michael Goerz]

Consider the following simplified script:

#!/usr/bin/perl -w
use strict;
use Gtk2;

sub stdin_handler {
my $line = <STDIN>;
if (defined($line)){
print "process: $line";
}
}

Gtk2->init;
Glib::IO->add_watch (fileno(STDIN), ['in'], sub{stdin_handler('dummy')});
Gtk2->main;
#EOF

What I want this to do is to handle commands received from STDIN, line
by line. In the real program, it's going to be data lines that will be
plotted real-time on a GUI.

When I just start the program, and then manually enter input on the
shell, it works as it should: for every line that I type, as soon as I
hit enter, I get back "process: line". However, when I use 'cat' to feed
data to the program (or pipe into it from anywhere else), it just takes
the first line of input, returns that, and then just sits there doing
nothing. What can I do to process STDIN line by line, treating each line
as a single event?

I can't slurp the input with @lines = <STDIN>, that reads all the input,
but the program will wait until EOF before it does any processing on the
GUI--I don't get the real-time plotting.

Any suggestions?

Michael

 

[zentara]

Consider the following simplified script:

#!/usr/bin/perl -w
use strict;
use Gtk2;

sub stdin_handler {

#> #my $line = <STDIN>;
my $line;
sysread STDIN, $line, 1024;

if (defined($line)){
print "process: $line";
}
}

Gtk2->init;
Glib::IO->add_watch (fileno(STDIN), ['in'], sub{stdin_handler('dummy')});
Gtk2->main;
#EOF

When I just start the program, and then manually enter input on the
shell, it works as it should: for every line that I type, as soon as I
hit enter, I get back "process: line". However, when I use 'cat' to feed
data to the program (or pipe into it from anywhere else), it just takes
the first line of input, returns that, and then just sits there doing
nothing. What can I do to process STDIN line by line, treating each line
as a single event?

Any suggestions?
Michael

The problem is that STDIN never reaches EOF or closes, so your
<STDIN> just sits there waiting for new input.

The easiest way is to use a non-blocking sysread. There are also
odd hacks which close and reopen STDIN, but you can google for
them if interested.

If you want to be able to have your program run both from command
line input, or a pipe, you will probably need to test STDIN with the -t
test to see if it's a tty or a pipe. There are subtle differences, which
you will see when you run this thru a pipe or normal.

This will meet most needs:

#!/usr/bin/perl -w
use strict;
use Gtk2;

sub pipe_handler {
my $line;
sysread STDIN, $line, 1024;
if (defined($line)){
print "process: $line";
}
}

#unused sub left in for comparison
sub stdin_handler {
my $line = <STDIN>;
if (defined($line)){
print "process: $line";
}
}

Gtk2->init;

if( -t STDIN ){print "normal input\n"}else{ print "pipe input\n" }

# call different subs here depending on the -t test results

Glib::IO->add_watch (fileno(STDIN), ['in'], sub{pipe_handler('dummy')});

Gtk2->main;
__END__


zentara

 

[Michael Goerz]

#> #my $line = <STDIN>;
my $line;
sysread STDIN, $line, 1024;

Thanks for your answer. This seems to work for my example script, but
not, unfortunately, for the actual program, in which the stdin_handler
is something like this:

sub stdin_handler {
my $drawing_area = shift;
my $line;
sysread STDIN, $line, 1024;
if (defined($line) and $line ne ''){
if ( $line =~ /([-0-9]+)\s+([-0-9]+)(\s+([ ,\w]+))?/ ) {
my $x = $1;
my $y = $2;
my $color = $4;
# [...]
draw_brush($drawing_area, $x, $y, $color);
# [...]
}
print $line;
}
}

What I get is that all input lines are printed out, but only the first
one is parsed and plotted via draw_brush ...?

The complete program is at
http://www.physik.fu-berlin.de/~goerz/download/cellaut/cellautdisplay.pl
and I'm feeding it the data in
http://www.physik.fu-berlin.de/~goerz/download/cellaut/data.csv
with 'cat data.csv | cellautdisplay.pl'

This really seems tricky...

Michael

 

[zentara]

Thanks for your answer. This seems to work for my example script, but
not, unfortunately, for the actual program, in which the stdin_handler
is something like this:

sub stdin_handler {
my $drawing_area = shift;
my $line;
sysread STDIN, $line, 1024;
if (defined($line) and $line ne ''){
if ( $line =~ /([-0-9]+)\s+([-0-9]+)(\s+([ ,\w]+))?/ ) {
my $x = $1;
my $y = $2;
my $color = $4;
# [...]
draw_brush($drawing_area, $x, $y, $color);
# [...]
}
print $line;
}
}

What I get is that all input lines are printed out, but only the first
one is parsed and plotted via draw_brush ...?

The complete program is at
http://www.physik.fu-berlin.de/~goerz/download/cellaut/cellautdisplay.pl
and I'm feeding it the data in
http://www.physik.fu-berlin.de/~goerz/download/cellaut/data.csv
with 'cat data.csv | cellautdisplay.pl'

This really seems tricky...
Michael

In my last post, I mentioned that there were subtle differences the way
sysread grabbed and displayed input. If you notice carefully, it grabs
the entire 1024 bytes, and stuffs it all into 1 $line.

So you either need to:

1. first split $line on newlines and store it in a temp array, then
loop thru the array, applying your regex to each element

2. make the regex global with the /g or possibly multiline match
modifiers to get all matches out of the $line string.

There may be problems, where sysread lops off the last few bytes
at the 1024 mark. So you may have to work out a system for handing that.
Is your csv data always going to be coming in 1 complete run, or
is it possible for it to pause and resume, like in a socket connection.
In other words, is the csv data always coming from a file, or is it
being piped in from a script, which may have spurts of output?

I'll look at your script and data later, and see what I would do.

zentara

 

[zentara]

This really seems tricky...
Michael

Here is the best way I found. It avoids the hacking of
the 1024 byte boundary on the input chunks.

One other thing, don't put
a sleep call in any gui apps, because it blocks the gui.
If you want to slow down the display of the blocks appearing
on the drawing area, push the data into and array, and call
a timer to shift them off, one at a time, and display them.

At the top of your script:

use FileHandle;
my $fh_in = FileHandle->new();
$fh_in = *STDIN{IO};

#then your sub

sub stdin_handler {
my $drawing_area = shift;

while (<$fh_in>) {
my $line = $_;
if (defined($line) and $line ne ''){
if ( $line =~ /([-0-9]+)\s+([-0-9]+)(\s+([ ,\w]+))?/ ) {
my $x = $1;
my $y = $2;
my $color = $4;
$color = 'black' if ((not defined($color)) or ($color eq ''));
# sleep 1;
draw_brush($drawing_area, $x, $y, $color);
} elsif ($line =~ /clear/){
clear($drawing_area);
}

print $line;
}
}
return 0;
}
__END__

zentara

 

[Michael Goerz]

Here is the best way I found. It avoids the hacking of
the 1024 byte boundary on the input chunks.

One other thing, don't put
a sleep call in any gui apps, because it blocks the gui.
If you want to slow down the display of the blocks appearing
on the drawing area, push the data into and array, and call
a timer to shift them off, one at a time, and display them.

Thanks so much for your answer! Your last comment actually provided the
solution: I use the stdin_handler only to read input and store
everything into a global array, and the I can have a timeout that shifts
off one line from that array once every interval. This way, the
animation of the data is it comes in is drawn out on the screen, just
like I intended it to be.

Thanks,
Michael