Defeating OS buffering?

[Eric Schwartz]

I'm trying to use Perl to automate a specific test process, but OS
buffering is defeating some of the main utility of it. I run a
program like this:

open(TEST, "$testcmd |") or die "bah-- $!";
while(<TEST>) {
# do stuff
}
close(TEST);

Part of the problem here is that the test referred to in $testcmd is a
bash script that execs a C program, which I do not have source to.
This C program doesn't print a lot of output, so as a consequence, I
don't see any of it until the C program exits. Is there any way,
short of modifying the C program, that I can defeat the OS buffering
of the C program's output?

-=Eric, fully expecting the answer to be "no".

 

[Vlad Tepes]

I'm trying to use Perl to automate a specific test process, but OS
buffering is defeating some of the main utility of it. I run a
program like this:

open(TEST, "$testcmd |") or die "bah-- $!";
while(<TEST>) {
# do stuff
}
close(TEST);

Part of the problem here is that the test referred to in $testcmd is a
bash script that execs a C program, which I do not have source to.
This C program doesn't print a lot of output, so as a consequence, I
don't see any of it until the C program exits. Is there any way,
short of modifying the C program, that I can defeat the OS buffering
of the C program's output?

-=Eric, fully expecting the answer to be "no".

How about reading single bytes bytes from the program?

open(TEST, "/home/t/c/a.out |") or die "bah-- $!";
$|++;
while( 0 != read TEST, $_, 1, 1 ) {
print;
}
close(TEST);

HTH,

 

[Eric Schwartz]

How about reading single bytes bytes from the program?

open(TEST, "/home/t/c/a.out |") or die "bah-- $!";

To reproduce this properly, you'd need a bash program that just did:

exec /home/t/c/a.out

and you'd invoke the bash program there.

$|++;

This unbuffers STDOUT, which doesn't help when the problem is that the
OS is buffering the TEST filehandle.

while( 0 != read TEST, $_, 1, 1 ) {
print;
}
close(TEST);

How is reading input that doesn't show up a character at a time going
to help? Either it's there, or it isn't; reading it in smaller chunks
is just making things more inefficient without actually changing
anything.

HTH,

I'm afraid not. Thanks for the effort, anyhow.

-=Eric

 

[Vlad Tepes]

To reproduce this properly, you'd need a bash program that just did:

exec /home/t/c/a.out

Ok, now I have created a bash-script that exec'ed the a.out, and used
that in the open-statement above. It worked nicely, doing what you
wanted.

Maybe I'm missing something?

The script I used:

#!/bin/bash
exec "/home/t/c/a.out";

The C-program:

#include <stdio.h>
#include <unistd.h>

int main ( void ) {
printf("Some text...");
fflush(NULL);
sleep(3);
printf("some more text...\n");
return(0);
}

The perl program:

#!/usr/bin/perl
use warnings;
use strict;

open(TEST, "/tmp/t.sh |") or die "bah-- $!";
$|++;
while( 0 != read TEST, $_, 1, 1 ){
print;
}
close(TEST);

Hope this helps,

 

[Trent Curry]

[Other code]

The perl program:

#!/usr/bin/perl
use warnings;
use strict;

open(TEST, "/tmp/t.sh |") or die "bah-- $!";
$|++;
while( 0 != read TEST, $_, 1, 1 ){
print;
}
close(TEST);

Or, as an FYI, you could rewrite the while as:

print while (0 != read TEST, $_, 1, 1);

 

[Trent Curry]

To reproduce this properly, you'd need a bash program that just did:

exec /home/t/c/a.out

and you'd invoke the bash program there.


This unbuffers STDOUT, which doesn't help when the problem is that the
OS is buffering the TEST filehandle.

From perldoc perlvar:

$OUTPUT_AUTOFLUSH
$|

If set to nonzero, forces a flush right away and after every write
or print on the currently selected output channel. [Snipped rest]
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

So in this case, TEST is the currently selected output handle.

 

[Eric Schwartz]

The C-program:

#include <stdio.h>
#include <unistd.h>

int main ( void ) {
printf("Some text...");
fflush(NULL);

^^^^^^^^^^^^^
<snip>

That's the problem. Take that out, and it won't work. As I said, I
don't have control over the C program, so I can't make it fflush.

-=Eric

 

[Eric Schwartz]

This unbuffers STDOUT, which doesn't help when the problem is that the
OS is buffering the TEST filehandle.

From perldoc perlvar:

$OUTPUT_AUTOFLUSH
$|

If set to nonzero, forces a flush right away and after every write
or print on the currently selected output channel. [Snipped rest]
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

So in this case, TEST is the currently selected output handle.

No, TEST is open for *INPUT*. Hint: if you can read from it, it's
open for input. If you can write to it, it's open for output. And
even if it were open for output, $| only works on filehandles that you
select with the one-arg select(), which Vlad did not do.

-=Eric

 

[Tad McClellan]

So in this case, TEST is the currently selected output handle.

Not unless there was a call to select() missing from the
code that was posted.

Why go on about output when the OP's problem is with _in_put?

 

[Trent Curry]

$|++;

This unbuffers STDOUT, which doesn't help when the problem is that the
OS is buffering the TEST filehandle.

From perldoc perlvar:

$OUTPUT_AUTOFLUSH
$|

If set to nonzero, forces a flush right away and after every write
or print on the currently selected output channel. [Snipped rest]
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

So in this case, TEST is the currently selected output handle.

No, TEST is open for *INPUT*. Hint: if you can read from it, it's
open for input. If you can write to it, it's open for output. And
even if it were open for output, $| only works on filehandles that you
select with the one-arg select(), which Vlad did not do.

Sorry, you are right, I over looked that little but very significant part.
Thanks for catching that.

 

[Trent Curry]

Not unless there was a call to select() missing from the
code that was posted.

Why go on about output when the OP's problem is with _in_put?

Sorry, that was my screw up. Over looked that part. My apologies.

 

[Vlad Tepes]

^^^^^^^^^^^^^
<snip>

That's the problem. Take that out, and it won't work. As I said, I
don't have control over the C program, so I can't make it fflush.

Then I have to say pass. :-(

I don't know if it is possible to force a program to flush it's output.
And if it is, I suppose the solution will be OS-dependant.

 

[Kevin Michael Vail]

Then I have to say pass. :-(

I don't know if it is possible to force a program to flush it's output.
And if it is, I suppose the solution will be OS-dependant.

Not that this is going to be useful, but if you can put a ptty between
the parent and the child, the child should revert to line buffering
because it'll think its output is going to a device. I have no idea how
to go about this in Perl, though (I used to know how to do it in C).
Also, only works under *NIX variants, as far as I know.

 

[nobull]

^^^^^^^^^^^^^
<snip>

That's the problem. Take that out, and it won't work. As I said, I
don't have control over the C program, so I can't make it fflush.

It will probably fflush() implicitly on newline if it thinks it's
STDOUT is going to an interactive terminal.

I Expect (hint, hint) you could find something on CPAN that would
allow you to fool a subprocess into thinking it's connected to a
terminal.