getting standard error and output from ruby script in real time

[Stefano Crocco]

I'm trying to write an editor for ruby in ruby using the KDE4 bindings for
ruby and I'm facing a problem which I don't know whether is caused by ruby or
by KDE. I want my editor to be able to run a ruby script and display both the
error messages and the normal output in a window. The problem is that all the
text sent to the standard output seems to be received together from my
application, and so does the text sent to standard error, even if the two are
mixed. For example, when I run the following script

5.times do |i|
warn "Warning #{i}"
puts i
end

the output window of my program displays this:
Warning 0
Warning 1
Warning 2
Warning 3
Warning 4
0
1
2
3
4

instead of this (which is what I get if I run the script from terminal)
Warning 0
0
Warning 1
1
Warning 2
2
Warning 3
3
Warning 4
4

Since I don't have much experience on interprocess communication, I can't, as
I said, understand if this issue is due to the way ruby prints its output or
to the tools I use to run the ruby script from my editor, which are specific
of KDE. I'd be glad if someone could give me a hint on this, so that at least
I can know whether I have to look for documentation about ruby or about KDE.

Thanks in advance

Stefano

 

[Dave Bass]

of KDE. I'd be glad if someone could give
me a hint on this, so that at least
I can know whether I have to look for
documentation about ruby or about
KDE.

It's to do with buffering. Buffering takes place within Ruby (I think)
but also in the operating system. Stderr is usually unbuffered whereas
stdout is usually buffered. I.e. the output is accumulated in a memory
buffer until it reaches a certain size, then the block is flushed to the
output stream. This is a lot more efficient that outputting it
character-by-character to the output device.

In my experience (various languages and OSes), buffering can cause
problems that are difficult or impossible to resolve.

 

[Xeno Campanoli]

I'm trying to write an editor for ruby in ruby using the KDE4 bindings for
ruby and I'm facing a problem which I don't know whether is caused by ruby or
by KDE. I want my editor to be able to run a ruby script and display both the
error messages and the normal output in a window. The problem is that all the
text sent to the standard output seems to be received together from my
application, and so does the text sent to standard error, even if the two are
mixed. For example, when I run the following script

5.times do |i|
warn "Warning #{i}"
puts i
end

the output window of my program displays this:
Warning 0
Warning 1
Warning 2
Warning 3
Warning 4
0
1
2
3
4

instead of this (which is what I get if I run the script from terminal)
Warning 0
0
Warning 1
1
Warning 2
2
Warning 3
3
Warning 4
4

Since I don't have much experience on interprocess communication, I can't, as
I said, understand if this issue is due to the way ruby prints its output or
to the tools I use to run the ruby script from my editor, which are specific
of KDE. I'd be glad if someone could give me a hint on this, so that at least
I can know whether I have to look for documentation about ruby or about KDE.

Thanks in advance

Stefano

STDERR.puts
puts is stdout.

 

[Stefano Crocco]

STDERR.puts
puts is stdout.

I know that puts writes on standard output, this is what I want it to do. It's
warn which outputs to standard error.

Stefano

 

[Stefano Crocco]

It's to do with buffering. Buffering takes place within Ruby (I think)
but also in the operating system. Stderr is usually unbuffered whereas
stdout is usually buffered. I.e. the output is accumulated in a memory
buffer until it reaches a certain size, then the block is flushed to the
output stream. This is a lot more efficient that outputting it
character-by-character to the output device.

In my experience (various languages and OSes), buffering can cause
problems that are difficult or impossible to resolve.

Thanks for the information. I think I'll look at the source code of some tool
which does what I want and see how it works.

Stefano

 

[Rob Biedenharn]

Thanks for the information. I think I'll look at the source code of
some tool
which does what I want and see how it works.

Stefano

$stdout.sync = true; $stdout.flush

The #sync setting of true means "unbuffered" (sync'd to the
destination every time). If you build up messages, you can call
#flush on the IO yourself instead.

print "hello "
print "world"
puts "!"
STDOUT.flush

-Rob

Rob Biedenharn http://agileconsultingllc.com
(e-mail address removed)

 

[Stefano Crocco]

$stdout.sync = true; $stdout.flush

The #sync setting of true means "unbuffered" (sync'd to the
destination every time). If you build up messages, you can call
#flush on the IO yourself instead.

print "hello "
print "world"
puts "!"
STDOUT.flush

I don't think that would work. If I understand correctly what you mean, that
should be put in the script which produces the output, not not in the one
displaying it. But since the output I want to display is from a script written
by the user, I can't do that.

Stefano

 

[Rob Biedenharn]

I don't think that would work. If I understand correctly what you =20
mean, that
should be put in the script which produces the output, not not in =20
the one
displaying it. But since the output I want to display is from a =20
script written
by the user, I can't do that.

Stefano

If you have no need to maintain separate streams do the equivalent of =20=

the shell's:

2>&1

so that stderr is the same as stdout.

$stderr.reopen($stdout)=01

You might want to do that in your child process before exec'ing the =20
user's script.

-Rob

Rob Biedenharn http://agileconsultingllc.com
(e-mail address removed)=

 

[ara.t.howard]

I don't think that would work. If I understand correctly what you
mean, that
should be put in the script which produces the output, not not in
the one
displaying it. But since the output I want to display is from a
script written
by the user, I can't do that.

what you want to do is easily accomplished using session.rb

http://codeforpeople.com/lib/ruby/session/session-2.4.0/README

gem install session


note that i wrote to do *exactly* what you are doing - but it was for
a tk gui. anyhow, you *must* do this in a background thread as there
are certain things child programs can do (like setvbuf) which can
defeat any counter buffering techniques you may try - so you must
always process the stdout/stderr asynchronously in a background thread
which updates the gui. session makes this trivial to accomplish.

kind regards.

a @ http://codeforpeople.com/

 

[Stefano Crocco]

what you want to do is easily accomplished using session.rb

http://codeforpeople.com/lib/ruby/session/session-2.4.0/README

gem install session


note that i wrote to do *exactly* what you are doing - but it was for
a tk gui. anyhow, you *must* do this in a background thread as there
are certain things child programs can do (like setvbuf) which can
defeat any counter buffering techniques you may try - so you must
always process the stdout/stderr asynchronously in a background thread
which updates the gui. session makes this trivial to accomplish.

kind regards.

a @ http://codeforpeople.com/

Thanks for telling me about session. Unfortunately, it still doesn't work. I
tried with the following code (there's no ui because I wanted to understand
how session works before using it in a more complex situation):

require 'session'

t=Thread.new do
sh = Session.new
sh.execute( 'ruby /home/stefano/documenti/scripts/prova.rb' ) do |out, err|
puts "Msg: #{out}" if out
puts "Err: #{err}" if err
end
end
t.join

The result is the same I already got, that is: all the error messages together
and all the stdout messages together. At any rate, tried modifiying my test
script:

$stdout.sync=true #added line
10.times{|i|
puts i
warn "Possible error #{i}"
sleep 1 #added line
}

With the introduction of the two lines marked with 'added line', the output is
the one expected (even if it seems that the order in wich the error message
and the stdout message are displayed for each iteration is random). Removing
one of those lines brings it back to the 'wrong' behavior.

However, I've tried to run the same script using other editors
(kdevelop, scite, netbeans) and the results are the same as mine. This seems
to mean that there's not an easy way to accomplish what I want to do.