expect, rexpect and others

[Ben Giddings]

I know that now Ruby includes "expect", and that's great, however I noticed
that it reads data one character at a time, I'm trying to use it on
something that spits out a *lot* of data, and don't think this is the best
solution to the problem.

I then came across this page:

http://www.rubygarden.org/ruby?RExpect

It attempts to fix some of the perceived flaws with expect, including
getting things one character at a time.

Does anybody have any experience with either, and have any suggestions to
someone wanting this kind of functionality?

Ben

 

[Ben Giddings]

http://www.rubygarden.org/ruby?RExpect

It attempts to fix some of the perceived flaws with expect, including
getting things one character at a time.

Since I had no answers, let me ask a few more questions. Having looked at
the RExpect stuff, it looks much better than the standard 'expect' library,
mainly because it isn't character based. So, has anybody used it? Does
anybody know if it exists in any form other than a web page?

Ben

 

[John Carter]

Since I had no answers, let me ask a few more questions. Having looked at
the RExpect stuff, it looks much better than the standard 'expect' library,
mainly because it isn't character based. So, has anybody used it? Does
anybody know if it exists in any form other than a web page?

Slightly unfair of me to comment since I wrote it....

But yes, I have used it to drive very many (100's) of real and virtual
devices simultaneously. So yes, it is robust and does scale, which I
found the standard one didn't.

That's not quite the absolute latest version I have but should be good
to try. (The API hasn't changed except I now through a TimeoutError
instead of EOFError on timeout.)

RubyForge didn't exist when I submitted it, I guess I should create a
RubyForge project for it. Except it is so small it is more appropriate
to included it as part of some other project.

Is there a "Standards Track" library project for Ruby
like the http://www.boost.org library for C++?


John Carter Phone : (64)(3) 358 6639
Tait Electronics Fax : (64)(3) 359 4632
PO Box 1645 Christchurch Email : (e-mail address removed)
New Zealand

A Million Monkeys can inflict worse things than just Shakespeare on
your system.

 

[Ben Giddings]

Slightly unfair of me to comment since I wrote it....

But yes, I have used it to drive very many (100's) of real and virtual
devices simultaneously. So yes, it is robust and does scale, which I
found the standard one didn't.
Great!

That's not quite the absolute latest version I have but should be good
to try. (The API hasn't changed except I now through a TimeoutError
instead of EOFError on timeout.)

Ok, a few questions then:


Is there some historical reason why the result of a match is "buffer", $1,
$2, $3...? To me, it would make more sense to do

matchdata = regexp.match(buffer)

return matchdata.to_a

I don't see why it is useful to return the whole buffer, and also this
allows really complex regexps with more than 9 elements.

Also, the other efficiency thing that concerns me is that both versions
attempt to match against the entire buffer. The majority of regexps
*could* match something huge, but are generally only used for something
less than say 80 chars long. I want to use an 'expect' type library
against something that is updating a count of each byte of a 200k+ file
written. After that progress meter it prints "Done.", which is all I
really want to match against. It would be more efficient if there were an
option to specify the maximum size of the buffer to use as a match, and to
throw out parts of a buffer bigger than that. Something like:

# Append the data to the match buffer, and shorten it if necessary
match_buff << read_buff
if match_buff.length > maxlength
match_buff = match_buff[-maxlength .. -1]
end

RubyForge didn't exist when I submitted it, I guess I should create a
RubyForge project for it. Except it is so small it is more appropriate
to included it as part of some other project.

It really isn't all that small, I mean, look at most of the ruby libs,
they're much smaller. I think it's worth adding to the stdlib.

Ben

 

[Simon Strandgaard]

Is there some historical reason why the result of a match is "buffer", $1,
$2, $3...? To me, it would make more sense to do

I think $1..$9 are inspired by perl..

matchdata = regexp.match(buffer)

Yes that makes much more sense.



BTW: I am working on a regexp engine written entirely in Ruby.
http://raa.ruby-lang.org/list.rhtml?name=regexp

 

[Jamis Buck]

Is there some historical reason why the result of a match is "buffer",
$1, $2, $3...? To me, it would make more sense to do

matchdata = regexp.match(buffer)

return matchdata.to_a

The other thing I would really like to see in this library is the
ability to specify an IO stream that will display the output of the
process... Perhaps there already is a way, but couldn't see it. What I
did was add an output_stream= setter, and then in the inputData routine
I just sent 'data' to the output stream, if it was set.

Does the library already support this functionality? Is there a better
way to do this? (I'm thinking along the lines of the 'interact' feature
of the python 'pexpect' library.)

 

[John Carter]

Ok, I'm setting up a RubyForge project for it, anyone interested in
being co-developers on it?

Is there some historical reason why the result of a match is "buffer", $1,
$2, $3...? To me, it would make more sense to do

Well I started it as a drop in replacement for the one in the Ruby
standard library.(Which is admirably brief and elegant), I already had
code that was using that, but it was too slow.

I think at one stage they did that. I see now they have
if mat=e_pat.match(buf) then
result = [buf,*mat.to_a[1..-1]]

One major API change relative to the standard expect.rb is the timeout
is the number of seconds the select blocks waiting for input.

So if the timeout is 2 seconds and single character trickles in every
second you never ever timeout. In RExpect I timeout if I haven't seen
a match in timeout seconds.

Also, the other efficiency thing that concerns me is that both versions
attempt to match against the entire buffer. The majority of regexps
*could* match something huge, but are generally only used for something
less than say 80 chars long. I want to use an 'expect' type library
against something that is updating a count of each byte of a 200k+ file
written. After that progress meter it prints "Done.", which is all I
really want to match against. It would be more efficient if there were an
option to specify the maximum size of the buffer to use as a match, and to
throw out parts of a buffer bigger than that. Something like:

Once I have it checked into Rubyforge you are welcome to hack and
patch it and either join me as a developer with CVS access rights or
just send me patches. My only request is to try keep the API
reasonably backward compatible, either by use of default parameters or
providing new interfaces.





John Carter Phone : (64)(3) 358 6639
Tait Electronics Fax : (64)(3) 359 4632
PO Box 1645 Christchurch Email : (e-mail address removed)
New Zealand

A Million Monkeys can inflict worse things than just Shakespeare on
your system.

 

[John Carter]

Does the library already support this functionality? Is there a better
way to do this? (I'm thinking along the lines of the 'interact' feature
of the python 'pexpect' library.)

I'm totally unfamiliar with pexpect, once I have commented RExpect and
checked it into RubyForge, will you compare with pexpect and make any
suggestions (or just hack them in your self)...



John Carter Phone : (64)(3) 358 6639
Tait Electronics Fax : (64)(3) 359 4632
PO Box 1645 Christchurch Email : (e-mail address removed)
New Zealand

A Million Monkeys can inflict worse things than just Shakespeare on
your system.