[ANN] FastRI 0.3.1: faster, Leopard compatibility, etc.

[Mauricio Fernandez]

FastRI is an alternative to the ri documentation browser for Ruby.

Why prefer it over ri? FastRI
* actually works
* is *much* faster than ri
* can perform fast full-text searching over all your rdoc documentation
* is smarter than ri, and can find classes anywhere in the hierarchy without
specifying the "full path"
* can be told to use different search strategies (completion, nested
namespaces, etc.)
* supports RubyGems much better than ri, and knows which gem a method/class
came from
* can serve RI documentation over DRb

Getting it
==========
Additional information, tarballs... at
http://eigenclass.org/hiki/fastri

FastRI can be installed with RubyGems:
gem install fastri
(if you get an old version/a 404 error, please allow some time after the
release until the package propagates to the RubyForge mirrors). Please read
below for an important note regarding the RubyGems packages.

Visible changes since version 0.3.0
===================================
* speed increases
* works on Leopard and with non-standard RubyGems setups
* -1 (--exact) option to disable second-guessing.
* -a (--all) option to show the documentation for all matching entries

Usage
=====
Starting from 0.3.0, fri can be used in either local mode (--local, -L) or
remote mode (using a fastri-server, -R, --remote).
By default, fri will run in remote mode. There's a new executable named
qri that defaults to local mode (it behaves otherwise identically to fri).

Remote mode is slightly faster (typically about 150ms) , since the
documentation index is kept in memory and needs not be read from disk.

Local mode
----------
$ qri Array
----------------------------------------------------------- Class: Array
Arrays are ordered, integer-indexed collections of any object.
Array indexing starts at 0, as in C or Java. A negative index is
...

$ qri compact
---------------------------------------------------------- Array#compact
array.compact -> an_array
------------------------------------------------------------------------
Returns a copy of self with all nil elements removed.


Remote mode
-----------
There are two parts to using FastRI in remote mode:
* the server: fastri-server
* the client: fri

FastRI uses a Rinda Ring to allow servers to be discovered automatically
without needing to indicate the DRb URIs manually. It can work across
machines if you make sure the ring server is bound to the correct interface,
and the ACL permissions are correct.

Running in remote mode:

$ fastri-server (creates the index on the first run, blocks)

Later, (times measured with a cold cache):
$ time ruby bin/fri -f plain Array#fetch
------------------------------------------------------------ Array#fetch
array.fetch(index) -> obj
[...]
real 0m0.287s (real 0m0.127s with a hot cache)
user 0m0.048s
sys 0m0.008s

Compare to:
$ time ri -T -f plain Array#fetch
------------------------------------------------------------ Array#fetch
[...]
real 0m10.136s (real ~ 1.5s with a hot cache)
user 0m1.140s
sys 0m0.464s

This illustrates FastRI's ability to locate classes deep in the class
hierarchy:

$ fri Base
------------------------------------------------------ Multiple choices:

ActionMailer::Base, ActionView::Base, ActionWebService::API::Base,
ActionWebService::Base, ActionWebService::Client::Base,
ActiveRecord::Base, MapReduce::ActiveRecord::Base,
RSS::Maker::Base, Scruffy::Components::Base,
Scruffy::Formatters::Base, Scruffy::Layers::Base,
Scruffy::Renderers::Base, Scruffy::Themes::Base

$ fri Themes::Base
------------------------------------------- Class: Scruffy::Themes::Base
Scruffy::Themes::Base
Author: Brasten Sager

Date: August 14th, 2006

Compare to
$ ri Themes::Base .... several seconds later ...
Nothing known about Themes::Base

A small note about RubyGems + FastRI.
=====================================
RubyGems adds a noticeable overhead to fri, making it run slower than if you
installed it directly from the tarball with setup.rb.

Compare the execution time when installed with RubyGems:
$ time fri -f plain String > /dev/null

real 0m0.385s
user 0m0.244s
sys 0m0.036s

to the time fri actually takes to run, without the overhead introduced by
RubyGems:
$ time ruby bin/fri -f plain String > /dev/null

real 0m0.088s
user 0m0.040s
sys 0m0.008s

If you care about those extra 300ms (and there are situations where they will
matter, e.g. when using fri for method completion), get FastRI from the
tarballs.

License
=======
FastRI is licensed under the same terms as Ruby. See LICENSE.

Feedback
========
Bug reports, patches, comments... are appreciated.
You can contact the author via <[email protected]>. Please add "fastri" to the
subject in order to bypass the spam filters.

 

[Joel VanderWerf]

Does this happen to anyone else?

$ qri String
/usr/local/lib/ruby/gems/1.8/gems/fastri-0.3.1.1/lib/fastri/ri_index.rb:354:in
`initialize': No such file or directory -
/usr/local/lib/ruby/gems/1.8/doc/mocha-0.4.0/ri/String/cdesc-String.yaml
(Errno::ENOENT)
...

I've tried uninstalling mocha (qri fails the same way), and reinstalling
mocha (ditto).

 

[Joel VanderWerf]

Does this happen to anyone else?

$ qri String
/usr/local/lib/ruby/gems/1.8/gems/fastri-0.3.1.1/lib/fastri/ri_index.rb:354:in
`initialize': No such file or directory -
/usr/local/lib/ruby/gems/1.8/doc/mocha-0.4.0/ri/String/cdesc-String.yaml
(Errno::ENOENT)
...

I've tried uninstalling mocha (qri fails the same way), and reinstalling
mocha (ditto).

Well, whatever it was, it got fixed by rebuilding the index:

fastri-server -b

(It would be nice if fri/qri could suggest this course of action when an
exception is raised while using the index.)

 

[botp]

FastRI is an alternative to the ri documentation browser for Ruby.

forgive the dumb question, but how can i make fastri work for ruby1.9?
thanks for fri/qri, mauricio. 'm very much dependent on it now
-botp

 

[Daniel Schömer]

FastRI is an alternative to the ri documentation browser for Ruby.
[...]
Visible changes since version 0.3.0
===================================
* speed increases
[...]
Compare the execution time when installed with RubyGems:
$ time fri -f plain String > /dev/null

real 0m0.385s
user 0m0.244s
sys 0m0.036s

to the time fri actually takes to run, without the overhead introduced by
RubyGems:
$ time ruby bin/fri -f plain String > /dev/null

real 0m0.088s
user 0m0.040s
sys 0m0.008s
[...]

Hm, the times are different on my box (P4 2.4 GHz, 1 G RAM,
Gentoo Linux, ruby 1.8.6 (2007-12-03 patchlevel 113) no threads):

$ time fri -f plain String > /dev/null

real 0m0.228s
user 0m0.152s
sys 0m0.019s

$ time ruby bin/fri -f plain String > /dev/null

real 0m0.147s
user 0m0.080s
sys 0m0.010s

fri installed with RubyGems is faster (0.228 / 0.385), fri from
the tarball ist slower (0.147 / 0.088) than or your box.

Is my box just slower than yours and is this the result of
improvements in the RubyGems-installed code?

Daniel

 

[Eric Hodel]

forgive the dumb question, but how can i make fastri work for ruby1.9?
thanks for fri/qri, mauricio. 'm very much dependent on it now

Are you really concerned about 1/10 of a second? (All times with hot
cache.)

$ time ri -f plain Array#fetch > /dev/null

real 0m0.589s
user 0m0.365s
sys 0m0.221s
$ time qri -f plain Array#fetch > /dev/null

real 0m0.110s
user 0m0.089s
sys 0m0.020s
$ time ri19 -f plain Array#fetch > /dev/null

real 0m0.208s
user 0m0.163s
sys 0m0.033s

 

[Mauricio Fernandez]

I haven't ported it to 1.9 yet, mainly because 1.9.0 is still too unstable.
It shouldn't be hard, though.

Are you really concerned about 1/10 of a second? (All times with hot
cache.)

$ time ri -f plain Array#fetch > /dev/null
real 0m0.589s [...]
$ time qri -f plain Array#fetch > /dev/null
real 0m0.110s [...]
$ time ri19 -f plain Array#fetch > /dev/null
real 0m0.208s

[...]

1/10th of a second does matter in some cases (e.g. if qri is being called by an
editor to display "live" documentation), but the behavior when the cache isn't
hot matters much more to me:

(worst case, cold cache)

$ time qri -f plain Array#fetch > /dev/null

real 0m3.690s
...
$ time ri -f plain Array#fetch > /dev/null

real 0m32.235s
...
$ time ri19 -f plain Array#fetch > /dev/null
[...]
[BUG] Segmentation fault
ruby 1.9.0 (2007-12-20) [i686-linux]

Aborted

real 0m6.141s


I rarely refer to the RI documentation nowadays, but I definitely don't want
to wait for several seconds (the average case observed in practice, with a
"temperate cache") when I do, to have ri tell me something like this:

$ ri -T collect_file
More than one method matched your request. You can refine
your search by asking for information on one of:

Test::Unit::Collector:ir#collect_file,
Test::Unit::Collector:ir#collect_file,
Test::Unit::Collector:ir#collect_file,
Test::Unit::Collector:ir#collect_file,
Test::Unit::Collector:ir#collect_file

Unfortunately, ri is so slow and buggy that many (me included) find it
unusable. FastRI is just an usable ri with some extra features (working
RubyGems support, full-text search, configurable lookup order, better search
strategies, etc.).

 

[Mauricio Fernandez]

]

fri installed with RubyGems is faster (0.228 / 0.385), fri from
the tarball ist slower (0.147 / 0.088) than or your box.

Is my box just slower than yours and is this the result of
improvements in the RubyGems-installed code?

The times you quoted were taken on a machine more or less comparable to yours.
RubyGems has improved recently, to the point that most people might not notice
the difference between running fri directly or through RubyGems. At this point
it only seems to matter if you're using fri in your editor to display info for
expansion candidates, etc.

 

[James Gray]

$ ri -T collect_file
More than one method matched your request. You can refine
your search by asking for information on one of:

Test::Unit::Collector:ir#collect_file,
Test::Unit::Collector:ir#collect_file,
Test::Unit::Collector:ir#collect_file,
Test::Unit::Collector:ir#collect_file,
Test::Unit::Collector:ir#collect_file

Unfortunately, ri is so slow and buggy that many (me included) find it
unusable. FastRI is just an usable ri with some extra features
(working
RubyGems support, full-text search, configurable lookup order,
better search
strategies, etc.).

Amen.

We had so many bug reports on TextMate's documentation command when we
used ri. We later switched to qri (part of FastRI) and added some
documentation about how to install it. Our bug reports are
practically gone now. When we do see one, we point them at the
install documentation and it solves their problem.

I really don't understand why we don't toss RI in the standard library
and replace it with the FastRI code. It's RI with more features, it's
faster, oh and it works. What's the downside exactly?

James Edward Gray II

 

[James Britt]

We had so many bug reports on TextMate's documentation command when we
used ri. We later switched to qri (part of FastRI) and added some
documentation about how to install it. Our bug reports are practically
gone now. When we do see one, we point them at the install
documentation and it solves their problem.

I really don't understand why we don't toss RI in the standard library
and replace it with the FastRI code. It's RI with more features, it's
faster, oh and it works. What's the downside exactly?

I'm curious if drbrain is planning any changes to rdoc that would change
how ri might behave differently.

One of my complaints with the current ri/rdoc situation is how elective
module mixins get pushed into various class docs by default. For
example, looking up String, I see that strings have a to_yaml method.
Except, of course, they don't unless you explicitly include the YAML module.

ri and rdoc are brain-dead with regard to dynamic class composition.

fri has the exact same behavior (since it's using the same broken
datastore as ri)

It gets worse as more gems are installed if they do any base class
alterations. (f)ri tells me that strings have a "_expand_ch" method. I
have no idea where that even comes from.




--
James Britt

"Serious engineering is only a few thousand years old. Our attempts at
deliberately producing very complex robust systems are immature at best."
- Gerald Jay Sussman

 

[Eric Hodel]

$ ri -T collect_file
More than one method matched your request. You can refine
your search by asking for information on one of:

Test::Unit::Collector:ir#collect_file,
Test::Unit::Collector:ir#collect_file,
Test::Unit::Collector:ir#collect_file,
Test::Unit::Collector:ir#collect_file,
Test::Unit::Collector:ir#collect_file

Unfortunately, ri is so slow and buggy that many (me included) find it
unusable. FastRI is just an usable ri with some extra features
(working
RubyGems support, full-text search, configurable lookup order,
better search
strategies, etc.).

Strange, I don't see a bug filed in the tracker matching this:

http://rubyforge.org/tracker/index.php?group_id=627&atid=2472

If "many" people are having this problem, I would expect one of them
to file a bug.

 

[Daniel Brumbaugh Keeney]

I really don't understand why we don't toss RI in the standard library
and replace it with the FastRI code. It's RI with more features, it's
faster, oh and it works. What's the downside exactly?

James Edward Gray II

I second. The first thing I tell new Ruby programmers is to use
FastRI. Ruby has a reputation for being slow, and RI really cements
that impression as people are initially learning the language. A
better implementation can be done, and has been done. It is time
FastRI was accepted as the standard.

Daniel Brumbaugh Keeney

 

[Mauricio Fernandez]

Amen.

We had so many bug reports on TextMate's documentation command when we
used ri. We later switched to qri (part of FastRI) and added some
documentation about how to install it. Our bug reports are
practically gone now. When we do see one, we point them at the
install documentation and it solves their problem.

I really don't understand why we don't toss RI in the standard library
and replace it with the FastRI code. It's RI with more features, it's
faster, oh and it works. What's the downside exactly?

It goes against the current trend of moving stuff out of the standard library
and distributing it as RubyGems packages.

Admittedly, the statu quo is not satisfactory: Ruby ships with a broken tool
that gives a very poor image and detracts new users from looking for something
better (one naturally expects the standard library to contain best-of-breed
components). There are three solutions to this:
(1) keep RI in the standard library and fix it. This amounts to rewriting
FastRI, though, and duplicating efforts this way would be stupid.
(2) remove the "ri" tool from the stdlib (*not* the RDoc/RI library parts, since
existing programs, including FastRI, depend on it) and distribute it as a
gem, or dump it altogether and let people install FastRI instead
(3) replace ri with FastRI's qri in the standard distribution.

(1) is silly. I favor (2) because it's the one that demands the least effort
from me and it's consistent with matz's intention to make the standard
distribution leaner.

 

[Mauricio Fernandez]

I'm curious if drbrain is planning any changes to rdoc that would change
how ri might behave differently.

One of my complaints with the current ri/rdoc situation is how elective
module mixins get pushed into various class docs by default. For
example, looking up String, I see that strings have a to_yaml method.
Except, of course, they don't unless you explicitly include the YAML module.

ri and rdoc are brain-dead with regard to dynamic class composition.

fri has the exact same behavior (since it's using the same broken
datastore as ri)

It gets worse as more gems are installed if they do any base class
alterations. (f)ri tells me that strings have a "_expand_ch" method. I
have no idea where that even comes from.

FastRI gets the documentation from ri's datastore(s) but does one thing
differently: it doesn't blindly merge the information from different sources,
it also remembers which gem a method/class came from. This information is not
exposed yet, however, but you can expect FastRI to allow you to restrict
lookups to a given set of gems (and maybe tell you where a method came from for
unscoped searches) eventually.

 

[Mauricio Fernandez]

Strange, I don't see a bug filed in the tracker matching this:

http://rubyforge.org/tracker/index.php?group_id=627&atid=2472

If "many" people are having this problem, I would expect one of them
to file a bug.

I attribute that to the "broken window effect". Once a piece of software is
buggy enough, people perceive it as essentially unmaintained and no longer
bother to file bug reports --- what for, if you believe they will go nowhere?
ri probably crossed that threshold a couple years ago. You might disagree on
this.

I don't see any gain in trying to fix the old ri anymore. There's nothing
particularly good or elegant in its code base, it's buggy and hard to refactor
because there are no tests, and where its functionality overlaps with
FastRI's, the latter does better.

Even though I haven't made any special effort to promote FastRI and I only
released a couple versions in the last year, several thousand people have
installed it. I believe most users were not drawn to it by the features it has
over ri such as full-text search or smarter lookup strategies. I rather think
that they experienced enough problems with ri to look for an alternative. ri
is included in Ruby's standard distribution, so you need a compelling reason
before you bother to locate and install an alternative. The situation can be
compared to that of IE and Firefox on MS Windows systems.

 

[James Gray]

It goes against the current trend of moving stuff out of the
standard library and distributing it as RubyGems packages.

I understand why we should do that when a standard library really
isn't needed in all cases. However, it doesn't get much more
important than documentation and what ships with Ruby is just broken.
I think it would be so much better to just replace it at this point.

In the 1.9 release we replaced CSV with FasterCSV. This seems about a
thousand times more important than that.

Admittedly, the statu quo is not satisfactory: Ruby ships with a
broken tool
that gives a very poor image and detracts new users from looking for
something
better (one naturally expects the standard library to contain best-
of-breed
components). There are three solutions to this:
(1) keep RI in the standard library and fix it. This amounts to
rewriting
FastRI, though, and duplicating efforts this way would be stupid.
(2) remove the "ri" tool from the stdlib (*not* the RDoc/RI library
parts, since
existing programs, including FastRI, depend on it) and distribute
it as a
gem, or dump it altogether and let people install FastRI instead
(3) replace ri with FastRI's qri in the standard distribution.

(1) is silly. I favor (2) because it's the one that demands the
least effort
from me and it's consistent with matz's intention to make the
standard
distribution leaner.

I favor (3), but I guess that really only works if you support it.

James Edward Gray II

 

[Trans]

I understand why we should do that when a standard library really
isn't needed in all cases. However, it doesn't get much more
important than documentation and what ships with Ruby is just broken.
I think it would be so much better to just replace it at this point.

+1

I just installed FastRI (been meaning to do it for a while). And I
have to say it is MUCH better. Very impressed. The only thing I need
now is a way to tell FastRI of alternate locations I want indexed.

T.

 

[James Britt]

FastRI gets the documentation from ri's datastore(s) but does one thing
differently: it doesn't blindly merge the information from different sources,
it also remembers which gem a method/class came from. This information is not
exposed yet, however, but you can expect FastRI to allow you to restrict
lookups to a given set of gems (and maybe tell you where a method came from for
unscoped searches) eventually.

Ah, good to hear. Thank you.

--
James Britt

www.ruby-doc.org - Ruby Help & Documentation
www.rubystuff.com - The Ruby Store for Ruby Stuff
www.risingtidesoftware.com - Wicked Cool Coding

 

[Trans]

Ah wait. Now I'm a bit confused. Does 'fastri-server -b' scan one's
installed libraries and indexes them (so rdoc is not directly needed),
or just grabs the ri files built by rdoc --ri-site/-system previously?
I was thinking it was the former, but now it's dawning on me that it's
probably the latter.

T.

 

[Trans]

Ah wait. Now I'm a bit confused. Does 'fastri-server -b' scan one's
installed libraries and indexes them (so rdoc is not directly needed),
or just grabs the ri files built by rdoc --ri-site/-system previously?
I was thinking it was the former, but now it's dawning on me that it's
probably the latter.

Please, please. Can I get some clarity on this?

T.