Fastri-server problem?

[bbiker]

I have downloaded and install fastri from the tarball.

Works fine and I qri eventhough it is slower than fri ... reason I
don't to keep the fastri-server window reduced or otherwise.

Fastri-server does seem to have problem with some gem packages.

Here's the scenario

I gem install the watir then I execute fastri-server -B so that fri
and qri know about watir.

c:\ruby>gem install watir
Bulk updating Gem source index for: http://gems.rubyforge.org
Successfully installed watir-1.4.1
Installing ri documentation for watir-1.4.1...
Installing RDoc documentation for watir-1.4.1...

c:\ruby>fastri-server -B
-- snip --
Indexing RI docs for tidy version 1.1.2.
Indexing RI docs for watir version 1.4.1.
-- snip --
C:/ruby/bin/fastri-server.rb:71:

Now the above result does not look good but is not alarming

Continuing the scenario ....

I uninstall another gem and the run fastri-server -B

c:\ruby>gem uninstall tidy
Successfully uninstalled tidy version 1.1.2

c:\ruby>fastri-server -B
-- snip --
Indexing RI docs for test-spec version 0.3.0.
Indexing RI docs for watir version 1.4.1.
-- snip --

C:/ruby/bin/fastri-server.rb:71: [BUG] Segmentation fault
ruby 1.8.5 (2006-12-25) [i386-mswin32]


This application has requested the Runtime to terminate it in an
unusual way.
Please contact the application's support team for more information.

Now that is alarming response!!!!

If I now uninstall watir then fastri-server -B completes without any
problem.


Please note that watir is not the only gem that fastri-ri has problem
with...right now I cannot remember
which gem.

 

[Mauricio Fernandez]

I have downloaded and install fastri from the tarball.

Works fine and I qri eventhough it is slower than fri ... reason I
don't to keep the fastri-server window reduced or otherwise.

You can also run fastri-server as a win32 service (IIRC the gem is named
"ruby-services").

Fastri-server does seem to have problem with some gem packages.

Here's the scenario

I gem install the watir then I execute fastri-server -B so that fri
and qri know about watir.

Note that fastri-server -B is used to build the full-text index (which will
be used when you do fri -S someterm --- qri behaves identically to fri in
this case), so you probably meant fastri-server -b.

c:\ruby>fastri-server -B
-- snip --
Indexing RI docs for tidy version 1.1.2.
Indexing RI docs for watir version 1.4.1.
-- snip --
C:/ruby/bin/fastri-server.rb:71:

Now the above result does not look good but is not alarming

Could you post the full backtrace info?

c:\ruby>fastri-server -B
-- snip --
Indexing RI docs for test-spec version 0.3.0.
Indexing RI docs for watir version 1.4.1.
-- snip --

C:/ruby/bin/fastri-server.rb:71: [BUG] Segmentation fault
ruby 1.8.5 (2006-12-25) [i386-mswin32] [...]

Now that is alarming response!!!!

If I now uninstall watir then fastri-server -B completes without any
problem.

fastri-server.rb:71 is

70 Dir["#{path}/**/*.yaml"].each do |yamlfile|
71 yaml = File.read(yamlfile)
72 begin
73 data = YAML.load(yaml.gsub(/ \!.*/, ''))

FastRI is pure-Ruby, so the interpreter should never segfault. As you can see,
when generating a full-text index FastRI will unserialize lots of YAML data
after some massaging. I believe the segfault is caused by a bug in syck.

I wasn't able to reproduce the problem, but if you can isolate the yamlfile
causing the segfault, ruby-core could help...

 

[bbiker]

You can also run fastri-server as a win32 service (IIRC the gem is named
"ruby-services").

I had download the ruby-services gem as a tarball ... fastrictl used
to work fine before
I upgraded to: ruby 1.8.5 (2006-12-25 patchlevel 12) [i386-mswin32]

I do not have any problem with gemctl ... it working fine with latest
ruby.
I have not tried webrickctl ... since I don't need for now.

First attempt to start after bootup:

C:\Documents and Settings\Owner>fastrictl -s
C:/ruby/bin/fastrictl.rb:68:in `start': The service did not respond to
the start or c
ontrol request in a timely fashion. (Win32::ServiceError)
from C:/ruby/bin/fastrictl.rb:68

Subsequent attempts fastrictl gets itself in an endless loop

C:\Documents and Settings\Owner>fastrictl -s
One moment, start pending
One moment, stopped
One moment, stopped
One moment, stopped
One moment, stopped
One moment, stopped
C:/ruby/bin/fastrictl.rb:75:in `sleep': Interrupt
from C:/ruby/bin/fastrictl.rb:75

Note that fastri-server -B is used to build the full-text index (which will
be used when you do fri -S someterm --- qri behaves identically to fri in
this case), so you probably meant fastri-server -b.

No. I usually do "fastri-server -B" after I install new gems.

For where I stand (ignorance), I thought that -B was more thorough job
than -b although
it takes longer to do the indexing. It also appears to rebuild the
index.

It has not been made clear when one should one over the over

I do use "fri -S someterm" or "qri -S someterm" ,,, just "fri
someterm" or "qri someterm"

(just did "qri -S" ... now I know ... will investigate to learn the
difference between
"qri someterm" and "qri -S someterm"

c:\ruby>fastri-server -B
-- snip --
Indexing RI docs for tidy version 1.1.2.
Indexing RI docs for watir version 1.4.1.
-- snip --
C:/ruby/bin/fastri-server.rb:71:

Now the above result does not look good but is not alarming

Could you post the full backtrace info?

c:\ruby>fastri-server -B
-- snip --
Indexing RI docs for test-spec version 0.3.0.
Indexing RI docs for watir version 1.4.1.
-- snip --

C:/ruby/bin/fastri-server.rb:71: [BUG] Segmentation fault
ruby 1.8.5 (2006-12-25) [i386-mswin32] [...]

Now that is alarming response!!!!

If I now uninstall watir then fastri-server -B completes without any
problem.

Correction on the above. I reinstalled tidy and I now get the same
error.

fastri-server.rb:71 is

70 Dir["#{path}/**/*.yaml"].each do |yamlfile|
71 yaml = File.read(yamlfile)
72 begin
73 data = YAML.load(yaml.gsub(/ \!.*/, ''))

FastRI is pure-Ruby, so the interpreter should never segfault. As you can see,
when generating a full-text index FastRI will unserialize lots of YAML data
after some massaging. I believe the segfault is caused by a bug in syck.

I wasn't able to reproduce the problem, but if you can isolate the yamlfile
causing the segfault, ruby-core could help...

I Dir["#{path}/**/*.yaml"].each do |yamlfile|
puts "yamlfile #{yamlfile}" <-- added
this line to fastri-server.rb

ran fastri-server -B and guess what? it did not fail!!!

okay ... reinstall watir and reran fastri-server -B

obviously snipped a lot ---

yamlfile c:/ruby/share/ri/1.8/system/Logger/LogDevice/new-c.yaml
yamlfile c:/ruby/share/ri/1.8/system/Logger/LogDevice/open_logfile-
i.yaml
yamlfile c:/ruby/share/ri/1.8/system/Logger/LogDevice/
previous_period_end-i.yaml
C:/ruby/bin/fastri-server.rb:72:
C:\Documents and Settings\Owner>

will now uninstall tidy and see if we get a segmentation fault.

no failure. ps this is the last file processed

yamlfile c:/ruby/lib/ruby/gems/1.8/doc/windows-pr-0.6.2/ri/Windows/
Window/GetWindowRect-i.yaml

uninstall watir -- no failure with fastri-server -B

reinstall tidy -- no failure with fastri-server -B

strange but at the start of my complying with your request .. I kept
getting the segmentation fault.

NOTE BUENE

I have just removed the "puts yamlfile #{yamlfile}" line from fastri-
server.rb

fastri-server -B terminates with:

C:/ruby/bin/fastri-server.rb:71: [BUG] Segmentation fault
ruby 1.8.5 (2006-12-25) [i386-mswin32]


This application has requested the Runtime to terminate it in an
unusual way.
Please contact the application's support team for more information.

 

[Mauricio Fernandez]

First attempt to start after bootup:

C:\Documents and Settings\Owner>fastrictl -s
C:/ruby/bin/fastrictl.rb:68:in `start': The service did not respond to
the start or c
ontrol request in a timely fashion. (Win32::ServiceError)
from C:/ruby/bin/fastrictl.rb:68

Subsequent attempts fastrictl gets itself in an endless loop

C:\Documents and Settings\Owner>fastrictl -s
One moment, start pending
One moment, stopped
One moment, stopped
One moment, stopped
One moment, stopped
One moment, stopped
C:/ruby/bin/fastrictl.rb:75:in `sleep': Interrupt
from C:/ruby/bin/fastrictl.rb:75

I don't really know anything about ruby-services. Dan?

No. I usually do "fastri-server -B" after I install new gems.

For where I stand (ignorance), I thought that -B was more thorough job than
-b although it takes longer to do the indexing. It also appears to rebuild
the index.

It has not been made clear when one should one over the over

I do use "fri -S someterm" or "qri -S someterm" ,,, just "fri
someterm" or "qri someterm"

(just did "qri -S" ... now I know ... will investigate to learn the
difference between "qri someterm" and "qri -S someterm"

qri/fri -S performs full-text searching against the index built with
fastri-server -B, so the query term is looked for in the whole ri
documentation. Therefore, qri -S test will return all the method/class
descriptions containing "test". You can think of it as a Fgrep over all your
RDocs, the main difference being that it'll find the docs in a few
milliseconds... (also, qri -S test Rails will only return documents whose
text matches both 'test' _and_ 'Rails').

fri/qri without -S try to match "document titles" (class/method names)
according to the search strategy specified with -O (see fri -h for details).

If you know what you're looking for, there's no need for -S; if you don't, -S
can be useful. For example, if you're having endianness problems and don't
know what to use,

$ fri -S byte -T order
7 hits
========================================================================
0.400320384512718
Found in system Array#pack

... v | Short, little-endian byte order
... E | Double-precision float, little-endian byte order
... | Double-precision float, network (big-endian) byte order
... G | Double-precision float, network (big-endian) byte
[...]

========================================================================
0.377964473009227
Found in facets 1.4.5 BinaryReader#byte_order

... default is native byte-order
[...]
========================================================================
0.235294117647059
Found in system String#unpack

... | | a float in little-endian byte order
... | | a double in little-endian byte order
... | | short in little-endian byte order
... | | short in network byte order
... | | float in network byte order
... | | long in network byte order
... | | a double in network byte order
... | | long in little-endian byte order
... | | short in native byte order
... | | in native byte order
[...]

I have just removed the "puts yamlfile #{yamlfile}" line from fastri-
server.rb

fastri-server -B terminates with:

C:/ruby/bin/fastri-server.rb:71: [BUG] Segmentation fault
ruby 1.8.5 (2006-12-25) [i386-mswin32]

Alright, we have pure-Ruby code and an intermittent bug which disappears when
you add or remove code that shouldn't change anything. For a long time, I've
blamed syck when this happened to me, and have often been right :-| The
standard procedure to follow is finding a way to reproduce it, submitting to
ruby-core and hoping somebody will find the culprit. I'm trying to have ruby
crash locally but it doesn't want to...

 

[Mauricio Fernandez]

fwiw, mine here is:

C:\family\ruby>fastri-server -b [...]
Indexed:
* 10240 methods
* 2046 classes/modules
Needed 3.984 seconds

ok, i think that's success alright.
Now,

C:\family\ruby>fastri-server -B
Indexing RI docs for ZenTest version 3.4.3.
<snip>....
Indexing RI docs for win32-sound version 0.4.0.
Indexing RI docs for win32console-1.0.8-i386 version mswin32.
Indexing RI docs for windows-pr version 0.6.2.
c:/ruby/lib/ruby/gems/1.8/gems/fastri-0.3.0.1/bin/fastri-server:70:
C:\family\ruby>

is that a success or what?

Don't really know. Do you get a new .fastri-fulltext directory with
full_text.dat and suffixes.dat in HOME?

i think the full rebuild is buggy, how can we debug it using just ruby?

It'd seem Ruby is crashing so hard it doesn't even give you a backtrace, but I
can't really tell. Maybe adding a rescue Exception; puts $!.backtrace to the
make_full_text_index method would tell us something, or it could make the
heisenbug go away.

Just realized that there could be a simple explanation to this: sometimes ruby
cannot detect stack overflows correctly and you get a segfault. However, ruby
doesn't use a recursive qsort (the most obvious possible culprit)...
If this is the reason, the problem should consistently fail to manifest itself
when you enlarge the stack (I don't know how to do that on win32, though).

 

[bbiker]

fwiw, mine here is:

C:\family\ruby>fastri-server -b [...]
Indexed:
* 10240 methods
* 2046 classes/modules
Needed 3.984 seconds

ok, i think that's success alright.
Now,

C:\family\ruby>fastri-server -B
Indexing RI docs for ZenTest version 3.4.3.
<snip>....
Indexing RI docs for win32-sound version 0.4.0.
Indexing RI docs for win32console-1.0.8-i386 version mswin32.
Indexing RI docs for windows-pr version 0.6.2.
c:/ruby/lib/ruby/gems/1.8/gems/fastri-0.3.0.1/bin/fastri-server:70:
C:\family\ruby>

is that a success or what?

Don't really know. Do you get a new .fastri-fulltext directory with
full_text.dat and suffixes.dat in HOME?

- Show quoted text -

on my WindowsXP machine
the ,fastri-fulltext folder is in C:\Documents and Settings\Owner

 

[Mauricio Fernandez]

C:\family\ruby>fastri-server -b [...]
Indexed:
* 10240 methods
* 2046 classes/modules
Needed 3.984 seconds

ok, i think that's success alright.
Now,

C:\family\ruby>fastri-server -B
Indexing RI docs for ZenTest version 3.4.3.
<snip>....
Indexing RI docs for win32-sound version 0.4.0.
Indexing RI docs for win32console-1.0.8-i386 version mswin32.
Indexing RI docs for windows-pr version 0.6.2.
c:/ruby/lib/ruby/gems/1.8/gems/fastri-0.3.0.1/bin/fastri-server:70:
C:\family\ruby>

is that a success or what?

Don't really know. Do you get a new .fastri-fulltext directory with
full_text.dat and suffixes.dat in HOME?

- Show quoted text -

on my WindowsXP machine
the ,fastri-fulltext folder is in C:\Documents and Settings\Owner

If fri -S test works after that then we can disregard that puzzling message.
As I said, I believe the crashes are due to a bug in either syck or the
mswin32 build. The former seemed more likely, but I haven't been able to
reproduce it :-|