[ANN] rcov 0.8.0: new output modes, fix for RSpec woes, superior Emacs integration

[Mauricio Fernandez]

Source code, additional information, screenshots... available at
http://eigenclass.org/hiki.rb?rcov
Release information:
http://eigenclass.org/hiki.rb?rcov-0.8.0
Sample fully cross-referenced report at
http://eigenclass.org/static/rcov-sample-report-full-crossref/

This release includes two RubyGems packages: a binary one for Win32 and a
platform-independent one for all those with a compiler, (or a lot of patience,
if willing to run rcov in pure-Ruby mode), so
gem install rcov
should work once the packages have propagated to the mirror network.

Overview
========
rcov is a code coverage tool for Ruby. It is commonly used for viewing overall
test coverage of target code. It features:
* fast execution: 20-300 times faster than previous tools
* multiple analysis modes: standard, bogo-profile, "intentional testing",
dependency analysis...
* detection of uncovered code introduced since the last run ("differential
code coverage")
* fairly accurate coverage information through code linkage inference using
simple heuristics
* cross-referenced XHTML and several kinds of text reports
* support for easy automation with Rake and Rant
* colorblind-friendliness

What's new in 0.8.0
===================
Features
--------
* --annotate mode, which dumps annotated source code which can be used to
follow the control flow (very useful when reading third-party code)
* --gcc option to display uncovered lines in GCC error format
* superior Emacs support: running rcov, jumping to uncovered code, navigate
through cross-referenced annotated code
* --[no-]validator-links

Bugfixes
--------
* differential code coverage reports work with filenames containing special
characters
* fixed recent segfaults happening with rspec
* more care name mangling

Minor enhancements
------------------
* relevant summary values are identified using separate CSS classes
(microformat-style)

Special thanks go to
====================
rubikitch
* implemented --annotate mode
* implemented --gcc option
* superior emacs support
* testing, refactoring...
* many other things, see darcs changes

Zed A. Shaw
* reported and fixed segfault triggered by rspec

Jan Svitok
* reported typo in rcovtask.rb's RDoc

Thomas Leitner
* fixed the bug which broke differential reports for filenames with special characters


How do I use it?
================
In the common scenario, your tests are under test/ and the target code
(whose coverage you want) is in lib/. In that case, all you have to do is
use rcov to run the tests (instead of testrb), and a number of XHTML files
with the code coverage information will be generated, e.g.

rcov -Ilib test/*.rb

will execute all the .rb files under test/ and generate the code coverage
report for the target code (i.e. for the files in lib/) under coverage/. The
target code needs not be under lib/; rcov will detect is as long as it is
require()d by the tests. rcov is smart enough to ignore "uninteresting"
files: the tests themselves, files installed in Ruby's standard locations,
etc. See rcov --help for the list of regexps rcov matches filenames
against.

rcov can also be used from Rake; see README.rake or the RDoc documentation
for more information.

rcov can output information in several formats, and perform different kinds
of analyses in addition to plain code coverage. See rcov --help for a
description of the available options.

Sample output
=============

See http://eigenclass.org/hiki.rb?rcov (once again) for screenshots.

The text report (also used by default in RcovTasks) resembles


+-----------------------------------------------------+-------+-------+--------+
| File | Lines | LOC | COV |
+-----------------------------------------------------+-------+-------+--------+
|lib/rcov.rb | 572 | 358 | 91.3% |
+-----------------------------------------------------+-------+-------+--------+
|Total | 572 | 358 | 91.3% |
+-----------------------------------------------------+-------+-------+--------+
91.3% 1 file(s) 572 Lines 358 LOC



The (undecorated) textual output with execution count information looks like this:

$ rcov --no-html --text-counts b.rb
================================================================================
./b.rb
================================================================================
| 2
a, b, c = (1..3).to_a | 2
10.times do | 1
a += 1 | 10
20.times do |i| | 10
b += i | 200
b.times do | 200
c += (j = (b-a).abs) > 0 ? j : 0 | 738800
end | 0
end | 0
end | 0


rcov can detect when you've added code that was not covered by your unit
tests:

$ rcov --text-coverage-diff --no-color test/*.rb
Started
.......................................
Finished in 1.163085 seconds.

39 tests, 415 assertions, 0 failures, 0 errors

================================================================================
!!!!! Uncovered code introduced in lib/rcov.rb

### lib/rcov.rb:207

def precompute_coverage(comments_run_by_default = true)
changed = false
lastidx = lines.size - 1
if (!is_code?(lastidx) || /^__END__$/ =~ @lines[-1]) && !@coverage[lastidx]
!! # mark the last block of comments
!! @coverage[lastidx] ||= :inferred
!! (lastidx-1).downto(0) do |i|
!! break if is_code?(i)
!! @coverage ||= :inferred
!! end
!! end
(0...lines.size).each do |i|
next if @coverage
line = @lines



License
-------
rcov is released under the terms of Ruby's license.
rcov includes xx 0.1.0, which is subject to the following conditions:

ePark Labs Public License version 1
Copyright (c) 2005, ePark Labs, Inc. and contributors
All rights reserved.

Redistribution and use in source and binary forms, with or without modification,
are permitted provided that the following conditions are met:

1. Redistributions of source code must retain the above copyright notice, this
list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright notice,
this list of conditions and the following disclaimer in the documentation
and/or other materials provided with the distribution.
3. Neither the name of ePark Labs nor the names of its contributors may be
used to endorse or promote products derived from this software without
specific prior written permission.

THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND
ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE LIABLE FOR
ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES
(INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES;
LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON
ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.


--
Mauricio Fernandez - http://eigenclass.org - singular Ruby
** Latest postings **
More logical programming in Ruby: solving Einstein's riddle (Zebra puzzle)
http://eigenclass.org/hiki.rb?einstein-riddle-solved-in-ruby
Bounded space memoization, another new twist
http://eigenclass.org/hiki.rb?bounded-space-memoization

 

[Jan Friedrich]

Thanks for new version. But I get an error:

rcov -v

rcov 0.8.0 2007-02-28

rcov -Ilib test/test_read.rb

Loaded suite /usr/bin/rcov
<snip/>
Finished in 2.204768 seconds.

7 tests, 25 assertions, 0 failures, 0 errors
(eval):229:in `method_missing': no id given (ArgumentError)
from (eval):229:in `method_missing'
from
/usr/lib/ruby/gems/1.8/gems/rcov-0.8.0.1/lib/rcov/report.rb:36:in
`initialize'
from
/usr/lib/ruby/gems/1.8/gems/rcov-0.8.0.1/lib/rcov/report.rb:61:in
`mangle_filename'
from
/usr/lib/ruby/gems/1.8/gems/rcov-0.8.0.1/lib/rcov/report.rb:691:in
`format_overview'
from (eval):104:in `create'
from (eval):80:in `tracking_additions'
from (eval):103:in `create'
from (eval):493:in `td_'
... 48 levels...
from /usr/lib/ruby/gems/1.8/gems/rcov-0.8.0.1/lib/rcov.rb:628:in
`dump_coverage_info'
from /usr/lib/ruby/gems/1.8/gems/rcov-0.8.0.1/lib/rcov.rb:628:in
`dump_coverage_info'
from /usr/lib/ruby/gems/1.8/gems/rcov-0.8.0.1/bin/rcov:405
from /usr/lib/ruby/1.8/test/unit.rb:285

regards
Jan

 

[Mauricio Fernandez]

Thanks for new version. But I get an error: [...]
7 tests, 25 assertions, 0 failures, 0 errors
(eval):229:in `method_missing': no id given (ArgumentError)
from (eval):229:in `method_missing'
from
/usr/lib/ruby/gems/1.8/gems/rcov-0.8.0.1/lib/rcov/report.rb:36:in
`initialize'
from

This was an interesting bug report, I've found a possible bug in Ruby
(new behavior introduced in 1.8.5) thanks to it.

Here's the part of the story that relates to your problem; the Ruby build
(more precisely the stdlib) you're using predates this:

Sat Jun 17 14:53:32 2006 Tanaka Akira <[email protected]>

* lib/pathname.rb: backport from 1.9.
(Kernel#Pathname): new method.

(iow. you're probably using ruby < 1.8.5).

There's a chance rcov will work if you apply this patch:

Wed Feb 28 15:29:58 CET 2007 Mauricio Fernandez <[email protected]>
* lib/rcov/report.rb: use Pathname.new instead of Pathname for compatibility with Ruby < 1.8.5.
diff -rN -u old-head/lib/rcov/report.rb new-head/lib/rcov/report.rb
--- old-head/lib/rcov/report.rb 2007-02-28 15:30:47.000000000 +0100
+++ new-head/lib/rcov/report.rb 2007-02-28 15:30:47.000000000 +0100
@@ -33,7 +33,7 @@
@callsite_index = nil

@mangle_filename = Hash.new{|h,base|
- h[base] = Pathname(base).cleanpath.to_s.gsub(%r{^\w:[/\\]}, "").gsub(/\./, "_").gsub(/[\\\/]/, "-") + ".html"
+ h[base] = Pathname.new(base).cleanpath.to_s.gsub(%r{^\w:[/\\]}, "").gsub(/\./, "_").gsub(/[\\\/]/, "-") + ".html"
}
end


I'd be grateful if you could run rcov after applying the above patch and tell
me if it works. If so, 0.8.1 will follow shortly.

The unit tests seem to indicate that cross-referenced reports might not work
under ruby <1.8.5, but the basic functionality (code coverage) does. I'll
document that.

--
Mauricio Fernandez - http://eigenclass.org - singular Ruby
** Latest postings **
More logical programming in Ruby: solving Einstein's riddle (Zebra puzzle)
http://eigenclass.org/hiki.rb?einstein-riddle-solved-in-ruby
Bounded space memoization, another new twist
http://eigenclass.org/hiki.rb?bounded-space-memoization

 

[David Chelimsky]

It looks like rcov won't cover an empty rescue clause. I posted a
detailed bug report on http://eigenclass.org/hiki.rb?rcov-0.8.0.

Awesome tool in general, however. THANKS!!!!!!!

Thanks for new version. But I get an error: [...]
7 tests, 25 assertions, 0 failures, 0 errors
(eval):229:in `method_missing': no id given (ArgumentError)
from (eval):229:in `method_missing'
from
/usr/lib/ruby/gems/1.8/gems/rcov-0.8.0.1/lib/rcov/report.rb:36:in
`initialize'
from

This was an interesting bug report, I've found a possible bug in Ruby
(new behavior introduced in 1.8.5) thanks to it.

Here's the part of the story that relates to your problem; the Ruby build
(more precisely the stdlib) you're using predates this:

Sat Jun 17 14:53:32 2006 Tanaka Akira <[email protected]>

* lib/pathname.rb: backport from 1.9.
(Kernel#Pathname): new method.

(iow. you're probably using ruby < 1.8.5).

There's a chance rcov will work if you apply this patch:

Wed Feb 28 15:29:58 CET 2007 Mauricio Fernandez <[email protected]>
* lib/rcov/report.rb: use Pathname.new instead of Pathname for compatibility with Ruby < 1.8.5.
diff -rN -u old-head/lib/rcov/report.rb new-head/lib/rcov/report.rb
--- old-head/lib/rcov/report.rb 2007-02-28 15:30:47.000000000 +0100
+++ new-head/lib/rcov/report.rb 2007-02-28 15:30:47.000000000 +0100
@@ -33,7 +33,7 @@
@callsite_index = nil

@mangle_filename = Hash.new{|h,base|
- h[base] = Pathname(base).cleanpath.to_s.gsub(%r{^\w:[/\\]}, "").gsub(/\./, "_").gsub(/[\\\/]/, "-") + ".html"
+ h[base] = Pathname.new(base).cleanpath.to_s.gsub(%r{^\w:[/\\]}, "").gsub(/\./, "_").gsub(/[\\\/]/, "-") + ".html"
}
end


I'd be grateful if you could run rcov after applying the above patch and tell
me if it works. If so, 0.8.1 will follow shortly.

The unit tests seem to indicate that cross-referenced reports might not work
under ruby <1.8.5, but the basic functionality (code coverage) does. I'll
document that.

--
Mauricio Fernandez - http://eigenclass.org - singular Ruby
** Latest postings **
More logical programming in Ruby: solving Einstein's riddle (Zebra puzzle)
http://eigenclass.org/hiki.rb?einstein-riddle-solved-in-ruby
Bounded space memoization, another new twist
http://eigenclass.org/hiki.rb?bounded-space-memoization

 

[Jan Friedrich]

This was an interesting bug report, I've found a possible bug in Ruby
(new behavior introduced in 1.8.5) thanks to it.

Yes:
% ruby -v
ruby 1.8.4 (2005-12-24) [i486-linux]

I'd be grateful if you could run rcov after applying the above patch and
tell
me if it works. If so, 0.8.1 will follow shortly.

Yes.

regards
Jan

 

[Mauricio Fernandez]

This was an interesting bug report, I've found a possible bug in Ruby
(new behavior introduced in 1.8.5) thanks to it.

Yes:
% ruby -v
ruby 1.8.4 (2005-12-24) [i486-linux]

I'd be grateful if you could run rcov after applying the above patch and
tell me if it works. If so, 0.8.1 will follow shortly.

Yes.

I finally decided to leave 0.8.1 for later, but I applied the patch to
0.8.0.2, available in tarball and gem formats.

--
Mauricio Fernandez - http://eigenclass.org - singular Ruby
** Latest postings **
More logical programming in Ruby: solving Einstein's riddle (Zebra puzzle)
http://eigenclass.org/hiki.rb?einstein-riddle-solved-in-ruby
Bounded space memoization, another new twist
http://eigenclass.org/hiki.rb?bounded-space-memoization

 

[Faisal N Jawdat]

I finally decided to leave 0.8.1 for later, but I applied the patch to
0.8.0.2, available in tarball and gem formats.

What does that do to the version on RubyForge?

-faisal

 

[Mauricio Fernandez]

What does that do to the version on RubyForge?

The version on Rubyforge is 0.8.0.2; both the gem and the tarball for 0.8.0.2
are available in the file area of the 'rcov' project. If you're using 1.8.5,
0.8.0 and 0.8.0.2 will behave identically. The only differences are the patch
I posted before and a similar modification in the same file that makes it work
with ruby <1.8.5.

--
Mauricio Fernandez - http://eigenclass.org - singular Ruby
** Latest postings **
More logical programming in Ruby: solving Einstein's riddle (Zebra puzzle)
http://eigenclass.org/hiki.rb?einstein-riddle-solved-in-ruby
Bounded space memoization, another new twist
http://eigenclass.org/hiki.rb?bounded-space-memoization