Literate testing?

[Massimiliano Mirra - bard]

I was documenting a class and found myself often copying unit tests
and pasting them as examples. I think I also read something about
tests embedded in doc strings in Python. So in the interest of DRY I
figured something like this could be desirable:

class Calculator
extend Literate

example
# Addition
add(2, 8) == 10
end

def add(a, b)
return a + b
end
end

I then came up with a stupid but working proof of concept:

class Calculator
extend Literate

example(__FILE__, __LINE__) do |calc|
# Addition
calc.add(2, 8) == 10
end

def add(a, b)
return a + b
end

example(__FILE__, __LINE__) do |calc|
# Integer division
calc.div(10, 2) == 5
end

def div(a, b)
return a / b + 1
end
end

When examples are run as tests and a test does not pass (as the div
example above), this is the output:

Test passed at line 17.
Test failed at line 27:
# Integer division
calc.div(10, 2) == 5

This is the rest:

module Literate
def examples
@examples ||= []
end

def example(file, line, &example)
@examples ||= []
@examples << [file, line, example]
end
end

Calculator.examples.each do |file, line, example|
c = Calculator.new
if example.call(c)
puts "Test passed at line #{line}."
else
puts "Test failed at line #{line}:"
puts File.readlines(file)[line, 2]
end
end

What do you think?



Massimiliano

 

[Joel VanderWerf]

class Calculator
extend Literate

example(__FILE__, __LINE__) do |calc|

...

Since you are passing a block, you can get the file and line from it, as in:


def get_fl(&bl)
eval "[__FILE__, __LINE__]", bl
end

class Foo
puts get_fl {}
puts [__FILE__, __LINE__]
end

 

[Joel VanderWerf]

I was documenting a class and found myself often copying unit tests
and pasting them as examples. I think I also read something about
tests embedded in doc strings in Python. So in the interest of DRY I
figured something like this could be desirable:

class Calculator
extend Literate

example
# Addition
add(2, 8) == 10
end

def add(a, b)
return a + b
end
end

I then came up with a stupid but working proof of concept:

class Calculator
extend Literate

example(__FILE__, __LINE__) do |calc|
# Addition
calc.add(2, 8) == 10
end

Are you planning to extract the example from the source code and
generate documentation files? This could be done using the file and line
information and some parsing or just good guesses about indentation.

 

[Mauricio Fernández]

I was documenting a class and found myself often copying unit tests
and pasting them as examples. I think I also read something about
tests embedded in doc strings in Python. So in the interest of DRY I
figured something like this could be desirable:

class Calculator
extend Literate

example
# Addition
add(2, 8) == 10
end

def add(a, b)
return a + b
end
end

I then came up with a stupid but working proof of concept:

class Calculator
extend Literate

example(__FILE__, __LINE__) do |calc|
# Addition
calc.add(2, 8) == 10
end

I believe you can make it look a bit better by using Kernel#caller
and Module#method_added; it would also be nice to generate test
cases for Test::Unit (for instance by defining the test_* methods in
MyClass::LiterateTests so that they can be included in a test suite
later).

Reminds me of Florian Groß' 'test extractor' [109712].
It works "the other way around" though, by extracting the tests from the
comments (similar to what you describe for Python).

 

[Massimiliano Mirra - bard]

I believe you can make it look a bit better by using Kernel#caller
and Module#method_added;

I confess I have absolutely no idea how.

it would also be nice to generate test
cases for Test::Unit (for instance by defining the test_* methods in
MyClass::LiterateTests so that they can be included in a test suite
later).

Nice. I tried, and the last call shows test_* methods get defined,
but Test::Unit says `No tests were specified.'.


require "test/unit"

module Literate
def method_missing(id, *args, &block)
if id.to_s =~ /example_/
name = Regexp.last_match.post_match
self::LiterateTests.senddefine_method, "test_#{name}".intern, &block)
else
super(id, *args, &block)
end
end
end

class Calculator
extend Literate

module LiterateTests
end

example_add do
assert @calc.add(2, 8) == 10
end

def add(a, b)
return a + b
end

example_div do
assert @calc.div(10, 2) == 5
end

def div(a, b)
return a / b + 1
end
end

class TestCalculator < Test::Unit::TestCase
include Calculator::LiterateTests

def setup
@calc = Calculator.new
end
end

p TestCalculator.public_instance_methods

 

[Massimiliano Mirra - bard]

Are you planning to extract the example from the source code and
generate documentation files?

I'd rather say `hoping', but yes, that would be the purpose.

This could be done using the file and line
information and some parsing or just good guesses about indentation.

Yep. Or by hacking rdoc a bit.

Thanks for the bit on eval.

Massimiliano

 

[Mauricio Fernández]

I confess I have absolutely no idea how.


Nice. I tried, and the last call shows test_* methods get defined,
but Test::Unit says `No tests were specified.'.

I think you were bitten by the IMHO bug explained in
http://blade.nagaokaut.ac.jp/cgi-bin/scat.rb/ruby/ruby-core/2829