test/unit autorun fails to set exit status on failure

[John Carter]

Consider the elementary unit test that fails...

=try.rb===============================================================
require 'test/unit'

class TC_OurTest < Test::Unit::TestCase
def test_fail
assert(false, 'Assertion was false.')
end
end
======================================================================
What is its exit status?

ruby -w try.rb
echo $?
0

If I look in
/usr/local/lib/ruby/1.8/test/unit.rb
I see at the end the following code...

----------------------------------------------------------------------
at_exit do
unless $! || Test::Unit.run?
exit Test::Unit::AutoRunner.run
end
end
----------------------------------------------------------------------

Now try this chunk of code...

==atexittoolate.rb====================================================

at_exit do
exit( 1)
end

======================================================================
ruby -w atexittoolate.rb;echo $?
0

If I change that to...
==atexittoolate.rb====================================================

at_exit do
exit( 1)
end
exit(2)
======================================================================
ruby -w atexittoolate.rb;echo $?
2


Bottom line: at_exit seems to be too late (Under ruby-1.8.6 linux
anyway) to set the exit status.


John Carter Phone : (64)(3) 358 6639
Tait Electronics Fax : (64)(3) 359 4632
PO Box 1645 Christchurch Email : (e-mail address removed)
New Zealand

 

[Tim Pease]

Consider the elementary unit test that fails...

Bottom line: at_exit seems to be too late (Under ruby-1.8.6 linux
anyway) to set the exit status.

Hmmm ....

$ ruby -v
ruby 1.8.5 (2006-12-25 patchlevel 12) [i386-cygwin]

$ cat tmp.rb
at_exit do
puts "Calling exit handler"
exit 2
end

exit

$ ruby tmp.rb
Calling exit handler

$ echo $?
2


There is nothing in eval.c that prevents the at_exit procs from
calling exit again with a new status code. When the at_exit proc is
called, it is first popped off the stack and then run. When it calls
exit at the end, the whole process starts again, and the next at_exit
proc is called which, too, can call exit with a new status code.

The status passed to the last exit call is the one that is returned.

Is there another at_exit proc that is registered before the Test::Unit
at_ext (and therefore is called afterwards) that is setting the exit
code to zero?

Now, this is all from the 1.8.5 source code. Perhaps something changed
in 1.8.6 -- but that seems a large-ish change.

Blessings,
TwP

 

[Gregory Brown]

Now, this is all from the 1.8.5 source code. Perhaps something changed
in 1.8.6 -- but that seems a large-ish change.

I think this is a bug in 1.8.6:
http://rubyforge.org/tracker/?func=detail&atid=1698&aid=9300&group_id=426

 

[Tim Pease]

I think this is a bug in 1.8.6:
http://rubyforge.org/tracker/?func=detail&atid=1698&aid=9300&group_id=426

Thanks for the link. Nobu checked in a fix for this bug. Should be
available in the next maintenance release.

TwP