Test::Unit feature request - regexp pattern of tests to run

[Simon Strandgaard]

It would be awesome if I could say

prompt> ruby test_me.rb --name "test_.*something.*"

so that all tests with 'something' in their method name gets invoked.

However on ruby 1.8.1.. only zero tests are being invoked.
So I guess it isn't supported.


Such feature would be awesome.

Thanks in advance.

 

[Alexander Kellett]

It would be awesome if I could say
prompt> ruby test_me.rb --name "test_.*something.*"

i use the following snippet of code
but obviously it would be much nicer
to have it builtin -

require "test/unit"

class TC_MyTest < Test::Unit::TestCase

DO_TEST = :test_undo_redo

def test_match
end

def test_undo_redo
end

public_instance_methods.each {
|meth|
next if meth !~ /^test.*/ or meth.to_sym == DO_TEST
remove_method meth.to_sym
} if defined? DO_TEST

end

require 'test/unit/ui/console/testrunner'
Test::Unit::UI::Console::TestRunner.run(TC_MyTest)

Alex

 

[Simon Strandgaard]

It would be awesome if I could say

prompt> ruby test_me.rb --name "test_.*something.*"

so that all tests with 'something' in their method name gets invoked.

However on ruby 1.8.1.. only zero tests are being invoked.
So I guess it isn't supported.


Such feature would be awesome.

Another related feature I would like to see in Test::Unit..
being able to execute tests which doesn't start with 'test' prefix.

For instance I have certain tests which im working on..
those I name #xtest_something..

However if I try to execute the test via the cli.. then it
just rejects it.. What im asking fore.. don't reject test names
supplied via the cli .


bash-2.05b$ ruby test_buffer.rb -n xtest_bracket_matcher13
Loaded suite test_buffer
Started

Finished in 0.001408 seconds.

0 tests, 0 assertions, 0 failures, 0 errors
bash-2.05b$ grep xtest test_buffer.rb
def xtest_string_paren_forward5
def xtest_bracket_matcher13
bash-2.05b$


Thanks in advance.

 

[Alexander Kellett]

Another related feature I would like to see in Test::Unit..
being able to execute tests which doesn't start with 'test' prefix.

yup. nod nod for this one also.

Alex

 

[David Heinemeier Hansson]

prompt> ruby test_me.rb --name "test_.*something.*"

so that all tests with 'something' in their method name gets invoked.

This is already in. You just need to pass a regular expression instead
of a string:

ruby test_me.rb -n /test_.*something.*/

Very useful...
--
David Heinemeier Hansson,
http://www.rubyonrails.org/ -- Web-application framework for Ruby
http://www.instiki.org/ -- A No-Step-Three Wiki in Ruby
http://www.basecamphq.com/ -- Web-based Project Management
http://www.loudthinking.com/ -- Broadcasting Brain
http://www.nextangle.com/ -- Development & Consulting Services

 

[Simon Strandgaard]

This is already in. You just need to pass a regular expression instead
of a string:

ruby test_me.rb -n /test_.*something.*/

Very useful...

Ok.. hmm Thanks.. (how did I overlooked that?)

However.. still methods without 'test_' prefix gets rejected.
It would be nice if Test::Unit didn't reject non tests.


bash-2.05b$ ruby test_buffer.rb -n /xtest_bracket_matcher13/
Loaded suite test_buffer
Started

Finished in 0.00142 seconds.

0 tests, 0 assertions, 0 failures, 0 errors
bash-2.05b$ grep xtest test_buffer.rb
def xtest_string_paren_forward5
def xtest_bracket_matcher13
bash-2.05b$ ruby -v
ruby 1.8.1 (2004-04-24) [i386-linux-gnu]
bash-2.05b$

 

[James Britt]

Another related feature I would like to see in Test::Unit..
being able to execute tests which doesn't start with 'test' prefix.

For instance I have certain tests which im working on..
those I name #xtest_something..

However if I try to execute the test via the cli.. then it
just rejects it.. What im asking fore.. don't reject test names
supplied via the cli .

I've started partitioning tests into separate files, each file holding
related tests (e.g., each file might focus on some particular feature or
behavior). The tests are called by a master script that executes files
matching a given glob pattern. By default, all of the test files are
run, but I can specify specific files or groups of files.

When I'm working on new tests, I put them in a new file and run just
that set. When I'm happy with them, I move them to a more permanent home.

James

James

 

[James Britt]

Ok.. hmm Thanks.. (how did I overlooked that?)

Yeah, same here. Quite nice.

James

 

[Nathaniel Talbott]

 

[Nathaniel Talbott]

[This time WITHOUT being signed. Sigh.]

i use the following snippet of code
but obviously it would be much nicer
to have it builtin -

We already chatted about this on IRC, but just so that everyone knows:
as David points out in this thread, you can do test matching on the
command-line by simply surrounding your pattern with //:

ruby test.rb -n /pattern/

However, there is no (easy) programmatic interface to this
functionality. Test::Unit is overdue for some serious TLC, and this has
made its way on to the TODO list.


Nathaniel
Terralien, Inc.

<(><

 

[Simon Strandgaard]

We already chatted about this on IRC, but just so that everyone knows:
as David points out in this thread, you can do test matching on the
command-line by simply surrounding your pattern with //:

ruby test.rb -n /pattern/

However, there is no (easy) programmatic interface to this
functionality. Test::Unit is overdue for some serious TLC, and this has
made its way on to the TODO list.

What about invoking disabled test's (I prefix them with x) via the
commandline. Is that a feature you will add ?


Another issue.. it could be nice if Test::Unit could detected if
there are disabled tests in the code.. and then output a warning:
WARNING: there is 3 disabled tests.
But it would dictate that all people also had to use an x prefix.
This last issue is less important to me.. but still something I
would be glad that Test::Unit had builtin. What do you think?

 

[Nathaniel Talbott]

Another related feature I would like to see in Test::Unit..
being able to execute tests which doesn't start with 'test' prefix.

For instance I have certain tests which im working on..
those I name #xtest_something..

However if I try to execute the test via the cli.. then it
just rejects it.. What im asking fore.. don't reject test names
supplied via the cli .

The problem isn't really that test/unit is rejecting the tests...
rather, because of the way suites are automatically constructed from
TestCase classes, test/unit has to have some criteria for knowing which
methods are tests and which methods aren't, and this criteria is that
the method name must start with 'test'. Thus, I would tend to do this
as James suggests, by partitioning tests that aren't working yet in to
their own file and/or TestCase. Note that you can also select TestCases
to run using a pattern:

ruby test.rb -t /^X/

Then you could name in progress TestCases 'XWhatever', and run them
using the above command. To run only completed test cases, just run:

ruby test.rb -t /^[^X]/

I will say that some ideas I have cooking for a future version of
test/unit may obviate this need, but that will be revealed in due
time...


Nathaniel
Terralien, Inc.

<(><

 

[Nathaniel Talbott]

What about invoking disabled test's (I prefix them with x) via the
commandline. Is that a feature you will add ?

No, it just isn't practical (implementation-wise). See the reply I just
sent to an earlier email of yours.

Another issue.. it could be nice if Test::Unit could detected if
there are disabled tests in the code.. and then output a warning:
WARNING: there is 3 disabled tests.
But it would dictate that all people also had to use an x prefix.
This last issue is less important to me.. but still something I
would be glad that Test::Unit had builtin. What do you think?

I remain completely opposed to adding another distinction to tests
(warnings). I've actually considering removing the distinction between
failures and errors, and might still do so at some point. Distinctions
between types of failures are something that I think should not be
(hard-coded) in the framework, but rather managed by the programmer. I
might provide some framework support for that management in the future,
though.


Nathaniel
Terralien, Inc.

<(><

 

[David Naseby]

-----Original Message-----

From: Nathaniel Talbott [mailto:[email protected]]

What about invoking disabled test's (I prefix them with x) via the
commandline. Is that a feature you will add ?

No, it just isn't practical (implementation-wise). See the reply I just
sent to an earlier email of yours.

Would it be possible to set up an ignored handler in test classes? Something
like (inspired by the metaprogramming of tests in the rather good NUnit
framework on .net):

class TC_something < Test::Unit::TestCase
ignore :test_fail

def test_fail
assert 2==1
end
end

I'd send in a patch or something useful, but I'm more of an ideas man...

David
http://homepages.ihug.com.au/~naseby/

 

[Nathaniel Talbott]

-----Original Message-----
From: Nathaniel Talbott [mailto:[email protected]]

What about invoking disabled test's (I prefix them with x) via the
commandline. Is that a feature you will add ?

No, it just isn't practical (implementation-wise). See the reply I
just
sent to an earlier email of yours.

Would it be possible to set up an ignored handler in test classes?
Something
like (inspired by the metaprogramming of tests in the rather good NUnit
framework on .net):

class TC_something < Test::Unit::TestCase
ignore :test_fail

def test_fail
assert 2==1
end
end

I'd send in a patch or something useful, but I'm more of an ideas
man...

Something like this is certainly possible (and might happen) - when I
said it wasn't practical, I was really referring specifically to
running test methods that don't start with 'test'.


Nathaniel
Terralien, Inc.

<(><