Benchmark in MiniTest

[Oren]

https://gist.github.com/813736

def test_hot_is_constant
assert_performance_constant 0.9999 do
@reddit.hot
end
end

def bench_my_algorithm
assert_performance_linear 0.9999 do |n| # n is a range value
n.times do
@reddit.hot
end
end
end

the output i get is:
TestReddit 1 10 100 1000 10000
bench_hot 0.000039 0.000052 0.000032 0.000033 0.000032
bench_hot2 0.000046 0.000047 0.000088 0.000596 0.005650


my question - Is there a way to set the maximum time for a method to run?
if it goes above it i want it to fail.
what does assert_performance_constant mean? is it a constant speed in relation to the argument the method accepts?

Thanks

 

[Ryan Davis]

https://gist.github.com/813736
=20
def test_hot_is_constant
assert_performance_constant 0.9999 do
@reddit.hot
end
end
=20
def bench_my_algorithm
assert_performance_linear 0.9999 do |n| # n is a range value
n.times do
@reddit.hot
end
end
end
=20
the output i get is:
TestReddit 1 10 100 1000 10000
bench_hot 0.000039 0.000052 0.000032 = 0.000033 0.000032
bench_hot2 0.000046 0.000047 0.000088 = 0.000596 0.005650
=20
=20
my question - Is there a way to set the maximum time for a method to = run?
if it goes above it i want it to fail.

No. I'm not sure it makes too much sense. On a slower or more loaded =
machine, you'd get false negatives.

what does assert_performance_constant mean? is it a constant speed in =

relation to the argument the method accepts?

8511 % ri MiniTest::Unit::TestCase.assert_performance_constant
=3D MiniTest::Unit::TestCase.assert_performance_constant

(from gem minitest-2.0.2)
=
--------------------------------------------------------------------------=
----
assert_performance_constant(threshold =3D 0.99, &work)

=
--------------------------------------------------------------------------=
----

Runs the given work and asserts that the times gathered fit to match a
constant rate (eg, linear slope =3D=3D 0) within a given error =
threshold.

Fit is calculated by #fit_constant.

Ranges are specified by ::bench_range.

Eg:

def bench_algorithm
assert_performance_constant 0.9999 do |x|
@obj.algorithm
end
end

 

[Benoit Daloze]

https://gist.github.com/813736

=A0def test_hot_is_constant
=A0 =A0assert_performance_constant 0.9999 do
=A0 =A0 [email protected]
=A0 =A0end
=A0end

=A0def bench_my_algorithm
=A0 =A0assert_performance_linear 0.9999 do |n| # n is a range value
=A0 =A0 =A0n.times do
=A0 =A0 =A0 [email protected]
=A0 =A0 =A0end
=A0 =A0end
=A0end

the output i get is:
TestReddit =A0 =A0 =A01 =A0 =A0 =A0 10 =A0 =A0 =A0100 =A0 =A0 1000 =A0 = =A010000
bench_hot =A0 =A0 =A0 =A00.000039 =A0 =A0 =A0 =A00.000052 =A0 =A0 =A0 =A0=

0.000032 =A0 =A0 =A0 =A00.000033 =A0 =A0 =A0 =A00.000032

bench_hot2 =A0 =A0 =A0 0.000046 =A0 =A0 =A0 =A00.000047 =A0 =A0 =A0 =A00.=

000088 =A0 =A0 =A0 =A00.000596 =A0 =A0 =A0 =A00.005650

my question - Is there a way to set the maximum time for a method to run?
if it goes above it i want it to fail.
what does assert_performance_constant mean? is it a constant speed in rel=

ation to the argument the method accepts?

Thanks

Please be aware your are testing if Integer#times is liner in your
second test, not @reddit.hot.
(which does not take any parameter from your code, and is then very
likely to have constant time).

A good example would be:

def fib(n)
# my super code to compute the n^th Fibonacci term in linear time
end

assert_performance_linear 0.9999 do |n|
fib(n)
end

The only good usage of Integer#times (except for testing #times
itself) in assert_performance_*, is to increase the time when the
block runs too fast (like all n <=3D 10 000 runs in less than 0.001 s)
You then want to use a constant to call #times on it (like 1000.times
{ fib(n) }), *never* n.
You could also use the :bench_range option, probably better.

Regards,
Benoit Daloze