[ANN] tinderbox 1.0.0 Released

[Eric Hodel]

tinderbox version 1.0.0 has been released!

http://seattlerb.rubyforge.org/tinderbox

== DESCRIPTION:

Tinderbox tests projects and tries to make them break by running them
on as
many different platforms as possible.

== FEATURES/PROBLEMS:

* Tests gems in a sandbox
* Submits gem test results to http://firebrigade.seattlerb.org
* Understands test/unit and RSpec

Changes:

== 1.0.0 / 2007-01-30

* Tests gems in a sandbox
* Submits results to Firebrigade
* Birthday!

http://seattlerb.rubyforge.org/tinderbox

 

[Bil Kleb]

http://seattlerb.rubyforge.org/tinderbox

Tinderbox tests projects and tries to make them break by running them on as
many different platforms as possible.

Five things:

1) Thank you; A most excellent contribution!

2) A pie chart!? Tufte would be sad.

3) How are projects selected to be tested? I.e., do I submit mine,
are they automatically added? (I'm too dense to find this info
on http://firebrigade.seattlerb.org or http://seattlerb.rubyforge.org/tinderbox/.)

4) What's with the "000 act aed asl ben" pagination headers/footers
for the project listings?

5) Probably been reported before and not related
to Tinderbox, but why doesn't Rubygems understand
repeated dependencies?

$ sudo gem install tinderbox
Need to update 34 gems from http://gems.rubyforge.org
..................................
complete
Install required dependency firebrigade_api? [Yn] Y
Install required dependency rc-rest? [Yn]
Install required dependency hoe? [Yn]
Install required dependency rubyforge? [Yn]
Install required dependency rspec? [Yn]
Install required dependency hoe? [Yn]
Successfully installed tinderbox-1.0.0
Successfully installed firebrigade_api-1.0.0
Successfully installed rc-rest-2.1.0
Successfully installed hoe-1.1.7
Successfully installed rubyforge-0.4.0
Successfully installed rspec-0.7.5.1
Successfully installed hoe-1.1.7
[..]

Later,

 

[Ryan Davis]

1) Thank you; A most excellent contribution!

2) A pie chart!? Tufte would be sad.

HEY! I had to relearn trig to get that stupid thing made! It even has
unit tests and is pixel perfect!

3) How are projects selected to be tested? I.e., do I submit mine,
are they automatically added? (I'm too dense to find this info
on http://firebrigade.seattlerb.org or http://
seattlerb.rubyforge.org/tinderbox/.)

All of them. Any and every gem submitted.

4) What's with the "000 act aed asl ben" pagination headers/footers
for the project listings?

Weird huh?

5) Probably been reported before and not related
to Tinderbox, but why doesn't Rubygems understand
repeated dependencies?

That looks like you still have a poisoned cache from December... The
pain and suffering continues, all thanks to one myopic developer.
Delete your personal and root's source_cache files and install again.

 

[Eric Hodel]

Five things:

1) Thank you; A most excellent contribution!

Its the most depressing Ruby project I've ever written.

2) A pie chart!? Tufte would be sad.

The pie chart makes me sad too, but for different reasons.

3) How are projects selected to be tested? I.e., do I submit mine,
are they automatically added? (I'm too dense to find this info
on http://firebrigade.seattlerb.org or http://
seattlerb.rubyforge.org/tinderbox/.)

If its in the gem index, tinderbox tries to test it.

4) What's with the "000 act aed asl ben" pagination headers/footers
for the project listings?

Encyclopedic pagination, so you know roughly where you're jumping
too. (But search is faster, and it kinda breaks down when you get to
the 'ruby*' projects.)

5) Probably been reported before and not related
to Tinderbox, but why doesn't Rubygems understand
repeated dependencies?

The Install is set for an overhaul in RubyGems 0.9.2. It takes a
recursive approach to installing dependencies, so may have these
kinds of issues.

 

[David Chelimsky]

tinderbox version 1.0.0 has been released!

http://seattlerb.rubyforge.org/tinderbox

== DESCRIPTION:

Tinderbox tests projects and tries to make them break by running them
on as
many different platforms as possible.

== FEATURES/PROBLEMS:

* Tests gems in a sandbox
* Submits gem test results to http://firebrigade.seattlerb.org
* Understands test/unit and RSpec

Sweeeeeeet! Thanks.

Question - Does tinderbox know which version of RSpec to run against?
I ask because RSpec is still evolving and I'd hate for someone's specs
to fail in tinderbox because of an RSpec version mis-match.

Thanks,
David

 

[Tim Pease]

Its the most depressing Ruby project I've ever written.

OK, I'm curious about why this was so depressing. Care to explain?

Frankly, I think it is wonderful, and I'm incredibly grateful for its
existence. The only depressing thing is how little green there is in
the overall pie chart view

The pie chart makes me sad too, but for different reasons.


If its in the gem index, tinderbox tries to test it.


Encyclopedic pagination, so you know roughly where you're jumping
too. (But search is faster, and it kinda breaks down when you get to
the 'ruby*' projects.)


The Install is set for an overhaul in RubyGems 0.9.2. It takes a
recursive approach to installing dependencies, so may have these
kinds of issues.

I now understand why you and Ryan were so adamant about having Hoe
included as a dependency in people's gems. Makes sense from the
perspective of firebrigade and tinderbox.

Blessings,
TwP

 

[Eric Hodel]

Question - Does tinderbox know which version of RSpec to run against?
I ask because RSpec is still evolving and I'd hate for someone's specs
to fail in tinderbox because of an RSpec version mis-match.

Tinderbox tries the latest RSpec. If your package require a
particular version of RSpec it needs to be marked as a dependency.

 

[Bil Kleb]

The pie chart makes me sad too, but for different reasons.

I can imagine, but the big visible chart (aka information
radiator) that you've created should help to remedy the
situation, no?

Now, we need the same thing for the Ruby language specification
tests against all the Ruby engines...

If its in the gem index, tinderbox tries to test it.

Ah. OK, because of #4, no mention on the "gem developers" page,
and not noticing the search box, I didn't see very many gems.

Encyclopedic pagination, so you know roughly where you're jumping too.

But it doesn't work?!

Thanks again for the wonderful BVC,

 

[pat eyler]

I can imagine, but the big visible chart (aka information
radiator) that you've created should help to remedy the
situation, no?

Now, we need the same thing for the Ruby language specification
tests against all the Ruby engines...

Sadly, there's not (yet) a good set of specification tests for Ruby.
There are several that are getting there, and the writers of them
are starting to work together which will help a lot.

On the other hand, getting the a tinderbox running and reporting
for the various implmentations would be a good thing too. There's
no spec like a big pile of running code. ;^)

 

[David Chelimsky]

Tinderbox tries the latest RSpec. If your package require a
particular version of RSpec it needs to be marked as a dependency.

Cool. Thanks again.

David

 

[Eric Hodel]

OK, I'm curious about why this was so depressing. Care to explain?

Frankly, I think it is wonderful, and I'm incredibly grateful for its
existence. The only depressing thing is how little green there is in
the overall pie chart view

Exactly this, too little green.

I now understand why you and Ryan were so adamant about having Hoe
included as a dependency in people's gems. Makes sense from the
perspective of firebrigade and tinderbox.

Tinderbox does a best-effort, but some gems fail tests even when run
with testrb.

 

[Eric Hodel]

I can imagine, but the big visible chart (aka information
radiator) that you've created should help to remedy the
situation, no?
Hopefully.

Now, we need the same thing for the Ruby language specification
tests against all the Ruby engines...

I'm hoping for a JRuby in the list 'o targets soon.

Ah. OK, because of #4, no mention on the "gem developers" page,
and not noticing the search box, I didn't see very many gems.


But it doesn't work?!

On the Projects page, yeah, wrong titles, right pages. I haven't
bothered to fix it yet since it isn't sending me emails.

 

[Tim Pease]

tinderbox version 1.0.0 has been released!

http://seattlerb.rubyforge.org/tinderbox

== DESCRIPTION:

Tinderbox tests projects and tries to make them break by running them
on as
many different platforms as possible.

== FEATURES/PROBLEMS:

* Tests gems in a sandbox
* Submits gem test results to http://firebrigade.seattlerb.org
* Understands test/unit and RSpec

Changes:

== 1.0.0 / 2007-01-30

* Tests gems in a sandbox
* Submits results to Firebrigade
* Birthday!

Does tinderbox have a set of self tests to ensure that the sandbox
environment is sane?

A few of my gems are failing on some targets because of file
permissions not being set properly -- i.e. I set a tmp directory to
0555 permissions and test to see if it is writeable; some targets
report the directory as being writeable

As a corallary, how do we contact the owner of a target to let them
know their tinderbox is not configured properly? There is no contact
information on the firebrigade site for target owners.

TwP

 

[Tim Pease]

Links?

Well, I'll have to link to the space between my ears :/

Turns out I was trying to delete a directory in my teardown when I
still had an open file in there. Poor test cleanup on my part.

I need to throw in an email, but I'll probably keep it private. You
can email me, and I can figure out if its a Tinderbox bug, or a
problem with the target's configuration. If it is a problem with the
target, I can email them and straighten things out.

I went with a soft launch to find these things out, since I knew
there'd be things I missed (like testrb and --program-suffix).

Yeah, testrb was the other problem I was seeing.

Again, many thanks and accolades for setting this up.

TwP

 

[Eric Hodel]

Well, turns out running older hardware can be an issue, I've had to
adjust the Timeout configuration on my machine to allow some tests to
run to completion.

Now it appears I've reported at least one gem (my own keybox none the
less) as failing because of timeout issues. But its really just my
hardware that is old and needed longer than 2 minutes to run. I does
not appear that I can force a resubmission of tinderbox results to
firebrigade without a new version of the gem being made available.

Or am I missing someting obvious?

The feature doesn't exist on purpose, I considered it, but didn't
know if I'd need it or not.

I've cleared out that build of keybox by hand. If it becomes a
regular occurrence I may add a tool.