G
Gregory Brown
Unity is a web based reporting tool for searching and analyzing
RubySpec across Ruby versions and implementations.
It aims to make it easier to find details about the specs themselves,
as well as their pass/fail/error stats.
What follows is a description of the goals of this project, a call for
community support, and a rough roadmap for what we hope to accomplish.
Those who don't read long emails but still like this idea are welcome
to go straight to pledgie and drop in a donation:
http://pledgie.com/campaigns/4640
For those who need more convincing, read on...
== Main Goals
In an interview[0] for the Ruby Best Practices blog, Brian Ford
discussed his ideal goals for the RubySpec project. With Unity, we
hope to do our part to address most of them. Here are the goals
Brian listed, followed by the ways we hope to support them in Unity:
1. Provide a reasonably precise definition of the Ruby programming
language so that there can be a certain degree of uniformity among the
implementations.
Unity will help make it easy for you to know what RubySpec defines
Ruby as. By just typing in the name of any Ruby method, Unity will
provide you with the descriptions of the examples used in RubySpec for
that method. It will also provide you with the test results across
Ruby language and implementation versions, so you can see just how
uniform things really are (or aren't).
2. Provide quality assurance and regression notification for MRI and
all implementations.
Unity will provide failure reports for every release of every
implementation we support. We will also make it easy for users to
find the right place to report unexpected failures. Further support
for advanced reporting and alerts may be planned later.
3. Improving the quality of Ruby as a language by discovering dusty
corners and inconsistencies.
While we won't be able to directly address this issue (at least at
first), having an easier way to look over how Ruby is defined
behaviorally will surely make it easier for community members to come
up with ideas on how to clean things up.
4. Experimentation with implementations and the Ruby language itself.
Unity will make it relatively clear which versions of which
implementations support a certain feature. This will help make Ruby
users and library maintainers more likely to try new implementations
or versions of the Ruby language, as it'll give them a clear sense of
what to expect. While this can be accomplished with RubySpec alone,
Unity will surely lower the barrier to entry and eliminate some
variables.
5. A learning tool that provides Ruby programmers with concrete
examples of code that demonstrates every single Ruby language feature.
Unity will provide links or inline source listings for all examples
within RubySpec. This should be as easy as clicking on an example
description within a method's report listing.
In summary, Unity does not so much have any *new* goals, but instead
is an additional tool for advancing the goals of the RubySpec project.
== Project Roadmap
Madriska Media Group[1] is partially sponsoring this project, by
covering some of my developmentment time ( a few hours a week, likely
), donating some of their own development time, and providing some
necessary resources. However, I'd like to put this project on an
accelerated schedule so that I can launch something by the end of the
summer. The easiest way for me to do that is to ask for just a little
bit of extra support from the community.
If you think these goals are worthwhile, consider making a donation so
that I can spend some additional dedicated time on this project.
http://pledgie.com/campaigns/4640
I will be working on the project no matter what, but if I can
guarantee an extra 10 hours a week of dev time, it's much more likely
that I'll have something useful launched within the next couple
months.
What follows is a bit more detail of what specifically we've already
done, and what we hope to do in the coming weeks.
== What we've done so far
A simple Rails 2.3, Ruby 1.9 based prototype that can model spec
output data and report against it.
For demo purposes, we have loaded data from MRI Ruby 1.8.6p369 and
YARV Ruby 1.9.1p129
Screenshot: http://is.gd/12Cu2
Code: http://github.com/madriska/unity/tree/master
e.g. Object#an_instance_method, Object.a_class_method
* displays example descriptions along with the Ruby versions they run on.
* displays pass-fail data for the implementations that it has data for
* data is loaded into database via an importer which reads from
real mspec output
* internally able to model testing environments based on both ruby
version and ruby implmentation version
* currently reports results based on ruby version and
implementation (without release version)
errors, it is not currently reported on
* does not support Class::method syntax for class methods (use
Class.method instead)
* does not support Nested::Modules yet
* relies on patch to mspec at:
http://github.com/madriska/mspec/commit/36b8bce62c272d546bcbd8ab9f136576ff4701e5
* very limited testing so far (Array specs on MRI 1.8.6 and YARV 1.9.1 only)
* mspec loading can be brittle -- no recourse if spec execution hangs.
== What needs to be done
In addition to the limitations discussed above, we would like to do
the following:
Core functionality:
- Patch back to mspec where needed
- Improve reporting output to more clearly see descriptions organized
by which Ruby versions they are relevant to
- Add filters to prune out specific ruby versions / implementations
from a given report
- Provide per-class reports so that it is easy to navigate to a
particular method, as well as see pass/fail/error data aggregated
- Provide per-release reports of P/F/E data for all the Ruby
implementations we support, faceted by class/module or method.
- Expose a service interface for importing mspec data, to decouple web
app from toolchain
Additional Features We Want:
- Provide a way to automate collection of report data across the Ruby
implementations we support. (Communicate over service API)
- Generate a 'state of Ruby' comprehensive report for a given
implementation release and ruby version.
- Provide some comparative analysis between Ruby versions
- Link back (or list inline) spec source text for each example.
== When Will This Be Ready?
We can't make any promises, but our goal is to have a preliminary site
launched by July 15th, 2009. Shortly after that, we'll post a revised
roadmap that describes what we hope to call 1.0, which we hope to have
ready by August 15th, 2009. While we won't be accepting patches or
formal feature requests between now and July 15th, we welcome you to
discuss this with us on our google group, and will definitely be
interested in code contributions after mid July.
http://groups.google.com/group/ruby-unity
Anyway, we look forward to having something more to show you soon!
- Greg & Brad
[0] http://blog.rubybestpractices.com/posts/gregory/006-brian-ford-rubyspec.html
[1] http://madriska.com
RubySpec across Ruby versions and implementations.
It aims to make it easier to find details about the specs themselves,
as well as their pass/fail/error stats.
What follows is a description of the goals of this project, a call for
community support, and a rough roadmap for what we hope to accomplish.
Those who don't read long emails but still like this idea are welcome
to go straight to pledgie and drop in a donation:
http://pledgie.com/campaigns/4640
For those who need more convincing, read on...
== Main Goals
In an interview[0] for the Ruby Best Practices blog, Brian Ford
discussed his ideal goals for the RubySpec project. With Unity, we
hope to do our part to address most of them. Here are the goals
Brian listed, followed by the ways we hope to support them in Unity:
1. Provide a reasonably precise definition of the Ruby programming
language so that there can be a certain degree of uniformity among the
implementations.
Unity will help make it easy for you to know what RubySpec defines
Ruby as. By just typing in the name of any Ruby method, Unity will
provide you with the descriptions of the examples used in RubySpec for
that method. It will also provide you with the test results across
Ruby language and implementation versions, so you can see just how
uniform things really are (or aren't).
2. Provide quality assurance and regression notification for MRI and
all implementations.
Unity will provide failure reports for every release of every
implementation we support. We will also make it easy for users to
find the right place to report unexpected failures. Further support
for advanced reporting and alerts may be planned later.
3. Improving the quality of Ruby as a language by discovering dusty
corners and inconsistencies.
While we won't be able to directly address this issue (at least at
first), having an easier way to look over how Ruby is defined
behaviorally will surely make it easier for community members to come
up with ideas on how to clean things up.
4. Experimentation with implementations and the Ruby language itself.
Unity will make it relatively clear which versions of which
implementations support a certain feature. This will help make Ruby
users and library maintainers more likely to try new implementations
or versions of the Ruby language, as it'll give them a clear sense of
what to expect. While this can be accomplished with RubySpec alone,
Unity will surely lower the barrier to entry and eliminate some
variables.
5. A learning tool that provides Ruby programmers with concrete
examples of code that demonstrates every single Ruby language feature.
Unity will provide links or inline source listings for all examples
within RubySpec. This should be as easy as clicking on an example
description within a method's report listing.
In summary, Unity does not so much have any *new* goals, but instead
is an additional tool for advancing the goals of the RubySpec project.
== Project Roadmap
Madriska Media Group[1] is partially sponsoring this project, by
covering some of my developmentment time ( a few hours a week, likely
), donating some of their own development time, and providing some
necessary resources. However, I'd like to put this project on an
accelerated schedule so that I can launch something by the end of the
summer. The easiest way for me to do that is to ask for just a little
bit of extra support from the community.
If you think these goals are worthwhile, consider making a donation so
that I can spend some additional dedicated time on this project.
http://pledgie.com/campaigns/4640
I will be working on the project no matter what, but if I can
guarantee an extra 10 hours a week of dev time, it's much more likely
that I'll have something useful launched within the next couple
months.
What follows is a bit more detail of what specifically we've already
done, and what we hope to do in the coming weeks.
== What we've done so far
A simple Rails 2.3, Ruby 1.9 based prototype that can model spec
output data and report against it.
For demo purposes, we have loaded data from MRI Ruby 1.8.6p369 and
YARV Ruby 1.9.1p129
Screenshot: http://is.gd/12Cu2
Code: http://github.com/madriska/unity/tree/master
* can look up spec descriptions and results by method descriptor (ala RI)
e.g. Object#an_instance_method, Object.a_class_method
* displays example descriptions along with the Ruby versions they run on.
* displays pass-fail data for the implementations that it has data for
* data is loaded into database via an importer which reads from
real mspec output
* internally able to model testing environments based on both ruby
version and ruby implmentation version
* currently reports results based on ruby version and
implementation (without release version)
* while full spec output is stored, including backtraces for
errors, it is not currently reported on
* does not support Class::method syntax for class methods (use
Class.method instead)
* does not support Nested::Modules yet
* relies on patch to mspec at:
http://github.com/madriska/mspec/commit/36b8bce62c272d546bcbd8ab9f136576ff4701e5
* very limited testing so far (Array specs on MRI 1.8.6 and YARV 1.9.1 only)
* mspec loading can be brittle -- no recourse if spec execution hangs.
== What needs to be done
In addition to the limitations discussed above, we would like to do
the following:
Core functionality:
- Patch back to mspec where needed
- Improve reporting output to more clearly see descriptions organized
by which Ruby versions they are relevant to
- Add filters to prune out specific ruby versions / implementations
from a given report
- Provide per-class reports so that it is easy to navigate to a
particular method, as well as see pass/fail/error data aggregated
- Provide per-release reports of P/F/E data for all the Ruby
implementations we support, faceted by class/module or method.
- Expose a service interface for importing mspec data, to decouple web
app from toolchain
Additional Features We Want:
- Provide a way to automate collection of report data across the Ruby
implementations we support. (Communicate over service API)
- Generate a 'state of Ruby' comprehensive report for a given
implementation release and ruby version.
- Provide some comparative analysis between Ruby versions
- Link back (or list inline) spec source text for each example.
== When Will This Be Ready?
We can't make any promises, but our goal is to have a preliminary site
launched by July 15th, 2009. Shortly after that, we'll post a revised
roadmap that describes what we hope to call 1.0, which we hope to have
ready by August 15th, 2009. While we won't be accepting patches or
formal feature requests between now and July 15th, we welcome you to
discuss this with us on our google group, and will definitely be
interested in code contributions after mid July.
http://groups.google.com/group/ruby-unity
Anyway, we look forward to having something more to show you soon!
- Greg & Brad
[0] http://blog.rubybestpractices.com/posts/gregory/006-brian-ford-rubyspec.html
[1] http://madriska.com