Ruby report generation tool

[Greg Brown]

For as long as I can remember the end of the summer meant slaving over
some Free Software project before I went back to school. This year
will be no exception. I am currently trying to develop and mature a
pure ruby reporting tool based on a very clever hack that James Edward
Gray II shared with me (along with some great ideas). Basically, I am
trying to make a tool that will allow you to run queries against a
database and then get them to painless output to basically whatever
format you'd like weilding the power of Erb and other great tools such
as PDF::Writer. So far, the system I have built is functional but far
from robust. It allows you to execute SQL statements passed as
strings, passed in from files, or even passed in from the database
itself and then gives you a row by row iterator which can be called
from within a template OR a pure ruby file. I am also currently
working on implementing a simple DSL wrapper around SQL to allow easy
generation of queries. I have a few ideas for functions I'd like to
add, but I figured the best bet would be to ask the community what
kinds of features they'd like to see in a pure ruby report generation
tool. If you let me know what you'd like to see in such a tool soon,
there is a good chance it will end up in Ruport 0.1.0 ([Ru]by
Re[port]s) which will be released on August 28th on RubyForge. So...
if you had your ideal reporting tool in Ruby, what would it be like?

My job is paying me to work on this project despite the fact that it is
under the GPL, therefore the more requests you make, the longer I can
justify working on this instead of the (gulp) .Net system they've been
developing.

Below I've posted a little snippet of Ruport calling a query out of
the database itself (config and query not shown) and then generating a
PDF. This is actual functional code which prints out customer labels.

Also... scouts honor on a first release of Gambit (The game design
software James and I built) before the end of the summer.

With no further ado, a (rudimentary) Ruport example.


require 'pdf/writer'
pdf = PDF::Writer.new
pdf.select_font "Times-Roman"
pdf.margins_mm(12,0,20,0)
pdf.start_columns(3,27)
sql_stored( "RUPORT_QUERIES", "RECALL") do |row|
row["DUEDATE"] = row["DUEDATE"].to_s.slice( 0 .. 9)
row["ZIP"] = row["ZIP"].to_s.slice(0..4)
pdf.start_new_page if pdf.lines_remaining < 5
pdf.text "Due: #{row["DUEDATE"]}\n" +
"#{row["FIRSTNAME"]} #{row["LASTNAME"]}\n" +
"#{row["STREET1"]}\n" +
( row["STREET2"].eql?("") ? "" : row["STREET2"] + "\n" ) +
"#{row["CITY"]}, #{row["STATE"]} #{row["ZIP"]}\n\n" +
( row["STREET2"].eql?("") ? "\n" : "" ),
:font_size => 11, :justification => :full
end
<%= pdf.render %>

 

[Greg Brown]

With no further ado, a (rudimentary) Ruport example.

<%= pdf.render %>

whoops... should be: pdf.render

Thanks James for pointing this out

 

[Randy Kramer]

My job is paying me to work on this project despite the fact that it is
under the GPL, therefore the more requests you make, the longer I can
justify working on this instead of the (gulp) .Net system they've been
developing.

This is probably pretty far OT for your current project, but I'll mention it
anyway, in the interests of saving you from .Net ;-)

I need something that would take text in TWiki markup (with some variations)
and convert it to (X)HTML. The source of the text could be (and ideally is)
anything (from "plain text" files (with markup) to similar (marked up) text
stored in a database or directly from a plain text editor (like nedit)).

Ideally, it would also feed into an (X)HTML renderer / browser that would be
"under the control" of the "something". In other words, sort of like a push
technology--the HTML renderer/browser displays what it is told to display
(what it is sent, when it is sent) rather than having the user work in the
browser to, for example, change pages.

Initially (but hopefully not for long), the HTML renderer might be limited to
displaying text sent to it. Later it would be nice:
* if it was closer to a real browser such that it had the tabs feature, and
could follow links on the "remotely controlled" page in new tabs
* it could display pictures and so forth (beyond plain text)

I've noted the recent thread where Hal Fulton is looking for a simple HTML
renderer and will be following that with interest.

regards,
Randy Kramer

 

[Greg Brown]

Regarding Ruport, I have received some requests to consider a license
other than the GPL. I am thinking of using the license of ruby itself,
which would be a disjunctive license of the GPL and the ruby license.
Would this be permissive enough for most people's needs? I am willing
to consider the Artistic license as well, though I'd prefer to use a
license which offers the GPL as an option. Feel free to voice your
opinion on or off list.

 

[Austin Ziegler]

Regarding Ruport, I have received some requests to consider a license
other than the GPL. I am thinking of using the license of ruby
itself, which would be a disjunctive license of the GPL and the ruby
license. Would this be permissive enough for most people's needs? I
am willing to consider the Artistic license as well, though I'd prefer
to use a license which offers the GPL as an option. Feel free to
voice your opinion on or off list.

When working on PDF::Writer, I deliberately chose the MIT-style licence
because I wanted it to be used as widely as possible. It can be used in
GNU GPLed projects without problems (modulo issues with Adobe's patents,
but that's a generic issue, not a GNU GPL issue) or under commercial
projects without problems.

If one makes the report generator GPLed, then you have severely
restricted it. If one makes the report generator MPLed (for example),
then the commercial and other OSS opportunities are greater, but the GPL
is incompatible with the MPL (this is mostly the GPL's fault).

Ultimately, the licence is up to you, but life is easier if you choose a
single minimally restrictive licence rather than dual-licence. (Don't
get me *started* on the licensing for Text::Hyphen because of jackasses
who licenced their hyphenation files under the GNU GPL.)

-austin
--=20
Austin Ziegler * (e-mail address removed)
* Alternate: (e-mail address removed)

 

[Tom Copeland]

When working on PDF::Writer, I deliberately chose the MIT-style licence
because I wanted it to be used as widely as possible. It can be used in
GNU GPLed projects without problems (modulo issues with Adobe's patents,
but that's a generic issue, not a GNU GPL issue) or under commercial
projects without problems.

+1. I BSD license my projects whenever possible - if someone wants to
use them in a product, more power to them. More than likely, they'll be
feeding bug fixes and whatnot back into the code, and even if not, it's
more mindshare for the project. It's worked out well for me in the Java
world with PMD - http://pmd.sf.net/ - good times.

Yours,

Tom

 

[Greg Brown]

Austin Ziegler wrote:

Ultimately, the licence is up to you, but life is easier if you choose a
single minimally restrictive licence rather than dual-licence. (Don't
get me *started* on the licensing for Text::Hyphen because of jackasses
who licenced their hyphenation files under the GNU GPL.

Okay, I'm fairly certain that I will not be releasing under just the
GPL, as it's clear that people think that it's not a great idea. Thank
you for your suggestions Austin, they were helpful.

I am going to consider releasing under the Clarified Artistic License.
Given the choice between this and the disjunctive license of Ruby, what
are your opinions. Anyone feel free to respond on or off list.

 

[Austin Ziegler]

Austin Ziegler wrote:

Okay, I'm fairly certain that I will not be releasing under just the
GPL, as it's clear that people think that it's not a great idea. Thank
you for your suggestions Austin, they were helpful.
=20
I am going to consider releasing under the Clarified Artistic License.
Given the choice between this and the disjunctive license of Ruby, what
are your opinions. Anyone feel free to respond on or off list.

Neither the Ruby licence nor the Clarified Artistic licences are
"approved" by the OSI (www.opensource.org). That said, the Ruby
licence is pretty clear as to what's allowed (pretty much "anything").
I still think that your best choice is going to be BSDish or MITish,
but if you go with anything else, then you'll probably need to do
CAL/GPL disjunctive or the Ruby licensing option.

Given that it's a Ruby package, the Ruby licence is probably okay enough.

-austin
--=20
Austin Ziegler * (e-mail address removed)
* Alternate: (e-mail address removed)

 

[Greg Brown]

Neither the Ruby licence nor the Clarified Artistic licences are
"approved" by the OSI (www.opensource.org).

That's not true. The 8th clause is the one that makes the Artistic
license GPL compatible, and the approval by OSI is listed below:

http://www.opensource.org/licenses/artistic-license.php

That said, the Ruby
licence is pretty clear as to what's allowed (pretty much "anything").
I still think that your best choice is going to be BSDish or MITish,
but if you go with anything else, then you'll probably need to do
CAL/GPL disjunctive or the Ruby licensing option.

I am honestly not interested in using a BSD or MIT style license. I
understand there is a strong tradition in the Ruby community to lean
this way, and I respect this, but I am in favor of the FSF's ideals and
am looking to preserve them as much as possible while still offering a
fair and pragmatic licensing scheme for my software.

 

[Austin Ziegler]

That's not true. The 8th clause is the one that makes the Artistic
license GPL compatible, and the approval by OSI is listed below:
=20
http://www.opensource.org/licenses/artistic-license.php

If that's the clause that makes the difference between the CAL and the
AL, then you're right. However, I was looking at the title.

I am honestly not interested in using a BSD or MIT style license. I
understand there is a strong tradition in the Ruby community to lean
this way, and I respect this, but I am in favor of the FSF's ideals
and am looking to preserve them as much as possible while still
offering a fair and pragmatic licensing scheme for my software.

Ah. That's where we differ. I disapprove of RMS and the FSF's methods
(not so much the ideals) so strongly that, given a choice, I will not
licence anything GPLed and if there's a choice between a GPLed library
and a non-GPLed library, I will preferentially use the non-GPLed library
even at the possible expense of features. I have no issue with the GNU
GPL *as a licence philosophy*, but I have major problems with the
politicised preamble that I do not wish to redistribute. Ever. I also
have major issues with the denial of the GNU GPL as a viral licence
(that's a feature of the licence, not a negative point), the whole
"software freedom" issue (as if software were something that could be
emancipated), and in general the deception that goes on surrounding the
GNU GPL.

IMO, there's similar licences (the Mozilla PL, for one) that are
friendlier, stronger, and have no political crap surrounding them. IIRC,
there's a licence that is a variant of the GNU GPL simply with the
preamble removed.

Note that disjunctive licensing is questionable at best, at leaves your
users open to choose whichever licence they want. Strictly speaking,
although I've preserved the intent of several packages (e.g.,
Text::Format, others) by keeping the GPL as one of the specified
licences, but I could have *simply* chosen to change the licence for the
Ruby version as the Artistic licence, completely ignoring the GNU GPL's
existence on the Perl version of Text::Format.

Not trying to be difficult here or even start a flamewar, but trying to
explain why I prefer the other licences to the GNU GPL.

-austin
--=20
Austin Ziegler * (e-mail address removed)
* Alternate: (e-mail address removed)

 

[Greg Brown]

Austin Ziegler wrote:
<snip lots of practical opinions about licensing>

I won't use this post as a soapbox for my support of the FSF so I won't
attempt to defend the GPL on list. However, it's safe to say that I do
believe in their philosophy, and I find the preamble to be one of the
best parts.

Not trying to be difficult here or even start a flamewar, but trying to
explain why I prefer the other licences to the GNU GPL.

I appreciate your honest opinion. I am fairly certain that I will use
the license of ruby. I.e. a disjunction between the GPL and the Ruby
license. This way no one needs to deal with the GPL if they don't want
to, and anyone who DOES want to support the use of the GPL may do so
freely. I feel that this is a fair and reasonable course of action for
someone who is most certainly a Free Software (and not necessarily an
Open Source) advocate.

Will anyone have trouble using Ruport under such a scheme?

 

[Austin Ziegler]

I appreciate your honest opinion. I am fairly certain that I will use
the license of ruby. I.e. a disjunction between the GPL and the Ruby
license. This way no one needs to deal with the GPL if they don't want
to, and anyone who DOES want to support the use of the GPL may do so
freely. I feel that this is a fair and reasonable course of action for
someone who is most certainly a Free Software (and not necessarily an
Open Source) advocate.
=20
Will anyone have trouble using Ruport under such a scheme?

I'll happily use it. (Just to be clear.)

-austin
--=20
Austin Ziegler * (e-mail address removed)
* Alternate: (e-mail address removed)

 

[Julian Leviston]

how do u earn money from them then?

+1. I BSD license my projects whenever possible - if someone wants to
use them in a product, more power to them. More than likely,
they'll be
feeding bug fixes and whatnot back into the code, and even if not,
it's
more mindshare for the project. It's worked out well for me in the
Java
world with PMD - http://pmd.sf.net/ - good times.

Yours,

Tom

 

[Austin Ziegler]

how do u earn money from them then?

I can't speak for Tom, but I've actually never earned a penny from my
open source projects. Not that I wouldn't *love* to, but for me, it
isn't about making money from these projects. Let me briefly go through
the various projects that I have and *why* I did them:

1. MIME::Types, Text::Format: These two were originally done for a tool
called RTidy/CD which ran HTMLTidy for the Fog Creek web site
editor/manager CityDesk (hence the /CD). I needed to format the
output of the tool nicely and handle binary files differently than
HTML files, so I ported these from Perl. RTidy/CD itself is open
source, but no one ever asked me for the source and CityDesk version
2 made RTidy/CD obsolete. I keep maintaining MIME::Types and
Text::Format because I have taken them on as responsibilities.

Text::Format saw a 1.0 release recently and probably won't see a lot
of work until Ruby 2.0 (or at least the M17N strings in Ruby 1.9)
comes out and I need to look at what may need to change for that.

MIME::Types is likely to see more releases, plus a change in the way
that the whole thing works so that the core library (MIME::Type, and
the MIME::Types lookup) can be released separately from the MIME
content type data. I will also be looking at integrating
libsharedmime and libmagic functionality. I should get a new version
out soon because I have noticed that the definitions are a little
out of date, but that depends on the other work that I do.

2. Uninheritable. This library will probably never see another release,
and I'd be surprised if anyone uses it. It was more of a proof of
concept and there seemed to be a rash of requests for making classes
"unable to be inherited from" at the time that I did it originally.

3. TeX::Hyphen, Text::Hyphen: These libraries were created because of
needs for Text::Format. Martin DeMello actually ported TeX::Hyphen,
and then I modified that to Text::Hyphen (with a lot more languages
supported) to fix some bugs that existed in the original Perl (e.g.,
not Martin's fault, and VERY complex to fix, hence the new library as
I wasn't interested in creating a TeX interpreter in Ruby). The
former probably won't see any more updates by me, although Martin is
an administrator/developer on that project so he may choose to do
something with it. Text::Hyphen may see periodic updates as people
report hyphenation issues or point out that there's an updated
version of a language that I don't have converted. It will also
probably see an M17N update when Ruby has M17N strings. After that,
the library Just Works, so I expect that it will probably see few
releases.

4. Transaction::Simple: Created for the initial (technology preview)
release of PDF::Writer. It's grown a bit since then, and there are
things that I *want* to do with it that I just haven't had time to
ask how they might be done of people who are much better at Ruby than
I am (one of the big flaws of Transaction::Simple is that it can end
up duplicating objects, and this has caused an API change in
PDF::Writer that I ultimately want to change back). This will see
more releases.

5. color-tools: Created for the second release of PDF::Writer (1.0) to
collect colour information in one place. It's grown since then, and
will probably continue to grow. I'm looking at incorporating
littleCMS-alike functionality (and possibly bindings to littleCMS for
performance) so that it can deal with ICC colour profiles and then
properly convert between various colour spaces meaningfully and
reversably (not all colour conversions are reversable right now).
This will see more releases.

6. Diff::LCS: Originally written to support Ruwiki. Interestingly, this
library shouldn't even have to exist, since Algorithm:iff did most
of what I needed at the time (Diff::LCS does more, now) but was
licenced as GPL, which was not compatible with the desired Ruwiki
licence. It will see more releases, probably ultimately including
better diff support and merge support.

7. Archive::Tar::Minitar: Adapted from code provided by Mauricio
Fernandez for Ruwiki's command-line deployment support. This will
also see more releases, as I plan on adding symlink and hardlink
support, but will probably stop at that point.

8. Ruwiki: Picked up because I needed a wiki. I have plans for further
work on Ruwiki, but the changes I plan are big, and I will be looking
at the implementations of some other wikis in the meantime. This will
probably be dependent upon changes to Diff::LCS moving forward. I'll
probably be looking at the AJAX stuff that one of the Rails wikis
just added, but Ruwiki will not be moving to a web framework (it will
still be a CGI-based Wiki) and will remain flat-file based at its
core. Someone -- I *think* Gavin Kistner -- did something with node
reparenting that I may be digging into, or even reusing the OWL stuff
for future versions. There have been changes in the Ruby environment
since I last worked on Ruwiki -- Ruwiki will adapt, and I will make
it possible to upgrade.

9. PDF::Writer: This was originally written because there was a hole in
Ruby's support for PDF. It's somewhat taken over my life at this
point, and as I pointed out in a different post today, I'm working on
a release right now (well, I'm procrastinating right now) and then
I'm starting work on version 2.0 -- which will be demonstrated, in
part, at RubyConf 2005 because I'm also adding a slideshow producer.

Don't get me wrong -- if anyone wants to donate to the "Encourage
Austin" fund, I'll not turn it down. It'll help defray my costs to
RubyConf I do this because I *love* it, and it keeps me thinking
about different programming mechanisms and paradigms that I don't get to
work with at my day job. This is *very* valuable, because I've been able
to present ideas that have been accepted even though they're a bit
"off-the-wall". Because I love doing this, why should I be selfish? I
share what I do.

-austin
--=20
Austin Ziegler * (e-mail address removed)
* Alternate: (e-mail address removed)

 

[Greg Brown]

how do u earn money from them then?

Austin put this quite well though I have to say that I've made my
living off of free software thus far. Essentially I have been funded
by companies to implement certain features into generic systems that
they don't have much of a stake in. I make this agreement with them
and all they care about is the fact that they get a product that does
what they want, and they leave me free to share.

To me, anything I can share with the community, I will. I do not take
straight proprietary jobs simply because I never want to lock my source
in a vault for life and throw away the key. There are a ton of reasons
to write Free Software, and Austin mentions many of them.

For me, it's just fun to share and get to know people and all of that
good stuff.

 

[Alexandru Popescu]

#: Greg Brown changed the world a bit at a time by saying on 8/11/2005 5:46 AM :#

For as long as I can remember the end of the summer meant slaving over
some Free Software project before I went back to school. This year
will be no exception. I am currently trying to develop and mature a
pure ruby reporting tool based on a very clever hack that James Edward
Gray II shared with me (along with some great ideas). Basically, I am
trying to make a tool that will allow you to run queries against a
database and then get them to painless output to basically whatever
format you'd like weilding the power of Erb and other great tools such
as PDF::Writer. So far, the system I have built is functional but far
from robust. It allows you to execute SQL statements passed as
strings, passed in from files, or even passed in from the database
itself and then gives you a row by row iterator which can be called
from within a template OR a pure ruby file. I am also currently
working on implementing a simple DSL wrapper around SQL to allow easy
generation of queries. I have a few ideas for functions I'd like to
add, but I figured the best bet would be to ask the community what
kinds of features they'd like to see in a pure ruby report generation
tool. If you let me know what you'd like to see in such a tool soon,
there is a good chance it will end up in Ruport 0.1.0 ([Ru]by
Re[port]s) which will be released on August 28th on RubyForge. So...
if you had your ideal reporting tool in Ruby, what would it be like?

My job is paying me to work on this project despite the fact that it is
under the GPL, therefore the more requests you make, the longer I can
justify working on this instead of the (gulp) .Net system they've been
developing.

Below I've posted a little snippet of Ruport calling a query out of
the database itself (config and query not shown) and then generating a
PDF. This is actual functional code which prints out customer labels.

Also... scouts honor on a first release of Gambit (The game design
software James and I built) before the end of the summer.

With no further ado, a (rudimentary) Ruport example.


require 'pdf/writer'
pdf = PDF::Writer.new
pdf.select_font "Times-Roman"
pdf.margins_mm(12,0,20,0)
pdf.start_columns(3,27)
sql_stored( "RUPORT_QUERIES", "RECALL") do |row|
row["DUEDATE"] = row["DUEDATE"].to_s.slice( 0 .. 9)
row["ZIP"] = row["ZIP"].to_s.slice(0..4)
pdf.start_new_page if pdf.lines_remaining < 5
pdf.text "Due: #{row["DUEDATE"]}\n" +
"#{row["FIRSTNAME"]} #{row["LASTNAME"]}\n" +
"#{row["STREET1"]}\n" +
( row["STREET2"].eql?("") ? "" : row["STREET2"] + "\n" ) +
"#{row["CITY"]}, #{row["STATE"]} #{row["ZIP"]}\n\n" +
( row["STREET2"].eql?("") ? "\n" : "" ),
:font_size => 11, :justification => :full
end
<%= pdf.render %>

Here are my .02 eurocents about a generic reporting tool:

i would probably split it into 2 modules: data feeders and report generator/formatter. This way you
will be able to find feedback from more sources for both modules. Moreover you will have a very
clean separation of concerns.

:alex |.::the_mindstorm::.|

 

[Greg Brown]

Here are my .02 eurocents about a generic reporting tool:
i would probably split it into 2 modules: data feeders and report
generator/formatter. This way you
will be able to find feedback from more sources for both modules. Moreover you
will have a very clean separation of concerns.

:alex |.::the_mindstorm::.|

This is a great idea and I will do my best to make Ruport model this
type of separation. Ruport will be on CVS sometime later this week for
a little scrutiny before it's initial release on the 28th, I will post
when it is up.

 

[Greg Brown]

In an effort to make Ruport either a) the most community inclusive
project ever, or b) a real pain in the neck to the regulars, I've
created a survey that you're welcome to come voice your opinion on how
you'd like to see the project turn out. It's really short, 11
questions, mostly yes or no's.

It's over at rubyforge:
http://rubyforge.org/survey/survey.php?group_id=856&survey_id=20

It'll determine a few key aspects of how I spend the next two weeks or
so, whether or not Ruport gets up on CVS before the 28th, whether or
not documentation will be included with the first release, whether or
not it will be split into two modules, and whether or not I'll consider
a license that's even less restrictive or complicated than the Ruby/GPL
combo. So if any of this matters to you, feel free to come and let
your opinion be known in statistical form.

It also lets you submit feature requests

 

[Alexandru Popescu]

#: Greg Brown changed the world a bit at a time by saying on 8/17/2005 12:18 AM :#

This is a great idea and I will do my best to make Ruport model this
type of separation. Ruport will be on CVS sometime later this week for
a little scrutiny before it's initial release on the 28th, I will post
when it is up.

Glad you like the idea.

Also passing through the survey I have noticed the 2nd question that is (again) imho quite tricky
. Definitely it should be easy to use to gain a community; once you have it you will need to
raise the bar and make it more and more customizable. This is something that I would say that fits
best: with start small philosophy ;-).

Even if I am a Ruby freshman I would be really happy to be able to see the sourcebase very soon and
even more to help.

cheers,
:alex |.::the_mindstorm::.|

 

[Greg Brown]

In an effort to make Ruport either a) the most community inclusive
project ever, or b) a real pain in the neck to the regulars, I've
created a survey that you're welcome to come voice your opinion on how
you'd like to see the project turn out. It's really short, 11
questions, mostly yes or no's.

I've updated this survey and it has nearly double the questions and all
the yes or no's have been changed to sliding scale, so you can be a
little more wishy washy with your feelings.

Linkage here:
http://rubyforge.org/survey/survey.php?group_id=856&survey_id=21

A couple important things added to the survey ask about your OS and
database just so I can see at of curiosity what systems and databases I
should prioritize my support for (though with all luck Ruport will be
cross platform). I usually stick it to Windows users when in doubt,
but for Ruport this will not happen because the company that's helping
fund it is reliant on a Windows farm, so for once I'll cater to you M$
guys out there

I will try to stop soliciting Ruport now, though all of the discussion
it has generated has become addictive. Hope you like what I have to
offer come the 28th.