[ANN] Tattle - The Ruby Census

[Vincent Fourmond]

In most cases, the dependencies, both inside the Ruby infrastructure and
in the interactions between Ruby and the compilers and operating
systems, are identical -- the *only* thing that differs is the physical
format of the package and whether or not compilable language components
are pre-compiled or not. And at least for Linux, the same is true for
Perl, Python, PHP, Tcl/Tk, R, LaTeX and probably lots of others. A --
perhaps "standardized" is a better word than "universal" -- package
management system would free up developers and users to focus on
domain-specific value creation and not worry about packaging. It's all
automated anyhow. That's part of my complaint -- the numerous package
systems also consume developer resources in developing and maintaining
the automation infrastructure.

You seem to forget one thing: language-specific packagings are in
principle cross-platform (including macos,*nix,win...), which is
obviously not the case of distribution-specific packaging. You'll never
get a global packaging solution, just like no language will be the
ultimate solution for all programming.

Maybe what we're missing are good conversion tools ? Say, a gem2deb
program that would get most of the debian developer's job done before
the final touch ?

Vince

 

[Eric-Olivier Lamey]

Maybe what we're missing are good conversion tools ? Say, a gem2deb
program that would get most of the debian developer's job done before
the final touch ?

I don't know about debian, but David Lutterkort produced a nice
solution for gem to rpm conversion:
http://people.redhat.com/dlutter/gem2spec.html.

 

[Gregory Seidman]

There's no reason it *can't* ... all I am saying is that there are some
very brilliant people spending significant amounts of time packaging
valuable open source software in multiple formats, and I think that's
wrong.

When we speak of packaging systems we expect several features, among them:

1) anything that's installable comes in a single, downloadable file (e.g.
.gem, .deb, .rpm)
2) there is a way of discovering which package(s) are needed for specific
functionality
3) the package installation process, once initiated, results in fully
installed and ready to use functionality
4) package files identify other packages on which their functionality
depends
5) the installation process therefore retrieves the package we request and
automatically retrieves all packages on which it depends, and installs
them all

It's really easy to get #1 working, and to keep #1 working when converting
between packaging systems. There is the aforementioned gem2spec, which will
turn a RubyGem into an RPM. There is alien, which will convert between RPM,
deb, Stampede (no idea), and tarball (Slackware). The resulting package
will be installable on whatever target system, but that doesn't mean it
will fulfill any of the other features mentioned above.

Those of us who have been using Linux for enough years remember RPM Hell,
which has been largely (though not completely) solved with the advent of
apt-rpm, yum, YaST, and similar. The problem wasn't that the RPM format was
inherently bad as a package format, the problem was that there was no
clearly defined and rigorously adhered to naming convention for
dependencies, nor significant quality control around third-party RPMs.

Debian solved this problem by laying out strict naming guidelines, by
making it much easier for packages that would otherwise be third-party to
be included in the Debian repository, and by running automated tests on the
packages in the repository to enforce quality control. (I'm not claiming
it's perfect, but I don't believe any other packaging system/distribution
has anywhere near as many packages or developers working together such
that, in the stable distribution at least, installing a package will never
step on the toes of another package and will always be installed with all
of its dependencies.)

There's a lot of overhead in all that, and generally special-purpose tools
(or dedicated packagers) are involved in repackaging foreign packages for
use on a Debian system. One example of this is java-pakcage, which provides
the make-jpkg tool that will convert a binary installer downloaded from Sun
(or, on PPC, IBM) into an installable .deb with an appropriate name,
appropriate dependencies, and appropriate "provides" (as in, the package is
named jdk-XXX-whatever but it provides java-compiler and java-runtime, on
which other packages depend). I believe there is ongoing work to do the
same for gems. The escape hatch, of course, is to install it in /usr/local
and manage it separately, which is what I do and will continue to do until
a gem-package is created to make them play nice with the rest of Debian.

Most of the common gems, for example, are wrapped in Gentoo
ebuilds. A Gentoo developer, most likely a volunteer, does that, so on
my system, I can either type "gem install --remote rails" and add a line
for rails to "/etc/portage/package.provided", or "emerge -v rails".
There's probably someone else that has to do this for Debian, and for
Red Hat/Fedora/CentOS, etc. And when a new Ruby package comes out,
there's a lag as it makes its way through the gem system to a .deb, a
.rpm, a .ebuild, etc. In addition, because of the labors of one
individual, a large number of R packages are packaged for Debian but
nobody has volunteered to do the same for Red Hat or Gentoo formats.

[...]

The point I'm making above is twofold: first, a universal packaging system
requires a whole lot more than just a file format, it requires buy-in from
everyone involved on naming conventions and file placement and a variety of
other things; second, the best solution is not to have packagers doing the
repackaging, but to have someone create a tool for the target package
management system to convert packages from the foreign package management
system.

M. Edward (Ed) Borasky, FBG, AB, PTA, PGS, MS, MNLP, NST, ACMC(P)

--Greg

 

[M. Edward (Ed) Borasky]

The point I'm making above is twofold: first, a universal packaging system
requires a whole lot more than just a file format, it requires buy-in from
everyone involved on naming conventions and file placement and a variety of
other things;

Agreed ... it requires a lengthy standardization process -- a committee
of stakeholders, a project management infrastructure, *funding*, etc. At
a certain level, I thought that's what the Linux Standard Base was all
about. Maybe that's still going on and I haven't heard anything about it
recently. In any event it only initially requires "buy-in" from
dissatisfied "customers", or someone like the US Government to get the
ball rolling. Then it doesn't require "buy-in", it requires "stay-in",
which is *much* more difficult to acquire.

second, the best solution is not to have packagers doing the
repackaging, but to have someone create a tool for the target package
management system to convert packages from the foreign package management
system.

Yes, automation is the best *way* to do it. To minimize the amount of
effort in building the tools, a standardized target needs to exist and
the tools to convert *from* the dozens of open source package systems
*to* the standard are the only tools that should be built. I suspect
when all the smoke clears, it could play out like the ASCII-EBCDIC war
in the 1960s. A large vendor, IBM, invented EBCDIC and made it a "de
facto" standard, but the US government mandated that all the computers
it bought had to support ASCII. So IBM and the clones used EBCDIC and
offered tools to work with ASCII, and everyone else supported ASCII only.

By analogy, I can imagine Red Hat and the clones using RPM and some
humongous RPM source repository, but the US government mandating the
Debian format and repository for all the Linux systems it buys, etc.,
because I think the Debian way is technologically superior to the RPM
way. And Debian supports more architectures in binary format than Red Hat.

 

[M. Edward (Ed) Borasky]

Agreed ... it requires a lengthy standardization process -- a
committee of stakeholders, a project management infrastructure,
*funding*, etc. At a certain level, I thought that's what the Linux
Standard Base was all about. Maybe that's still going on and I haven't
heard anything about it recently. In any event it only initially
requires "buy-in" from dissatisfied "customers", or someone like the
US Government to get the ball rolling. Then it doesn't require
"buy-in", it requires "stay-in", which is *much* more difficult to
acquire.

The good news is: Linux Standard Base (LSB) is very much alive and well.
And there is more good news -- they just (December 2006) established a
working group on packaging. See
http://www.freestandards.org/en/Packaging and
http://ianmurdock.com/?p=388 for some links.

The bad Red Hat, SuSE and Debian are represented, Gentoo is not.
The other bad Perl and Python are represented, Ruby and R are not.

I'll go bug the Gentoo and R folks -- somebody here want to to represent
Ruby?