Ruby vs. PHP

[Marcin Raczkowski]

I don't think saying that PHP serves the customer better is such an open
and shut case. I also don't see how building a Ruby web framework from
scratch wastes customer dollars any more than building a PHP web
framework from scratch.

Based on what I've seen here, most of the arguments against doing what
the OP proposes are based on:

1. PHP is more popular, so getting more (on average, low quality)
programmers familiar with the technology is easier.

2. PHP is more popular, so migrating from one shared host to another is
easier (even though shared hosting probably isn't what's intended
here).

3. PHP is more popular, so managers are less likely to balk at it.

4. Rails already exists, so it's better to build a new framework than
to build a new framework. (One of those two is in PHP, the other in
Ruby.)

I'm just not buying any of that as technical reasons to choose PHP
instead of non-Rails Ruby.



. . . which is a great reason to use LAMP if that's the only way you're
going to get the contract. It's not such a great reason if you're
looking for the best technologies to use for a given project.



. . . even if it means using what's best, rather than what's most
popular.

Thank God someone that understands that are reasons why i want to use
ruby and mix of rails/custom framework, let's face it, php programmers
are usually not programmers at all.

Rails is great but heavy and slow.

I want to use best instead of most popular, and i asked you what's best
in ruby, and all i heard until this post was why php is more popular,
and CTO should have all the answers anyway. geez

 

[Marcin Raczkowski]

I'm jumping in late here, and I've only skimmed most of the replies,
so pardon me if I'm repeating anything already said:



On the other hand, there are lots of folks working on optimizing
Rails, how many are working on optimizing your custom tailored
framework.

If you don't want to use Rails, then why not look at something which
already has somewhat of a community like merb?

well, I'm still considering merb, in fact my custom framework borrowed
lot's from merb, but basic benchmarks on mostly used (by me at least)
features showed that my custom framework was 20 times faster then rails,
and 2-3 times faster then merb :]

i just wrote what i needed, not what i thought someone will need

So even though the population of Rails savvy programmers might be
smaller than the PHP community, there's a much higher probability that
any of then could pick up and existing Rails app than a random PHP
programmer could understand a random PHP application.

I had expirience with PHP "programmers" and let me just say they waren't
programmers, I remember one writing loop with use of two wariables, one
to count iterations and one to be set to true if iterations reach
certain level.

also due to fact that php is usually view centric, there is loots of
doubled code, no clear directory structure, sql queries inside views,
configuration scattered between 10-20 files, and some other juicy stuff

On the other hand, how many will understand your custom MVC framework?

anyone who knows how Ruby works and know MVC pattern?
I'm using Haml for views, Sequel or postgres for DB, directory layout is
really similar to rails, what's the problem ?

Of course if you're looking for job security AFTER you get the gig....

Currently, i have to put down job offers, I don't really need job
security, but i enjoy working with people i work with right now, and
fact that they let me do cool and new stuff, that's why i want to do it
for few more months instead of going back to PHP or C++ programming :]

 

[M. Edward (Ed) Borasky]

Amazon's hiring seems to focus more on Perl than any other single
language, actually -- and has for a long time. As such, while the amount
of dynamic language usage may actually be increasing, I think one might
better represent the situation by saying that Amazon has used dynamic
languages to a significant degree for a long time.

That's "long time" in Intarweb years, of course.

Two tidbits about Amazon:

1. I don't remember how I stumbled across it, but somewhere in the great
electronic beyond there is a blog posting by someone who describes the
hiring process at Amazon. One of the things they look for is someone who
can bang out a quick solution to massive text crunching problems using
regular expressions, either in Perl or more primitive command line
tools. I don't recall whether Python or Ruby were "acceptable".

2. A year or so ago there was an ACM conference on functional languages
here in Portland. I didn't go to the conference itself, but I did attend
a Saturday Erlang workshop and a Sunday Scheme workshop. At the Erlang
workshop, the small (30 or so) room at Portland State University was
packed to overflowing -- at least 60 jammed into it -- and at least a
dozen of them were from a couple of teams at Amazon, who were hiring.

 

[Chad Perrin]

2. A year or so ago there was an ACM conference on functional languages
here in Portland. I didn't go to the conference itself, but I did attend
a Saturday Erlang workshop and a Sunday Scheme workshop. At the Erlang
workshop, the small (30 or so) room at Portland State University was
packed to overflowing -- at least 60 jammed into it -- and at least a
dozen of them were from a couple of teams at Amazon, who were hiring.

Missed opportunities . . .

Actually, I like my non-corporate work lifestyle. If I was looking for a
corporate coder job, though, that'd be the sort of thing that might be
high on my list -- getting picked up by a team attending an Erlang
workshop. I'd be even more impressed if that happened at the Scheme
workshop.

 

[Julian Tarkhanov]

I'd say that PHP should have taken a hint from environment
variables if
it wanted to use those sigils more effectively -- by not using them
except in certain cases where they are being interpolated. They're
just
line noise *most of the time* in PHP.

To my understanding, ruby-talk is too noble a place to discuss the
syntax merits of PHP.

 

[Greg Donald]

To my understanding, ruby-talk is too noble a place to discuss the syntax
merits of PHP.

or the Java clone it's becoming.

 

[Eugene Kalenkovich]

Two tidbits about Amazon:

1. I don't remember how I stumbled across it, but somewhere in the great
electronic beyond there is a blog posting by someone who describes the
hiring process at Amazon. One of the things they look for is someone who
can bang out a quick solution to massive text crunching problems using
regular expressions, either in Perl or more primitive command line tools.
I don't recall whether Python or Ruby were "acceptable".

Sorry for OT, but I did not start it . The intervew question for quick
text crunching came from the same SteveY. His huge productivity in writing
blogs has a significant impact on any company he works for .
What you use at the interview - perl, python, ruby or [put here whatever you
want] - does not make any difference, as long as you use it right (and have
in your backpack at least one of more conventional OO languages too)

-- EK

 

[M. Edward (Ed) Borasky]

Missed opportunities . . .

Actually, I like my non-corporate work lifestyle. If I was looking for a
corporate coder job, though, that'd be the sort of thing that might be
high on my list -- getting picked up by a team attending an Erlang
workshop. I'd be even more impressed if that happened at the Scheme
workshop.

Well ... I must say I enjoyed the Scheme workshop, but, yes, Scheme is
probably always going to be a lab rat rather than a workhorse. There
were one or two papers from people with "industrial-strength"
applications written in Scheme, though.

I'm so happy I get paid to program in R -- it's really one of the better
languages out there, although if you don't do number crunching or
statistics, there isn't much motivation to learn it. It's a nice
functional language, lexically scoped, and is well suited to processing
arrays of data with small amounts of code. It's what Lisp should have
been. Or rather, S is what Lisp should have been and R is a lexically
scoped dialect of S.

 

[Isaac Gouy]

=3Drub...


i found this one, and also it's interprete ruby not YARV or JRbuy

Yes it does!

Core Ruby 1.9.0 and JRuby 1.0

http://shootout.alioth.debian.org/gp4sandbox/index.php

 

[M. Edward (Ed) Borasky]

This has been discussed a lot on this list, and I believe that a
majority of the community is concerning the shootout site a site
without any value.
For sure I do.

Cheers
Robert

I guess that depends on how you define "value" and "majority". As a
practicing performance engineer, here's my view on the subject:

1. The site is very clear about what it is doing, how it is doing it,
why it is doing it and to a great extent how one can (not) and should
(not) interpret the results. However, anyone is free to misinterpret any
of that out of either ignorance or competitive marketing zeal.

2. If you actually look at the benchmark timings for the three main Ruby
implementations tested, jRuby, Ruby 1.8.6 and Ruby 1.9.x, you'll find
that they give roughly the same comparative speeds on these benchmarks
as they do on the virtual machine benchmarks bundled into the Ruby 1.9.x
code. In fact, I think some of them are in both sets. This is a good sign.

3. Given 2, one can be (and I am) encouraged to go farther and use the
site to compare implementations of other languages with implementations
of Ruby. I have not done a recent comprehensive analysis, but if the raw
data can be downloaded in a form suitable for it, I'll take a stab at it.

4. I suspect the main problem the "majority of the community" has with
the site is that it shows that the current (MRI) Ruby 1.8.6
implementation is slower than other languages commonly used to implement
web applications. And that is mostly competitive marketing zeal. Of
course the Ruby community wants Ruby to be faster than Perl, Python, PHP
or Java. And when it turns out that it isn't faster, there are two
things that can be done:

(a) Argue about how much more productive Ruby is, saying that you can
throw hardware at the scaling problem with the dollars you save
on programmers, and/or

(b) Make faster implementations of Ruby.

Neither of these options is "right" or "wrong" IMHO. For (a), I claim
that the economics of programmer productivity and the economics of web
application deployment and scaling are in fact two very different
studies. You can get degrees in either and earn a comfortable living
doing either. I've spent more time on the second because I think it's
more fun.

For (b), I think the results of Ruby 1.9.x (KRI) and jRuby speak for
themselves, and at some point in the not-too-distant future we should
have some IronRuby and Rubinius numbers as well. And perhaps the Parrot
effort will change the playing field for Perl, Python, PHP *and* Ruby in
one bold beak-stroke!

 

[Chad Perrin]

1. The site is very clear about what it is doing, how it is doing it,
why it is doing it and to a great extent how one can (not) and should
(not) interpret the results. However, anyone is free to misinterpret any
of that out of either ignorance or competitive marketing zeal.

I agree. The site is useful -- just not in the immediately obvious way.
For that purpose, it's not useful (where "useful" includes "good at
providing strong technical evidence"). That's because the "immediately
obvious way" is in comparing languages to decide what should be used for
performance purposes between languages that are very close on the
execution performance scale, and micro-benchmarks are notoriously bad at
that, for a number of reasons.

4. I suspect the main problem the "majority of the community" has with
the site is that it shows that the current (MRI) Ruby 1.8.6
implementation is slower than other languages commonly used to implement
web applications. And that is mostly competitive marketing zeal. Of
course the Ruby community wants Ruby to be faster than Perl, Python, PHP
or Java. And when it turns out that it isn't faster, there are two
things that can be done:

I don't think that's entirely the case (or even close-to-entirely). I
don't just participate in the Ruby community -- I'm as much an on-again,
off-again member of a couple other language communities as I am of the
Ruby community here at ruby-talk (if not more so). One in particular is
the Perl community, and just to make a point I'll compare the two on this
subject.

When people in each community look at comparative benchmarks for Ruby and
Perl, and when they see that Perl fairly consistently kicks the crap out
of Ruby for execution speed, I see that in *both* communities they tend
to follow that up with something like ". . . but micro-benchmarks are
pretty much useless. Only real-world performance testing matters, and
usually even that is just a case of premature optimization." In other
words, considering that even the usual "winner" in such a "contest"
dismisses those benchmarks as mostly useless in that respect, I don't
think the problem people in the "losing" camp have with the way people
tend to want to use benchmarks is primarily one of wanting to ignore what
shines unfavorable light in their direction.

I see the same things being said by Perlists and Rubyists: that
programmer productivity, flexibility, "real-world" performance,
maintainability, and other factors are more important. In either camp,
people then tend to provide evidence that their chosen language is the
best for these factors in some way -- at least for the purposes of the
person talking. As someone who likes and uses both Perl and Ruby, I find
that both "sides" are right, too. They're just right for differet use
cases.

 

[Marc Heiler]

And perhaps the Parrot

effort will change the playing field for Perl, Python, PHP *and*
Ruby in one bold beak-stroke!

I can hear the ice crunching and breaking under your feet
already ;P

 

[M. Edward (Ed) Borasky]

I agree. The site is useful -- just not in the immediately obvious way.
For that purpose, it's not useful (where "useful" includes "good at
providing strong technical evidence"). That's because the "immediately
obvious way" is in comparing languages to decide what should be used for
performance purposes between languages that are very close on the
execution performance scale, and micro-benchmarks are notoriously bad at
that, for a number of reasons.

Well ...

1. The site compares programming languages using micro-benchmarks, or,
more accurately, "benchmarks small enough to be easily implemented using
a variety of languages and covering a variety of commonly-used data
types and algorithms."

2. I've asked the Shootout team for a spreadsheet of all the timings for
all the languages, and if I get it, I'll post an analysis here (relative
to MRI) and on the Shootout forum (relative to gcc).

3. I define "useful" the same way the Shootout defines it -- a way of
comparing languages on a common set of benchmarks. If you have a problem
with the Shootout, take it up with the Shootout, not with me.

2.

I don't think that's entirely the case (or even close-to-entirely). I
don't just participate in the Ruby community -- I'm as much an on-again,
off-again member of a couple other language communities as I am of the
Ruby community here at ruby-talk (if not more so). One in particular is
the Perl community, and just to make a point I'll compare the two on this
subject.

When people in each community look at comparative benchmarks for Ruby and
Perl, and when they see that Perl fairly consistently kicks the crap out
of Ruby for execution speed, I see that in *both* communities they tend
to follow that up with something like ". . . but micro-benchmarks are
pretty much useless. Only real-world performance testing matters, and
usually even that is just a case of premature optimization." In other
words, considering that even the usual "winner" in such a "contest"
dismisses those benchmarks as mostly useless in that respect, I don't
think the problem people in the "losing" camp have with the way people
tend to want to use benchmarks is primarily one of wanting to ignore what
shines unfavorable light in their direction.

That's an easy way to dismiss the results, but you're assuming

1. That micro-benchmarks are pretty much useless. I don't agree.
Processors, compilers, interpreters, virtual machines and languages can
only be measured and optimized using micro-benchmarks.

Consider Intel (or AMD, if you prefer). Could they measure *every*
real-world application, ranging from reading email to web servers to
scientific matrix algorithms to video editing to multi-player shootouts
to text indexing to ... with *every* commonly-used compiler and
interpreter? Of course not!

You *have* to reduce the benchmark set to something reasonable and make
some simplifying assumptions. And in the end, you have to have some
*statistical* way of saying that a chip design is "good enough", not
that it is perfect. I don't call that "premature optimization", I call
it PhD-level electrical engineering!

2. That competition doesn't matter. Again, I don't agree. If it doesn't
matter, why do we do it? We long ago stopped racing horses because it
"improved the breed" and continue to race horses only because it is a
source of entertainment, and of tax revenue in some jurisdictions. But
we race computer chips and languages and algorithms because it *does*
improve the breed.

Maybe in the 60 or so years since computer racing started, we're
starting to reach the point where it isn't improving the breed any more
and is only a source of entertainment. But we still don't have a
definitive answer to whether P = NP, we have "grand challenge" problems
that call for peta-scale computing platforms and languages to make such
problems easily solved, there appears to be a limitless market for
devices that put the entire historical collection of recorded music in
one's pocket, and so on.

To sum up, as a practicing performance engineer, I can say that there
are ways to go from micro-benchmark results to "good enough but not
perfect" predictions of real-world performance. That's why we do it and
why it matters in a nutshell.

I see the same things being said by Perlists and Rubyists: that
programmer productivity, flexibility, "real-world" performance,
maintainability, and other factors are more important. In either camp,
people then tend to provide evidence that their chosen language is the
best for these factors in some way -- at least for the purposes of the
person talking. As someone who likes and uses both Perl and Ruby, I find
that both "sides" are right, too. They're just right for differet use
cases.

Well ... in the specific case of Perl vs. Ruby, I'm not by any means an
expert on the use cases that differentiate between the two. Ruby is by
design a *successor* to Perl, IIRC. So I would expect Ruby to provide a
better "user experience" than Perl for a lot of use cases.

But I don't see how it's possible that language/implementation A can be
significantly faster than language/implementation B on a varied set of
micro-benchmarks and yet language/implementation B can be significantly
faster than language/implementation A on more than, say, *one*,
real-world application. Computers and mathematics just don't work that way.

 

[Chad Perrin]

That's an easy way to dismiss the results, but you're assuming

1. That micro-benchmarks are pretty much useless. I don't agree.
Processors, compilers, interpreters, virtual machines and languages can
only be measured and optimized using micro-benchmarks.

No, I'm not assuming that. I'm pointing out that objections to using
those micro-benchmarks to choose a language to use for general purpose
programming are common, and that I find them agreeable. Micro-benchmarks
are *very* useful for determining problem areas in execution performance,
if you happen to be on the core implementation maitenance team, but
that's not what I was talking about when I said that as an overall
performance measure for purposes of choosing a language they aren't so
useful.

In other words, the problem is that you're assuming that I'm assuming
something that I am not, in fact, assuming. Put otherwise, I basically
said that for the purpose to which most people seem tempted to put those
micro-benchmarks when they drag them out while trolling, they're useless
in any practical sense -- and you generalized from there to assume that I
was saying they're never useful for anything.

2. That competition doesn't matter. Again, I don't agree. If it doesn't
matter, why do we do it? We long ago stopped racing horses because it
"improved the breed" and continue to race horses only because it is a
source of entertainment, and of tax revenue in some jurisdictions. But
we race computer chips and languages and algorithms because it *does*
improve the breed.

Again, you're putting words in my mouth. I didn't say competition
doesn't matter. What I suggested is that "my interpreter is faster than
yours" competition based on micro-benchmarks is pointless for languages
in the same general neighborhood of execution performance, and that
usually other concerns are *far* more important within a given
neighborhood of performance than execution speed. If you want faster
execution speed, you should be in a completely different performance
neighborhood, not quibbling over whether Ruby or Python is faster for an
arbitrary set of micro-benchmarks, in other words.

To sum up, as a practicing performance engineer, I can say that there
are ways to go from micro-benchmark results to "good enough but not
perfect" predictions of real-world performance. That's why we do it and
why it matters in a nutshell.

Yes, we can -- but the margin for error is so great that, again, petty
microsecond disputes between Perl, Python, and Ruby are just that: petty.
I say this despite the fact that, overall, Perl kicks the tuckus of both
these other close competitors -- and Perl is one of my favorite
languages. I would have said roughly the same thing even before I
started using Ruby.

Well ... in the specific case of Perl vs. Ruby, I'm not by any means an
expert on the use cases that differentiate between the two. Ruby is by
design a *successor* to Perl, IIRC. So I would expect Ruby to provide a
better "user experience" than Perl for a lot of use cases.

I don't know if it was intended that way, but I certainly don't view it
that way. They are sufficiently different that they are simply
*different* languages, not *successive* languages, in practice. Granted,
I'd definitely be willing to use Ruby as Perl's successor within the
specific niche of "object oriented programming", since Perl's OO syntax
is so onerous to deal with (by comparison), but OOP is not the be-all and
end-all of programming (the last fifteen years of Java industry
domination notwithstanding).

But I don't see how it's possible that language/implementation A can be
significantly faster than language/implementation B on a varied set of
micro-benchmarks and yet language/implementation B can be significantly
faster than language/implementation A on more than, say, *one*,
real-world application. Computers and mathematics just don't work that way.

Who said anything about "significantly faster"? I'm just talking about
the fact that Perl consistently kicking Ruby's tail in performance
benchmarks doesn't mean a whole lot when choosing languages for projects,
generally speaking, since if performance is enough of a concern to
override the other factors involved you should be using something like C,
Objective Caml, or at least compiled Lisp instead. In other words, if
you're trying to decide between Perl and Ruby based on performance
micro-benchmarks, you might as well be choosing between two paint shops
for your car's new coat of paint based on which one has a better record
for using paint that provides a finish with a lower drag coefficient.
Obviously, it's such a miniscule difference in the grand scheme of things
that you're better of deciding based on whether you like the color,
whether a given shop's paint jobs tend to last longer, how much it'll
cost you at one shop vs. the other, and so on.

 

[M. Edward (Ed) Borasky]

No, I'm not assuming that. I'm pointing out that objections to using
those micro-benchmarks to choose a language to use for general purpose
programming are common, and that I find them agreeable. Micro-benchmarks
are *very* useful for determining problem areas in execution performance,
if you happen to be on the core implementation maitenance team, but
that's not what I was talking about when I said that as an overall
performance measure for purposes of choosing a language they aren't so
useful.

Well ... OK ... you agree with me that they are useful for core
implementation. And I *still* think they're useful when comparing
performance of languages for general purpose programming, provided the
benchmark suite is comprehensive and the statistical comparison
methodology is correct.

I further think that the Alioth Shootout Suite is comprehensive enough
to distinguish between the performance of those languages for which
*most* of the benchmarks actually ran. The trick is the statistical
methodology you use.

Ideally, of course, you'd have a benchmark suite that matches the
application set. *That's* where people get into trouble with benchmark
suites -- they assume that a given suite, for example, the SPEC integer
suite, is representative of their application.

Now I could look at the benchmarks themselves and throw out those that
were obviously orthogonal to web servers when comparing languages for
implementing web servers. But I don't think I need to go that far --
it's easier to just throw out the statistical outliers and come up with
a set of numbers that's a general characterization of the relative
performance of the languages. That's why I asked for the raw data (which
I received, by the way).

What I suggested is that "my interpreter is faster than
yours" competition based on micro-benchmarks is pointless for languages
in the same general neighborhood of execution performance, and that
usually other concerns are *far* more important within a given
neighborhood of performance than execution speed. If you want faster
execution speed, you should be in a completely different performance
neighborhood, not quibbling over whether Ruby or Python is faster for an
arbitrary set of micro-benchmarks, in other words.

Of course. I'm still in the process of crunching the numbers, but what
I've seen so far is that MRI Ruby is *not* in the general neighborhood
of Perl, Python or PHP, but YARV *is* in that neighborhood. I do need to
tighten up the calculations slightly and get rid of some bogus points,
but I'm not expecting a major change in that.

As for things in the "fast lane", I started with gcc, and there are only
a few things faster than gcc. The one that jumped out and bit me on the
nose was a *functional* language called "clean". I'm just about to
download it and see what makes it tick (or to stretch the analogy, how
tightly it's wound). And the slow lane has such snails as "groovy" and
"rebol" lumbering away.

Yes, we can -- but the margin for error is so great that, again, petty
microsecond disputes between Perl, Python, and Ruby are just that: petty.
I say this despite the fact that, overall, Perl kicks the tuckus of both
these other close competitors -- and Perl is one of my favorite
languages. I would have said roughly the same thing even before I
started using Ruby.

Well ... let me finish with the numbers, but I think you'll be surprised
at how close Perl and Python are to each other and how far away MRI is
from the two of them. It's not a "petty microsecond dispute" for MRI.

Who said anything about "significantly faster"? I'm just talking about
the fact that Perl consistently kicking Ruby's tail in performance
benchmarks doesn't mean a whole lot when choosing languages for projects,
generally speaking, since if performance is enough of a concern to
override the other factors involved you should be using something like C,
Objective Caml, or at least compiled Lisp instead.

Of course. But if the choice is between "dynamic languages with a
sufficient supply of competent developers available", for example, Perl,
Python, PHP and Ruby, and the assumption is that they are equally
productive -- you're just trying to assess how much hardware you're
going to have to throw at it if your customer growth takes off -- you're
going to want the fastest one. Or, more to the point, you're probably
going to dismiss the slowest one and move on to the other decision factors.

 

[M. Edward (Ed) Borasky]

--------------090700090302050407010606
Content-Type: text/plain; charset=ISO-8859-1; format=flowed
Content-Transfer-Encoding: 7bit

Now I could look at the benchmarks themselves and throw out those that
were obviously orthogonal to web servers when comparing languages for
implementing web servers. But I don't think I need to go that far --
it's easier to just throw out the statistical outliers and come up with
a set of numbers that's a general characterization of the relative
performance of the languages. That's why I asked for the raw data (which
I received, by the way).
[snip]

Of course. I'm still in the process of crunching the numbers, but what
I've seen so far is that MRI Ruby is *not* in the general neighborhood
of Perl, Python or PHP, but YARV *is* in that neighborhood. I do need to
tighten up the calculations slightly and get rid of some bogus points,
but I'm not expecting a major change in that.
[snip]

Well ... let me finish with the numbers, but I think you'll be surprised
at how close Perl and Python are to each other and how far away MRI is
from the two of them. It's not a "petty microsecond dispute" for MRI.

[snip]

OK ... here's the analysis. The attached PDF is what's known as a box
and whisker plot, usually shortened to "boxplot". The raw numbers that
went into this are from the Alioth shootout page, and what I *haven't*
done is checked out which versions of Perl, Python, YARV, Ruby, jRuby
and PHP these tests used. They could be years old or they could have
been run yesterday morning. I discarded all the tests for which there
were any missing values.

The value plotted is the ratio of seconds for the benchmark on the
dynamic language to the seconds for the benchmark with gcc. Thus, gcc
equals 1.0 across the board and lower is better/faster. How do you
interpret these plots?

First of all, the bar across the middle is the median. Half the values
are above the median and half below it. I sorted them by increasing
median, so gcc, being the fastest, is on the left. Next comes YARV, then
Python, Perl, PHP, Ruby and jRuby.

The whisker on the bottom is approximately the 5th percentile. In other
words, only five percent of the time is your performance going to be
that good or better. And the whisker on the top is approximately the
95th percentile -- only five percent of the time is it going to be that
bad or worse.

The bottom of the box is the 25th percentile. 25 percent of the time,
the performance will be that good or better. The top of the box is the
75th percentile. 25 percent of the time it will be that bad or worse.

So now we can make more precise Chad Perrin's notion of "general
neighborhoods of performance". We do this by looking at the height of
the median and the width of the box. And we see that relative to gcc,
YARV, Python, Perl and PHP have fairly close medians and the boxes are
about the same width. So they are all "in the same neighborhood" and gcc
is faster. Ruby and jRuby are *not* in the same neighborhood.

Now suppose your boss comes to you, as bosses do, and says, "well, all
them high-falutin' box plots are dandy, but the board of directors wants
one number for each language!" It turns out (and I'll let Google and
Wikipedia fill in the blanks for you) that the one number you want to
give your boss, aside from your cell phone number, is the *geometric
mean* of all the benchmark ratios. Again, smaller is better. So here's
how the languages stack up:

gcc 1.0
yarv 13.2
perl 14.0
python 14.2
php 15.5
ruby 29.9
jruby 55.0

So yes, you can pretty much expect the same performance from YARV, Perl,
Python and PHP. And you can pretty much expect something like a 13 - 16
to one speed improvement if you decide to rewrite your application in C.

It's pretty clear to me from these numbers is that the only reason that
deploying web applications on the LAMP stack and its cousins using
PostgreSQL, Perl, Python and Ruby is economically viable is that they
spend most of their time either in the database or on the network. The
bad news is that for an application with 100 percent dynamic language
code -- no C libraries for the intensive calculations, no highly-tuned
web servers, databases or other components -- you're going to end up
throwing twice as much hardware at scaling problems in Ruby as you will
in PHP, Perl or Python. The good news is that YARV will level the
playing field.

One final note to implementers and language designers ... Python gets an
extra little pat on the back from me for having such a low spread. YARV
and other Ruby implementations need to pay attention to the fact that
their boxes are wider than Perl's and PHP's and a *lot* wider than
Python's. In other words, look at the benchmarks where you really suck
first.

In case you want the numbers that go with the boxplots, here they are:

gcc yarv python perl php ruby jruby
Low 1 1.2 1.4 0.93 1.4 1.5 3.4
Q1 1 4.8 4.6 2.90 3.1 6.3 12.0
Median 1 8.7 14.0 26.00 31.0 34.0 50.0
Q3 1 68.0 45.0 55.00 55.0 170.0 340.0
High 1 150.0 98.0 67.00 110.0 380.0 410.0

--------------090700090302050407010606
Content-Type: application/pdf;
name="shootout.pdf"
Content-Transfer-Encoding: base64
Content-Disposition: inline;
filename="shootout.pdf"

JVBERi0xLjEKJYHigeOBz4HTXHIKMSAwIG9iago8PAovQ3JlYXRpb25EYXRlIChEOjIwMDcx
MDA2MTY1MzA0KQovTW9kRGF0ZSAoRDoyMDA3MTAwNjE2NTMwNCkKL1RpdGxlIChSIEdyYXBo
aWNzIE91dHB1dCkKL1Byb2R1Y2VyIChSIDIuNi4wKQovQ3JlYXRvciAoUikKPj4KZW5kb2Jq
CjIgMCBvYmoKPDwKL1R5cGUgL0NhdGFsb2cKL1BhZ2VzIDMgMCBSCj4+CmVuZG9iago1IDAg
b2JqCjw8Ci9UeXBlIC9Gb250Ci9TdWJ0eXBlIC9UeXBlMQovTmFtZSAvRjEKL0Jhc2VGb250
IC9aYXBmRGluZ2JhdHMKPj4KZW5kb2JqCjYgMCBvYmoKPDwKL1R5cGUgL1BhZ2UKL1BhcmVu
dCAzIDAgUgovQ29udGVudHMgNyAwIFIKL1Jlc291cmNlcyA0IDAgUgo+PgplbmRvYmoKNyAw
IG9iago8PAovTGVuZ3RoIDggMCBSCj4+CnN0cmVhbQ0KcQpRIHEgNTkuMDQgNzMuNDQgNjMw
LjcyIDQwNy41MiByZSBXIG4KMC4wMDAgMC4wMDAgMC4wMDAgUkcKMi4yNSB3CltdIDAgZAox
IEoKMSBqCjEwLjAwIE0KOTAuNzQgODkuMjkgbSAxNTcuNDkgODkuMjkgbCBTCjAuNzUgdwpb
IDMuMDAgNS4wMF0gMCBkCjEyNC4xMSA4OS4yOSBtIDEyNC4xMSA4OS4yOSBsIFMKMTI0LjEx
IDg5LjI5IG0gMTI0LjExIDg5LjI5IGwgUwowLjc1IHcKW10gMCBkCjEwNy40MyA4OS4yOSBt
IDE0MC44MCA4OS4yOSBsIFMKMTA3LjQzIDg5LjI5IG0gMTQwLjgwIDg5LjI5IGwgUwoxMDcu
NDMgODkuMjkgbSAxNDAuODAgODkuMjkgbCBTCjEwNy40MyA4OS4yOSBtIDE0MC44MCA4OS4y
OSBsIFMKMTA3LjQzIDg5LjI5IG0gMTQwLjgwIDg5LjI5IGwgUwoxMDcuNDMgODkuMjkgbSAx
NDAuODAgODkuMjkgbCBTCjEwNy40MyA4OS4yOSBtIDE0MC44MCA4OS4yOSBsIFMKMTA3LjQz
IDg5LjI5IG0gMTQwLjgwIDg5LjI5IGwgUwoxMDcuNDMgODkuMjkgbSAxNDAuODAgODkuMjkg
bCBTCjEwNy40MyA4OS4yOSBtIDE0MC44MCA4OS4yOSBsIFMKMTA3LjQzIDg5LjI5IG0gMTQw
LjgwIDg5LjI5IGwgUwoxMDcuNDMgODkuMjkgbSAxNDAuODAgODkuMjkgbCBTCjEwNy40MyA4
OS4yOSBtIDE0MC44MCA4OS4yOSBsIFMKMTA3LjQzIDg5LjI5IG0gMTQwLjgwIDg5LjI5IGwg
Uwo5MC43NCA4OS4yOSBtCjE1Ny40OSA4OS4yOSBsCjE1Ny40OSA4OS4yOSBsCjkwLjc0IDg5
LjI5IGwKOTAuNzQgODkuMjkgbApTCjIuMjUgdwpbXSAwIGQKMTc0LjE3IDk1LjEyIG0gMjQw
LjkxIDk1LjEyIGwgUwowLjc1IHcKWyAzLjAwIDUuMDBdIDAgZAoyMDcuNTQgODkuNDggbSAy
MDcuNTQgOTIuMTQgbCBTCjIwNy41NCAyMDAuODAgbSAyMDcuNTQgMTM5Ljk4IGwgUwowLjc1
IHcKW10gMCBkCjE5MC44NiA4OS40OCBtIDIyNC4yMyA4OS40OCBsIFMKMTkwLjg2IDIwMC44
MCBtIDIyNC4yMyAyMDAuODAgbCBTCjE5MC44NiA4OS40OCBtIDIyNC4yMyA4OS40OCBsIFMK
MTkwLjg2IDIwMC44MCBtIDIyNC4yMyAyMDAuODAgbCBTCjE5MC44NiA4OS40OCBtIDIyNC4y
MyA4OS40OCBsIFMKMTkwLjg2IDIwMC44MCBtIDIyNC4yMyAyMDAuODAgbCBTCjE5MC44NiA4
OS40OCBtIDIyNC4yMyA4OS40OCBsIFMKMTkwLjg2IDIwMC44MCBtIDIyNC4yMyAyMDAuODAg
bCBTCjE5MC44NiA4OS40OCBtIDIyNC4yMyA4OS40OCBsIFMKMTkwLjg2IDIwMC44MCBtIDIy
NC4yMyAyMDAuODAgbCBTCjE5MC44NiA4OS40OCBtIDIyNC4yMyA4OS40OCBsIFMKMTkwLjg2
IDIwMC44MCBtIDIyNC4yMyAyMDAuODAgbCBTCjE5MC44NiA4OS40OCBtIDIyNC4yMyA4OS40
OCBsIFMKMTkwLjg2IDIwMC44MCBtIDIyNC4yMyAyMDAuODAgbCBTCjE3NC4xNyA5Mi4xNCBt
CjI0MC45MSA5Mi4xNCBsCjI0MC45MSAxMzkuOTggbAoxNzQuMTcgMTM5Ljk4IGwKMTc0LjE3
IDkyLjE0IGwKUwoyLjI1IHcKW10gMCBkCjI1Ny42MCA5OS4wMSBtIDMyNC4zNCA5OS4wMSBs
IFMKMC43NSB3ClsgMy4wMCA1LjAwXSAwIGQKMjkwLjk3IDg5LjYxIG0gMjkwLjk3IDkxLjk4
IGwgUwoyOTAuOTcgMTYyLjQwIG0gMjkwLjk3IDEyMi4yMCBsIFMKMC43NSB3CltdIDAgZAoy
NzQuMjkgODkuNjEgbSAzMDcuNjYgODkuNjEgbCBTCjI3NC4yOSAxNjIuNDAgbSAzMDcuNjYg
MTYyLjQwIGwgUwoyNzQuMjkgODkuNjEgbSAzMDcuNjYgODkuNjEgbCBTCjI3NC4yOSAxNjIu
NDAgbSAzMDcuNjYgMTYyLjQwIGwgUwoyNzQuMjkgODkuNjEgbSAzMDcuNjYgODkuNjEgbCBT
CjI3NC4yOSAxNjIuNDAgbSAzMDcuNjYgMTYyLjQwIGwgUwoyNzQuMjkgODkuNjEgbSAzMDcu
NjYgODkuNjEgbCBTCjI3NC4yOSAxNjIuNDAgbSAzMDcuNjYgMTYyLjQwIGwgUwoyNzQuMjkg
ODkuNjEgbSAzMDcuNjYgODkuNjEgbCBTCjI3NC4yOSAxNjIuNDAgbSAzMDcuNjYgMTYyLjQw
IGwgUwoyNzQuMjkgODkuNjEgbSAzMDcuNjYgODkuNjEgbCBTCjI3NC4yOSAxNjIuNDAgbSAz
MDcuNjYgMTYyLjQwIGwgUwoyNzQuMjkgODkuNjEgbSAzMDcuNjYgODkuNjEgbCBTCjI3NC4y
OSAxNjIuNDAgbSAzMDcuNjYgMTYyLjQwIGwgUwoyNTcuNjAgOTEuOTggbQozMjQuMzQgOTEu
OTggbAozMjQuMzQgMTIyLjIwIGwKMjU3LjYwIDEyMi4yMCBsCjI1Ny42MCA5MS45OCBsClMK
Mi4yNSB3CltdIDAgZAozNDEuMDMgMTA4LjI4IG0gNDA3Ljc3IDEwOC4yOCBsIFMKMC43NSB3
ClsgMy4wMCA1LjAwXSAwIGQKMzc0LjQwIDg5LjIzIG0gMzc0LjQwIDkwLjcwIGwgUwozNzQu
NDAgMTM5LjI5IG0gMzc0LjQwIDEzMC4xNCBsIFMKMC43NSB3CltdIDAgZAozNTcuNzEgODku
MjMgbSAzOTEuMDkgODkuMjMgbCBTCjM1Ny43MSAxMzkuMjkgbSAzOTEuMDkgMTM5LjI5IGwg
UwozNTcuNzEgODkuMjMgbSAzOTEuMDkgODkuMjMgbCBTCjM1Ny43MSAxMzkuMjkgbSAzOTEu
MDkgMTM5LjI5IGwgUwozNTcuNzEgODkuMjMgbSAzOTEuMDkgODkuMjMgbCBTCjM1Ny43MSAx
MzkuMjkgbSAzOTEuMDkgMTM5LjI5IGwgUwozNTcuNzEgODkuMjMgbSAzOTEuMDkgODkuMjMg
bCBTCjM1Ny43MSAxMzkuMjkgbSAzOTEuMDkgMTM5LjI5IGwgUwozNTcuNzEgODkuMjMgbSAz
OTEuMDkgODkuMjMgbCBTCjM1Ny43MSAxMzkuMjkgbSAzOTEuMDkgMTM5LjI5IGwgUwozNTcu
NzEgODkuMjMgbSAzOTEuMDkgODkuMjMgbCBTCjM1Ny43MSAxMzkuMjkgbSAzOTEuMDkgMTM5
LjI5IGwgUwozNTcuNzEgODkuMjMgbSAzOTEuMDkgODkuMjMgbCBTCjM1Ny43MSAxMzkuMjkg
bSAzOTEuMDkgMTM5LjI5IGwgUwozNDEuMDMgOTAuNzAgbQo0MDcuNzcgOTAuNzAgbAo0MDcu
NzcgMTMwLjE0IGwKMzQxLjAzIDEzMC4xNCBsCjM0MS4wMyA5MC43MCBsClMKMi4yNSB3Cltd
IDAgZAo0MjQuNDYgMTExLjc1IG0gNDkxLjIwIDExMS43NSBsIFMKMC43NSB3ClsgMy4wMCA1
LjAwXSAwIGQKNDU3LjgzIDg5LjYxIG0gNDU3LjgzIDkwLjg0IGwgUwo0NTcuODMgMTcxLjY0
IG0gNDU3LjgzIDEyOS43NCBsIFMKMC43NSB3CltdIDAgZAo0NDEuMTQgODkuNjEgbSA0NzQu
NTEgODkuNjEgbCBTCjQ0MS4xNCAxNzEuNjQgbSA0NzQuNTEgMTcxLjY0IGwgUwo0NDEuMTQg
ODkuNjEgbSA0NzQuNTEgODkuNjEgbCBTCjQ0MS4xNCAxNzEuNjQgbSA0NzQuNTEgMTcxLjY0
IGwgUwo0NDEuMTQgODkuNjEgbSA0NzQuNTEgODkuNjEgbCBTCjQ0MS4xNCAxNzEuNjQgbSA0
NzQuNTEgMTcxLjY0IGwgUwo0NDEuMTQgODkuNjEgbSA0NzQuNTEgODkuNjEgbCBTCjQ0MS4x
NCAxNzEuNjQgbSA0NzQuNTEgMTcxLjY0IGwgUwo0NDEuMTQgODkuNjEgbSA0NzQuNTEgODku
NjEgbCBTCjQ0MS4xNCAxNzEuNjQgbSA0NzQuNTEgMTcxLjY0IGwgUwo0NDEuMTQgODkuNjEg
bSA0NzQuNTEgODkuNjEgbCBTCjQ0MS4xNCAxNzEuNjQgbSA0NzQuNTEgMTcxLjY0IGwgUwo0
NDEuMTQgODkuNjEgbSA0NzQuNTEgODkuNjEgbCBTCjQ0MS4xNCAxNzEuNjQgbSA0NzQuNTEg
MTcxLjY0IGwgUwo0MjQuNDYgOTAuODQgbQo0OTEuMjAgOTAuODQgbAo0OTEuMjAgMTI5Ljc0
IGwKNDI0LjQ2IDEyOS43NCBsCjQyNC40NiA5MC44NCBsClMKMi4yNSB3CltdIDAgZAo1MDcu
ODkgMTEzLjkwIG0gNTc0LjYzIDExMy45MCBsIFMKMC43NSB3ClsgMy4wMCA1LjAwXSAwIGQK
NTQxLjI2IDg5LjYzIG0gNTQxLjI2IDkzLjMwIGwgUwo1NDEuMjYgMzc1LjA3IG0gNTQxLjI2
IDIxNC44OCBsIFMKMC43NSB3CltdIDAgZAo1MjQuNTcgODkuNjMgbSA1NTcuOTQgODkuNjMg
bCBTCjUyNC41NyAzNzUuMDcgbSA1NTcuOTQgMzc1LjA3IGwgUwo1MjQuNTcgODkuNjMgbSA1
NTcuOTQgODkuNjMgbCBTCjUyNC41NyAzNzUuMDcgbSA1NTcuOTQgMzc1LjA3IGwgUwo1MjQu
NTcgODkuNjMgbSA1NTcuOTQgODkuNjMgbCBTCjUyNC41NyAzNzUuMDcgbSA1NTcuOTQgMzc1
LjA3IGwgUwo1MjQuNTcgODkuNjMgbSA1NTcuOTQgODkuNjMgbCBTCjUyNC41NyAzNzUuMDcg
bSA1NTcuOTQgMzc1LjA3IGwgUwo1MjQuNTcgODkuNjMgbSA1NTcuOTQgODkuNjMgbCBTCjUy
NC41NyAzNzUuMDcgbSA1NTcuOTQgMzc1LjA3IGwgUwo1MjQuNTcgODkuNjMgbSA1NTcuOTQg
ODkuNjMgbCBTCjUyNC41NyAzNzUuMDcgbSA1NTcuOTQgMzc1LjA3IGwgUwo1MjQuNTcgODku
NjMgbSA1NTcuOTQgODkuNjMgbCBTCjUyNC41NyAzNzUuMDcgbSA1NTcuOTQgMzc1LjA3IGwg
Uwo1MDcuODkgOTMuMzAgbQo1NzQuNjMgOTMuMzAgbAo1NzQuNjMgMjE0Ljg4IGwKNTA3Ljg5
IDIxNC44OCBsCjUwNy44OSA5My4zMCBsClMKMi4yNSB3CltdIDAgZAo1OTEuMzEgMTI2LjAw
IG0gNjU4LjA2IDEyNi4wMCBsIFMKMC43NSB3ClsgMy4wMCA1LjAwXSAwIGQKNjI0LjY5IDkx
LjA4IG0gNjI0LjY5IDk3LjgzIGwgUwo2MjQuNjkgNDAxLjcxIG0gNjI0LjY5IDM0MS45NiBs
IFMKMC43NSB3CltdIDAgZAo2MDguMDAgOTEuMDggbSA2NDEuMzcgOTEuMDggbCBTCjYwOC4w
MCA0MDEuNzEgbSA2NDEuMzcgNDAxLjcxIGwgUwo2MDguMDAgOTEuMDggbSA2NDEuMzcgOTEu
MDggbCBTCjYwOC4wMCA0MDEuNzEgbSA2NDEuMzcgNDAxLjcxIGwgUwo2MDguMDAgOTEuMDgg
bSA2NDEuMzcgOTEuMDggbCBTCjYwOC4wMCA0MDEuNzEgbSA2NDEuMzcgNDAxLjcxIGwgUwo2
MDguMDAgOTEuMDggbSA2NDEuMzcgOTEuMDggbCBTCjYwOC4wMCA0MDEuNzEgbSA2NDEuMzcg
NDAxLjcxIGwgUwo2MDguMDAgOTEuMDggbSA2NDEuMzcgOTEuMDggbCBTCjYwOC4wMCA0MDEu
NzEgbSA2NDEuMzcgNDAxLjcxIGwgUwo2MDguMDAgOTEuMDggbSA2NDEuMzcgOTEuMDggbCBT
CjYwOC4wMCA0MDEuNzEgbSA2NDEuMzcgNDAxLjcxIGwgUwo2MDguMDAgOTEuMDggbSA2NDEu
MzcgOTEuMDggbCBTCjYwOC4wMCA0MDEuNzEgbSA2NDEuMzcgNDAxLjcxIGwgUwo1OTEuMzEg
OTcuODMgbQo2NTguMDYgOTcuODMgbAo2NTguMDYgMzQxLjk2IGwKNTkxLjMxIDM0MS45NiBs
CjU5MS4zMSA5Ny44MyBsClMKUSBxCjAuMDAwIDAuMDAwIDAuMDAwIFJHCjAuNzUgdwpbXSAw
IGQKMSBKCjEgagoxMC4wMCBNCjEyNC4xMSA3My40NCBtIDYyNC42OSA3My40NCBsIFMKMTI0
LjExIDczLjQ0IG0gMTI0LjExIDY2LjI0IGwgUwoyMDcuNTQgNzMuNDQgbSAyMDcuNTQgNjYu
MjQgbCBTCjI5MC45NyA3My40NCBtIDI5MC45NyA2Ni4yNCBsIFMKMzc0LjQwIDczLjQ0IG0g
Mzc0LjQwIDY2LjI0IGwgUwo0NTcuODMgNzMuNDQgbSA0NTcuODMgNjYuMjQgbCBTCjU0MS4y
NiA3My40NCBtIDU0MS4yNiA2Ni4yNCBsIFMKNjI0LjY5IDczLjQ0IG0gNjI0LjY5IDY2LjI0
IGwgUwpCVAowLjAwMCAwLjAwMCAwLjAwMCByZwovRjIgMSBUZiAxMi4wMCAwLjAwIC0wLjAw
IDEyLjAwIDExNC43OCA0Ny41MiBUbSAoZ2NjKSBUagovRjIgMSBUZiAxMi4wMCAwLjAwIC0w
LjAwIDEyLjAwIDE5Ni4xNSA0Ny41MiBUbSAoeWFydikgVGoKL0YyIDEgVGYgMTIuMDAgMC4w
MCAtMC4wMCAxMi4wMCAyNzMuMTQgNDcuNTIgVG0gKHB5dGhvbikgVGoKL0YyIDEgVGYgMTIu
MDAgMC4wMCAtMC4wMCAxMi4wMCAzNjQuMzEgNDcuNTIgVG0gKHBlcmwpIFRqCi9GMiAxIFRm
IDEyLjAwIDAuMDAgLTAuMDAgMTIuMDAgNDQ3LjgyIDQ3LjUyIFRtIChwaHApIFRqCi9GMiAx
IFRmIDEyLjAwIDAuMDAgLTAuMDAgMTIuMDAgNTI5LjYyIDQ3LjUyIFRtIChydWJ5KSBUagov
RjIgMSBUZiAxMi4wMCAwLjAwIC0wLjAwIDEyLjAwIDYxMS43MSA0Ny41MiBUbSAoanJ1Ynkp
IFRqCkVUCjU5LjA0IDg4LjUzIG0gNTkuMDQgNDY1Ljg3IGwgUwo1OS4wNCA4OC41MyBtIDUx
Ljg0IDg4LjUzIGwgUwo1OS4wNCAxNjQuMDAgbSA1MS44NCAxNjQuMDAgbCBTCjU5LjA0IDIz
OS40NyBtIDUxLjg0IDIzOS40NyBsIFMKNTkuMDQgMzE0LjkzIG0gNTEuODQgMzE0LjkzIGwg
Uwo1OS4wNCAzOTAuNDAgbSA1MS44NCAzOTAuNDAgbCBTCjU5LjA0IDQ2NS44NyBtIDUxLjg0
IDQ2NS44NyBsIFMKQlQKL0YyIDEgVGYgMC4wMCAxMi4wMCAtMTIuMDAgMC4wMCA0MS43NiA4
NS4yMCBUbSAoMCkgVGoKL0YyIDEgVGYgMC4wMCAxMi4wMCAtMTIuMDAgMC4wMCA0MS43NiAx
NTMuOTkgVG0gKDEwMCkgVGoKL0YyIDEgVGYgMC4wMCAxMi4wMCAtMTIuMDAgMC4wMCA0MS43
NiAyMjkuNDYgVG0gKDIwMCkgVGoKL0YyIDEgVGYgMC4wMCAxMi4wMCAtMTIuMDAgMC4wMCA0
MS43NiAzMDQuOTMgVG0gKDMwMCkgVGoKL0YyIDEgVGYgMC4wMCAxMi4wMCAtMTIuMDAgMC4w
MCA0MS43NiAzODAuMzkgVG0gKDQwMCkgVGoKL0YyIDEgVGYgMC4wMCAxMi4wMCAtMTIuMDAg
MC4wMCA0MS43NiA0NTUuODYgVG0gKDUwMCkgVGoKRVQKNTkuMDQgNzMuNDQgbQo2ODkuNzYg
NzMuNDQgbAo2ODkuNzYgNDgwLjk2IGwKNTkuMDQgNDgwLjk2IGwKNTkuMDQgNzMuNDQgbApT
ClEKZW5kc3RyZWFtCmVuZG9iago4IDAgb2JqCjYyODgKZW5kb2JqCjMgMCBvYmoKPDwKL1R5
cGUgL1BhZ2VzCi9LaWRzIFsKNiAwIFIKXQovQ291bnQgMQovTWVkaWFCb3ggWzAgMCA3MjAg
NTQwXQo+PgplbmRvYmoKNCAwIG9iago8PAovUHJvY1NldCBbL1BERiAvVGV4dF0KL0ZvbnQg
PDwgL0YxIDUgMCBSIC9GMiAxMCAwIFIgPj4KL0V4dEdTdGF0ZSA8PCA+Pgo+PgplbmRvYmoK
OSAwIG9iago8PAovVHlwZSAvRW5jb2RpbmcKL0Jhc2VFbmNvZGluZyAvV2luQW5zaUVuY29k
aW5nCi9EaWZmZXJlbmNlcyBbIDQ1L21pbnVzIDk2L3F1b3RlbGVmdAoxNDQvZG90bGVzc2kg
L2dyYXZlIC9hY3V0ZSAvY2lyY3VtZmxleCAvdGlsZGUgL21hY3JvbiAvYnJldmUgL2RvdGFj
Y2VudAovZGllcmVzaXMgLy5ub3RkZWYgL3JpbmcgL2NlZGlsbGEgLy5ub3RkZWYgL2h1bmdh
cnVtbGF1dCAvb2dvbmVrIC9jYXJvbiAvc3BhY2VdCj4+CmVuZG9iagoxMCAwIG9iaiA8PAov
VHlwZSAvRm9udAovU3VidHlwZSAvVHlwZTEKL05hbWUgL0YyCi9CYXNlRm9udCAvSGVsdmV0
aWNhCi9FbmNvZGluZyA5IDAgUgo+PiBlbmRvYmoKeHJlZgowIDExCjAwMDAwMDAwMDAgNjU1
MzUgZiAKMDAwMDAwMDAyMSAwMDAwMCBuIAowMDAwMDAwMTYzIDAwMDAwIG4gCjAwMDAwMDY3
MzYgMDAwMDAgbiAKMDAwMDAwNjgxOSAwMDAwMCBuIAowMDAwMDAwMjEyIDAwMDAwIG4gCjAw
MDAwMDAyOTUgMDAwMDAgbiAKMDAwMDAwMDM3NSAwMDAwMCBuIAowMDAwMDA2NzE2IDAwMDAw
IG4gCjAwMDAwMDY5MTIgMDAwMDAgbiAKMDAwMDAwNzE2OSAwMDAwMCBuIAp0cmFpbGVyCjw8
Ci9TaXplIDExCi9JbmZvIDEgMCBSCi9Sb290IDIgMCBSCj4+CnN0YXJ0eHJlZgo3MjY2CiUl
RU9GCg==
--------------090700090302050407010606--

 

[Chad Perrin]

Well ... OK ... you agree with me that they are useful for core
implementation. And I *still* think they're useful when comparing
performance of languages for general purpose programming, provided the
benchmark suite is comprehensive and the statistical comparison
methodology is correct.

I still think you're missing my main point -- which is that, in general,
comparing languages within a few points of standard deviation (that is to
say *not necessarily* within standard deviation, but at least close) is
almost fruitless for most purposes. The reason I think it's pretty much
irrelevant to good decision-making is that other factors will far
outweigh the miniscule performance benefits you might achieve by
examining benchmarks you hope will be representative of your application
design needs -- factors like syntactic construct support for the
architecture you'll implement, general readability, metaprogramming
capabilities, library support, an agreeable inheritance model,
availability in your deployment environment, et cetera.

Would you rather implement a complex object oriented application
architecture in Ruby or Perl, knowing Perl's obtuse OOP syntax and with a
benchmark-suggested 1.1% performance improvement for Perl, all else being
equal? I'd look at the 1.1% performance improvement and say "So f-ing
what? It's OOP. Use Ruby."

On the other hand, if we were comparing Ruby and OCaml, I'd have to
reconsider if performance is a concern, because the performance
improvement would be more like an order of magnitude rather than a
percentage point or two.

Of course. I'm still in the process of crunching the numbers, but what
I've seen so far is that MRI Ruby is *not* in the general neighborhood
of Perl, Python or PHP, but YARV *is* in that neighborhood. I do need to
tighten up the calculations slightly and get rid of some bogus points,
but I'm not expecting a major change in that.

I'll have a look at the document you provided in your next email (which
I've only skimmed very briefly so far) to see what you consider a
"neighborhood", then.

As for things in the "fast lane", I started with gcc, and there are only
a few things faster than gcc. The one that jumped out and bit me on the
nose was a *functional* language called "clean". I'm just about to
download it and see what makes it tick (or to stretch the analogy, how
tightly it's wound). And the slow lane has such snails as "groovy" and
"rebol" lumbering away.

I've heard incredibly good things about Clean. I've also witnessed
something like a meltdown on a mailing list because someone reached a
frustration saturation point dealing with the language. I don't know if
that was more the language's fault, or the fault of the programmer's
background in something like Java. I'm interested in giving it a look at
some point, but I have other stuff in the queue first. It's probably way
back there around the level of Erlang and Forth for me -- both of which
are languages about which I've heard great things, but are well outside
the range of my aims right now.

Well ... let me finish with the numbers, but I think you'll be surprised
at how close Perl and Python are to each other and how far away MRI is
from the two of them. It's not a "petty microsecond dispute" for MRI.

Okay, fair enough.

Of course. But if the choice is between "dynamic languages with a
sufficient supply of competent developers available", for example, Perl,
Python, PHP and Ruby, and the assumption is that they are equally
productive -- you're just trying to assess how much hardware you're
going to have to throw at it if your customer growth takes off -- you're
going to want the fastest one. Or, more to the point, you're probably
going to dismiss the slowest one and move on to the other decision factors.

They're *not* equally productive in many cases, however. For OOP, I'd
choose Ruby or Python over PHP or Perl any day of the week, because OOP
in Perl is painful, and in PHP is downright suicidal. I'd also choose
Ruby over Python just as quickly, because I find Python just generally
painful -- but that's more personal taste than anything else, and
wouldn't necessarily dictate my suggestion for someone else that likes
Python.

 

[M. Edward (Ed) Borasky]

Would you rather implement a complex object oriented application
architecture in Ruby or Perl, knowing Perl's obtuse OOP syntax and with a
benchmark-suggested 1.1% performance improvement for Perl, all else being
equal? I'd look at the 1.1% performance improvement and say "So f-ing
what? It's OOP. Use Ruby."

Well ... I guess I don't think Perl's OOP syntax is all that bad. I can
read and write Perl OO scripts (now) and, as I've noted before, I am one
of several Perl programmers and one of only one Ruby programmers in my
general work neighborhood. I haven't tried Python yet -- I don't even
read Python -- so I wouldn't know how its object features compare with
Ruby's. But I'll also admit that if it were a mandated OO project, it
would be difficult to rule out Java for any "scripting language".

I've heard incredibly good things about Clean. I've also witnessed
something like a meltdown on a mailing list because someone reached a
frustration saturation point dealing with the language. I don't know if
that was more the language's fault, or the fault of the programmer's
background in something like Java. I'm interested in giving it a look at
some point, but I have other stuff in the queue first. It's probably way
back there around the level of Erlang and Forth for me -- both of which
are languages about which I've heard great things, but are well outside
the range of my aims right now.

Well, I already *know* Forth. Unless you're just a hard-core language
freak or want to get involved with embedded systems, you'd probably be
just as well off ignoring Forth.

Besides, I'm only downloading Clean because the Vector Pascal download
site is down!

 

[Chad Perrin]

M. Edward (Ed) Borasky wrote:

OK ... here's the analysis. The attached PDF is what's known as a box
and whisker plot, usually shortened to "boxplot". The raw numbers that
went into this are from the Alioth shootout page, and what I *haven't*
done is checked out which versions of Perl, Python, YARV, Ruby, jRuby
and PHP these tests used. They could be years old or they could have
been run yesterday morning. I discarded all the tests for which there
were any missing values.

Nice. I wasn't expecting anything as effective for a quick, by-the-pants
analysis as a box plot. What did you use to develop the graphic? I'm
curious. . . .

The value plotted is the ratio of seconds for the benchmark on the
dynamic language to the seconds for the benchmark with gcc. Thus, gcc
equals 1.0 across the board and lower is better/faster. How do you
interpret these plots?

Correct me if I'm wrong, but . . . won't doing this as a gcc-comparative
ratio bias slower languages toward an exaggerated upper bound on the box?

I admit I haven't done this sort of thing in a while, so my sorta
heuristic analysis of what I see may be prone to minor glitches.

The whisker on the bottom is approximately the 5th percentile. In other
words, only five percent of the time is your performance going to be
that good or better. And the whisker on the top is approximately the
95th percentile -- only five percent of the time is it going to be that
bad or worse.

Both yarv and ruby appear to be prone to low-end statistical outliers, or
perhaps to a "long tail" on the slow side of things.

The bottom of the box is the 25th percentile. 25 percent of the time,
the performance will be that good or better. The top of the box is the
75th percentile. 25 percent of the time it will be that bad or worse.

So now we can make more precise Chad Perrin's notion of "general
neighborhoods of performance". We do this by looking at the height of
the median and the width of the box. And we see that relative to gcc,
YARV, Python, Perl and PHP have fairly close medians and the boxes are
about the same width. So they are all "in the same neighborhood" and gcc
is faster. Ruby and jRuby are *not* in the same neighborhood.

Actually, the ruby and php medians (using lower-case to denote
implementations) are almost identical, and perl's isn't far off,
according to this. It appears that only in instances approaching
worst-case does ruby really begin to suffer. Also, if I'm not entirely
missing the implications of a gcc-ratio comparison, I'm not entirely sure
that we can trust the huge visual differences in the height of the boxes
to indicate significant performance problem areas.

I'm also not sure the jruby implementation's numbers here are
representative of JRuby's strengths. Does JRuby benefit from the
benefits of Java's optimizing VM for long-running processes? I know it's
probably suffering, in micro-benchmarks, from increased start-up times as
it loads the VM.

I'm surprised to see python's median so low. Are these benchmarks heavy
on bytecode-optimized Python, or do they tend to use standard interpreted
Python?

Now suppose your boss comes to you, as bosses do, and says, "well, all
them high-falutin' box plots are dandy, but the board of directors wants
one number for each language!" It turns out (and I'll let Google and
Wikipedia fill in the blanks for you) that the one number you want to
give your boss, aside from your cell phone number, is the *geometric
mean* of all the benchmark ratios. Again, smaller is better. So here's
how the languages stack up:

gcc 1.0
yarv 13.2
perl 14.0
python 14.2
php 15.5
ruby 29.9
jruby 55.0

That's a little clearer (barring the problem of whether jruby benefits
from the optimizations for long-running processes), but of course suffers
from the lack of attention to statistical outliers and low-end
performance curves. That's one of the reasons I like box plots.

So yes, you can pretty much expect the same performance from YARV, Perl,
Python and PHP. And you can pretty much expect something like a 13 - 16
to one speed improvement if you decide to rewrite your application in C.

It's pretty clear to me from these numbers is that the only reason that
deploying web applications on the LAMP stack and its cousins using
PostgreSQL, Perl, Python and Ruby is economically viable is that they
spend most of their time either in the database or on the network. The
bad news is that for an application with 100 percent dynamic language
code -- no C libraries for the intensive calculations, no highly-tuned
web servers, databases or other components -- you're going to end up
throwing twice as much hardware at scaling problems in Ruby as you will
in PHP, Perl or Python. The good news is that YARV will level the
playing field.

Unless you're implying a different set of working conditions, you're not
going to throw twice as much hardware at the Ruby implementation, because
the bottlenecks for dynamic language web development (assuming good
design) still haven't changed. They're still I/O and network traffic of
various sorts, not the languages.

In my experience, that's pretty much the sort of thing that happens with
*everything* that is lax enough on performance needs to "settle" for
something like Perl, et al. That's the neighborhood I'm talking about
for performance: not so slow it can't be used for a common desktop app,
not so fast that it should be used for hard-core number-crunching or
graphics-intensive game development.

Neighborhoods are relative, after all. There's the neighborhood with
C/C++, OCaml, and so on, way up at the top; there's the neighborhood with
Perl, Python, Ruby, and so on, somewhere in the middle; there's the
neighborhood with the languages so slow nobody really uses them, way down
at the bottom. You're more likely to find you need to make finer
distinctions in execution speed up near the top, where performance
*really* matters.

One final note to implementers and language designers ... Python gets an
extra little pat on the back from me for having such a low spread. YARV
and other Ruby implementations need to pay attention to the fact that
their boxes are wider than Perl's and PHP's and a *lot* wider than
Python's. In other words, look at the benchmarks where you really suck
first.

No kidding.

Those of us picking a language for a project might want to check the
specifics of where a language really sucks, too, to determine whether
that's going to be a problem -- but other than that, as long as your
performance needs are simple enough to allow for a dynamic language like
Perl et alii, you might as well just pick the one you like the best. At
least, that's my take on the matter -- look for problem areas that are
deal-breakers, and otherwise don't worry about it too much unless you're
writing code that has to fit in eight bits and run like the dickens (for
example). Anything else strikes me as a case of "premature
optimization", especially considering the difference a good algorithm can
make.

In case you want the numbers that go with the boxplots, here they are:

gcc yarv python perl php ruby jruby
Low 1 1.2 1.4 0.93 1.4 1.5 3.4
Q1 1 4.8 4.6 2.90 3.1 6.3 12.0
Median 1 8.7 14.0 26.00 31.0 34.0 50.0
Q3 1 68.0 45.0 55.00 55.0 170.0 340.0
High 1 150.0 98.0 67.00 110.0 380.0 410.0

Thanks muchly for the statistics-wrangling. It's instructive.

 

[Chad Perrin]

Well ... I guess I don't think Perl's OOP syntax is all that bad. I can
read and write Perl OO scripts (now) and, as I've noted before, I am one
of several Perl programmers and one of only one Ruby programmers in my
general work neighborhood. I haven't tried Python yet -- I don't even
read Python -- so I wouldn't know how its object features compare with
Ruby's. But I'll also admit that if it were a mandated OO project, it
would be difficult to rule out Java for any "scripting language".

Perl's is bolted on and fugly. I usually disagree when people say Perl's
syntax is inherently ugly -- it can lend itself to very elegant (and
readable!) code in the hands of someone who knows what (s)he's doing. In
the case of OO Perl, however, I fear I must agree that it's just hideous,
and has a little more scaffolding cluttering things up than it should.

Python reads a bit like Ruby without Ruby's elegant idioms (such as
iterator blocks), with additional explicitness in cases like calls to
self, and (for me at least) a disconcerting tendency to make my eyes want
to run off the right-hand edge of the "page" and to find myself wondering
where the rest of a block of code went, even if it's actually complete as
written. Python just generally looks inherently broken, like someone
left a messy, half-finished bunch of hacked-up clutter on the page. It's
amazing how much an end-delimiter (or lack thereof) makes in readability
for me; with Python, I have to kind of reorder my brain just to get past
the fact that the code always looks broken and unfinished before I can
make sense of it. I'm also mildly put off by the mixture of methods and
core functions -- which, in contrast to Perl's bolted-on OOP, makes
Python's non-OOP look bolted on (but less avoidable than Perl's OOP
stuff).

The general OO structure of it is roughly comparable to Ruby's in many
ways, though.

As for Java, I'd find it difficult to convince myself to use it for
anything where the benefits of the optimizing VM's long-running
performance characteristics weren't critical to the app.

Well, I already *know* Forth. Unless you're just a hard-core language
freak or want to get involved with embedded systems, you'd probably be
just as well off ignoring Forth.

I have some interest in embedded systems -- eventually. I also like
learning things in general, and learning things that give me new
perspectives and help me broaden the way I think about problems in
specific. What I've heard about Forth makes me think that'd be a good
one to add to the repertoire . . . some day. Haskell's a bit further up
the list, though.

Besides, I'm only downloading Clean because the Vector Pascal download
site is down!

Whatever excuse helps you get up in the morning, I guess.