Which is one of the many reasons that I have a problem with the
alioth shootout. Ultimately, it tries to treat compiled, semi-
compiled, and interpreted languages equally. They're not. Compiled
languages will be faster than semi-compiled (bytecode), which will
be faster than interpreted. The level of speed difference is a
matter of scale, and that scale also matters on development time and
clarity.
I have to say that I tend to agree with the astonishment.
Here's what I mean:
Someone comes along and asks about python vs. Ruby performance and
references the alioth benchmarks. Several folks jump on the alioth
benchmarks and call them bogus. I see this as blaming the messanger.
The fact is that these benchmarks exist and they cover lots of different
types of algorithms. They're on the web and people will look at them -
no way to stop that from happening.
Someone mentioned the Ackermann benchmark so I had a look and found that
gawk did much better than Ruby - that's pathetic folks. Now, I know that
if you could do this in gawk it should be possible to do it in Ruby as well.
There aren't even mumbers for N=8 or N=9 for the Ruby version. Maybe it's
because the Ruby version was written poorly. Maybe it's because of some
issue within the Ruby interpreter itself - I don't know, I didn't really
take time to look into it just now. Either way, wouldn't it be valuable
to try to improve our score on that benchmark? If it's because it's
poorly written, then it's gravy. If it's an issue with the interpreter
then maybe it would be good to know about the issue for the YARV design.
Now sure you can say that the benchmarks are bogus (and who knows, maybe
they are) and then take all your marbles and go home, but the benchmarks
remain and other languages communities are trying to hone their numbers.
If we totally sit it out, well, that's not going to look too good.
Sure these benchmarks don't tell the whole story about how nice it is to
develop in Ruby vs. gawk, but performance _is_ an issue for some people
and they will use this set of benchmarks (or another) as a factor in
deciding which langauge to use. If Ruby consistently shows up down
towards the bottom of the list... well, it can give a false impression.
If we tell them that they really shouldn't worry about performance (and
who knows, maybe we would be right in doing so) it just sounds like so
much excuse-making.
I think that as a community we should be trying to compete in these
benchmarks just like other langauge communities are doing. Perhaps the
results can help us as we do move to a VM - if nothing else to show
that we're making improvements (look at them as a set of unit tests for
performance). We know that Ruby needs help in the
performance area. If we keep telling ourselves that Ruby is 'fast
enough' for our application (and it may well be) are we going to be
sitting still while other languages improve performance? Have we been
the hare for the last few years sitting around while the tortise (TCL,
for example, once considered a lot slower than Ruby) is passing us up
(by creating or improving their VMs, for example).
Of course we know that we can do all sorts of things to speed things up
by writing C extension code, for example - that's a given, and the fact
that it's so easy to do in Ruby makes it possible to get into a lot of
areas where you might normally need to consider a compiled langauge,
however, a lot of people out there want to see how Ruby performs when
you just write Ruby code (C is a compiled language, of course). In some
sense marrying Ruby and C might seem like cheating to them, I suppose (try
to see it from their perspective).
If it looks like some of the benchmarks are misleading or stacked against
us, maybe we can propose other benchmarks where Ruby shines (BTW: why
does it say that the Object Methods benchmark is being deprecated?),
after all, it appears to be an open source process.
Bottom line: Ruby needs help in the performance area. Let's admit that
and work on improving in that area instead of shooting the messenger -
it'll make us look like whiners if we keep doing the latter. I'd really
rather not have the Ruby community perceived that way from the outside.
In the meantime, perhaps we could have some Ruby quizzes that focus on
choosing a benchmark and optimizing it. I think we would be
better off as a community if we participate in the process rather than
boycott it.
....just my 2cents.
Phil