Ruby Weekly News 21st - 27th March 2005

[Tim Sutherland]

http://www.rubyweeklynews.org/20050327.html

Ruby Weekly News 21st - 27th March 2005
---------------------------------------

Ruby Weekly News is a summary of the week's activity on the ruby-talk
mailing list / the comp.lang.ruby newsgroup, brought to you by Tim
Sutherland.

Articles and Announcements
--------------------------

* Ruby article on DevSource

Hal Fulton wrote an article for DevSource.com entitled
"Five Things You Didn't Know You Could Do with Ruby".

From the article,

"When DevSource asked me to write this article, I had to fight two
temptations. One was to make it very language-oriented, talking
about Ruby's syntax and semantics. Another was to make it very
technology-oriented, full of buzzwords such as RSS and XMLRPC.

Though I've used the occasional buzzword here, I took neither of
those paths. I've tried instead to present Ruby from a high-level
view, almost a management perspective, to show that Ruby is mature,
powerful, versatile, and gaining industry acceptance on a daily
basis."

* Getting the word to conventional programmers

A related DevSource article, "The State of the Scripting Universe",
quotes representatives from the Perl, PHP, Python, Ruby and Tcl
communities.

Dave Thomas provides the Ruby viewpoint.

* RubyForge now supports SFTP...

Tom Copeland announced that, thanks to Brian Candler, RubyForge now
supports the SFTP protocol. (Like SSH+FTP.)

* RubyConf 2005 Preregistration now open!

David A. Black announced that pre-registration for RubyConf 2005 is
open. The conference is in San Diego, California, USA on October
14-16, 2005.

Several people outside the U.S. said that they would not be attending
because of the current travel policies of the U.S. government. Francis
Hwang (a U.S. citizen) sympathised and suggested that next year's
conference be held in Canada.

More positively, David's post quoted Rails author David Heinemeier
Hansson: "You have to come. I went last year and had the most
wonderful three days in the company of most of the best Ruby hackers.
I'm so most definitely going again this year. You have to come."

Quote of the Week
-----------------

Continuing the recent theme of having the Quote come from anywhere
other than the mailing list, this week's is from the #ruby-lang
IRC channel. (2005/03/25 logs.)

Matz:

"I have talked with Koichi Sasada, and he agreed to merge his YARV in the
current interpreter.
I _hope_ VARV will be core engine before the end of this year."

The goal of YARV is "to develop the fastest Virtual Machine for Ruby in
the world". Koichi Sasada's work is supported by the IPA
(Information-technology Promotion Agency, Japan) "Exploratory Software
Project 2004 (youth)".

Threads
-------

Interesting threads this week included:

Ruby, brother of VB?
--------------------

Mike Cox was looking for a language to switch to from Visual Basic. After
finding Ruby, he felt that "Ruby may possibly be the brother of VB, and
that Ruby is the son of Algol 60. Just a cursery glance over Ruby
confirmed that my beloved "begin" and "end" are there."

Phil Tomson remarked "some might call them fight'n words", and James Britt
warned Mike to "Forget all the VB you've ever learned. Really. I was a VB
hacker for some time. I liked it, it paid the rent, got me published. But
habits acquired using VB will get in the way of using Ruby."

Free Rails hosting?
-------------------

Aquila was looking for free Rails web hosting. There are many free PHP
hosts, why not Rails?

Dick Davies suggested hosting on your own server as an alternative to paid
hosting, and Wes Moxam pointed out an article of his describing how to set
up a Rails site on freeshell.org (free shell account).

Lothar Scholz said that the CPU and memory usage of Rails is comparable to
Java web applications, so it isn't well suited to free hosting.

Eric Hodel disagreed with that assessment. He manages 43 Things, which
peaks at 200,000 page views per day. It runs off two servers, but can run
on a single 3GHz Xeon with 2GB ram with no performance degradation. He has
also heard of people running Rails in a 64MB virtual server, and thought
that this could be a useful configuration for free hosts. (Doug Beaver
also gave some data showing that Java solutions can work under similar
memory constraints.)

Lothar thought that even 64MB was far too much for a free host. "You can
without problems put 1000 PHP Users on one machine. Many free hosters have
even more then this."

strip and its evil brother strip!
---------------------------------

Aquila began a thread about the return value of "bang" methods (those
ending in "!"). This naming convention is used to indicate that the method
modifies its receiver.

There is another convention used in the standard library: bang methods
typically return the receiver if a change was made and nil otherwise.

The discussion covered the merits of chaining bang methods (e.g.
a.strip!.upcase!).

Nikolai Weibull said that bang methods were simply an optimisation -
modifying objects in-place can be faster than creating new ones. He
suggested avoiding using bang methods at all unless there is a compelling
performance reason to do so. Nikolai also gave a benchmark where he ran
line.strip.downcase one million times, and compared it to line.strip!;
line.downcase!. The former version took 1.7 seconds, while the latter was
1.2 - not a large performance difference.

Bill Kelly pointed out a post from Matz in 2001 which said that there were
two reasons for the "return nil if no change" behaviour of bang methods.
The first is that it's a cheap way of detecting modification. The second
is because he wants to discourage chaining of bang methods.

unifying hash and proc syntax
-----------------------------

Joel VanderWerf had the idea of removing the distinction between Hashes
and procs.

For example, the following imaginary syntax would create an object that
behaves like a Hash with a default_proc:

{
1 => 2,
2 => 3,
3 => 4,
|x|
x + 1
}

Csaba Henk thought that the idea was clever, but a bit dubious in treating
these as the same. "A hash is basicly transparent, a proc is basicly
opaque."

Support for 10x Productivity Increase with Rails!
-------------------------------------------------

Curt Hibbs related a success story from some developers who got a great
productivity increase through using Ruby on Rails instead of the Java
framework they were previously using. (They used Rails to re-implement a
Java application.)

What makes this story special is that the developers were Bruce Tate and
Justin Gehtland, and the Java framework they had been using was
Spring/Hibernate.

Bruce is the author of the books "Better, Faster, Lighter Java", "Bitter
Java" and "Bitter EJBs", and Justin "is a recognized *expert* in Spring
and Hibernate" (and was a novice in both Ruby and Rails).

"Spring/Hibernate are not your typical Java framework (i.e., not Struts),
but are the current best-of-breed."

A pleasant surprise was that the Rails solution ran faster than the Java
one.

As a "PS", Curt added "Bruce Tate and David Geary (author of Core Java
Server Faces, Core JSTL, and more) have signed with O'Reilly to write a
"Rails Developer Notebook" (obviously one of O'Reilly's new Developer
Notebooks series)."

There were some responses thinking about the productivity differences
between Rails and Java frameworks, and in particular commenting on whether
Rails was being hurt by "hype".

CD Baby:

"The constant hype is what kept me AWAY from Rails for so long! I just
couldn't stomach the constant self-congratulations to it all.

Now I can't stomach the constant other-technology-bashing. (Java, PHP,
etc)

Nobody wants their current beliefs insulted."

FCGI 0.8.5-patch for major memory leaks
---------------------------------------

Kirk Haines posted a patch to fix a major memory leak in Ruby FCGI.
(FastCGI, used in web application servers.) The bug only affects the C
extension version, not the pure Ruby one.

Using DL on Windows
-------------------

Stephan Kämper was finding that DL could not locate the DLL he was using
when he referred to it with DL.dlopen("libs/one.dll").

Dave Burt explained that DLLs need to be accessible in one of the
following ways:

* in the working directory
* in the PATH
* registered (regsvr32.exe one.dll)
* referenced directly

IORCC Crossword Puzzle Tips and Hints!
--------------------------------------

Todd Nathan announced a series of "Tips and Hints" for the IORCC Crossword
Puzzle.

prime genrator in 34 bytes
--------------------------

Simon Strandgaard reported that "JIX" came up with a 34 character Ruby
program that lists all prime integers: i=l=1;(l%i+=1)>i-2&&p(i)while l*=i

Dwemthy's Array-the Ruby mini_adventure
---------------------------------------

why the lucky stiff introduced the sixth chapter of Why's (Poignant) Guide
to Ruby.

"This chapter covers metaclasses and method_missing with the help of
Dwemthy's demonic list."

"Since you were a very young rabbit in little cotton pants, Dwemthy's
Array has breathed on your neck and you're tired of it. You draw your
glistening Hero's Sword from its sheath and cut a low-circling falcon in
half as a confidence building exercise. Your eyes flick about in a cool
and freaky way.

You're ready... It's time to stare this construct in the face."

Ruby newbie: 3 week learning project
------------------------------------

Al Abut decided to blog every day of a 3 week course in learning Ruby
(using the book Sams Teach Yourself Ruby in 21 Days).

He invited others to learn with him at the same time.

English Numerals
----------------

Timothy Byrd came up with this week's Ruby Quiz.

"While we normally write numbers using Arabic (or since Quiz #22, Roman)
numerals, numbers can also be written out as English phrases."

For example,

"7 seven (the hard way)
42 forty-two (a very important number)
2001 two thousand and one (a space odyssey)
1999 (party like it's) nineteen hundred and ninety-nine"

The problem this week is taken from a Pi Mu Epsilon newsletter. (A U.S.
mathematics club.)

"When the integers 1 to 10_000_000_000 are written in the English
language, then sorted as strings, which odd number appears first in the
list?"

The task is to write Ruby code to turn a number into its English
representation, and to then write a program that solves the above problem.

A Poll
------

jeem started a poll, asking the following two questions:

"1. Do you use Ruby in your "day job"?
a.) Yes! Lots!
b.) Some.
c.) I little bit when I can get away with it.
d.) I wish!

2. What are your one or two principal programming languages other than
Ruby?"

There were many responses, and Phil Tomson began a thread comment on
today's poll and more questions, noting that there were a large number of
"a.) Yes! Lots!" answers. "This seems to be a marked change in the Ruby
community since a year ago when the `a' answer would have been much
rarer."

koders has indexed rubyforge.org
--------------------------------

Martin Ankerl noticed that koders.com (a site which allows you to search
through web-available source code) has indexed rubyforge.org.

Hiroshi Nakamura said that the RAA has opened its own experimntal source
code search interface.

OO database concepts...
-----------------------

This thread discusses "Object-Oriented Databases", for example ORDBMS vs
OODBs.

Evaluator for a mini-Ruby in Haskell
------------------------------------

Daniel Berger spotted a simple evaluator for a subset of Ruby, written in
Haskell.

It also shows how to define a Ruby method that behaves like while. (Using
retry.)

New Releases
------------

* FireRuby 0.2.0

Peter Wood released the second version of FireRuby, a
library that provides support for accessing the Firebird RDBMS.
Queries now return Row objects instead of arrays of data. Row provides
information like column name and row number.

* Ruby wrapper for Flickr, flickr-ruby 0.2

Premshree Pillai wrote a Ruby interface to the Flickr photo-sharing
site. It allows you to get and create photosets, search and tag.

Version 0.2 was later released, adding support for uploading pictures.

* Multiblocks 0.1.0-emulating Smalltalk type multiblock calls

Csaba Henk announced the first release of a framework which allows you
to define a "callgroup": a group of methods that behave like a single
one.

* midilib 0.8.4

Jim Menard updated midilib, a pure Ruby library for reading and
writing MIDI files, and dealing with MIDI event data. There were some
API improvements, plus new methods to turn note names like "32nd",
"dotted quarter" into notes.

* Revolution - a Ruby binding to Evolution

Tom Copeland released the first version of Revolution, a Ruby binding
to the Evolution email and personal information management (PIM)
application. Example uses are for extracting contact, calendar and
task information.

* Ruby Facets 0.6.2

TRANS announced a new version of Ruby Facets ("Fantastic Atomic Core
Extensions").

Additions include Time methods (3.hours and so on), module/redirect
(like alias, but wraps methods instead of copying them) and
enumerable/uniq_by.

* Syntax 0.7.0

Jamis Buck improved Syntax, a library for performing lexical analysis.
(In particular, for syntax highlighting text.) Accuracy and robustness
have been improved.

* FastCST 0.5.2

Zed A. Shaw let a "major release" of FastCST escape the laboratory.
FastCST is the "Fast Change Set Tool", which "implements a working
repository that actually functions like a version control tool
should".

* Bangkok: Musical Chess

Jim Menard released Bangkok, much to the relief of its ten million
residents. Ahem. "Bangkok reads chess game descriptions and re-play
the games."

It comes with a listener that converts the game into MIDI music.

* sys-admin 1.0.0

Daniel Berger was "happy" to announce the first release of sys-admin,
intended to be a cross-platform replacement for Etc. The win32-etc
package has now been deprecated, and will be removed from the Win32
Utils project in the near future.

* Hash::MixIn and Python style Object#dict

Florian Gross wrote a quick implementation of Hash::MixIn ("you
provide .keys, .fetch, .store and .delete and get a full-blown Hash
interface"), plus Python-style access to the methods of an object.

* eXPlain Project Management Tool - 1.0

John Wilger was "pleased" to announce the initial public release of
the eXPlain Project Management Tool. It's a web-based project
management system that supports the Agile software development
methodologies.

* First release of Rant

Stefan Lang wrote a build tool, similar to Rake. "The equivalent to a
Makefile for make is the Rantfile. An Rantfile is actually a valid
Ruby script that is read by the rant command."

Stefan explained the differences between Rant and Rake: Rant has
built-in support for C#, and has cleaner internals.

* Tar2RubyScript 0.4.5

Erik Veenstra made some changes to Tar2RubyScript, a tool for
transforming a directory tree that contains a Ruby application into a
single Ruby script.

* AllInOneRuby 0.2.1

Erik also released an update to the AllInOneRuby tool. This creates a
compressed executable for Windows, Linux or MacOS X that contains the
Ruby interpreter and runtime libraries, providing a "just-in-time and
temporary installation of Ruby".

* RubyScript2Exe 0.3.3

Another release from Erik: RubyScript2Exe. This one is used to create
a single executable containing your application, the Ruby interpreter
and runtime libraries, easing distribution. It works on Windows, Linux
and MacOS X.

* Ruby/GtkMozEmbed 0.3

Mirko Maischberger introduced Ruby/GtkMozEmbed, a Ruby binding to
GtkMozEmbed, a Gtk+ widget that embeds the Mozilla browser.

* ruby-oci8 0.1.10

Takehiro Kubo released the latest Oracle8 client library. There were
several changes, including improved cursor support.

* Rake 0.5.0

Jim Weirich "finally" came out with a new version of the Rake build
tool. Bugs have been fixed, recursive rules are now supported, and a
man documentation file was added.

The following people also contributed to this release: Tilman
Sauerbeck, Brian Chandler, Eric Hodel and Jani Monoses.

* ruby-dazuko 0.1.0

Aredridel announced ruby-dazuko, a Ruby interface to dazuko. Dazuko is
a kernel module for Linux and FreeBSD that allows userspace programs
to monitor file accesses, and optionally deny access.

* RubyCocoa 0.4.1

kimura wataru delivered a new RubyCocoa release. RubyCocoa is a Ruby
binding for the Cocoa API in MacOS X.

MacOS X 10.3 and Xcode are now supported. There is also a new option
to allow the RubyCocoa framework to be bundled with youir application,
allowing it to be used on a system that otherwise lacks RubyCocoa.

 

[Tim Sutherland]

Quote of the Week
-----------------

Continuing the recent theme of having the Quote come from anywhere
other than the mailing list, this week's is from the #ruby-lang
IRC channel. (2005/03/25 logs.)

Matz:

"I have talked with Koichi Sasada, and he agreed to merge his YARV in the
current interpreter.
I _hope_ VARV will be core engine before the end of this year."

The goal of YARV is "to develop the fastest Virtual Machine for Ruby in
the world". Koichi Sasada's work is supported by the IPA
(Information-technology Promotion Agency, Japan) "Exploratory Software
Project 2004 (youth)".

[...]

I'm surprised no-one has commented on this yet... I was expecting a chorus
of "wow!"s.

It is VERY BIG NEWS.

By the end of 2005, Matz thinks/hopes the OFFICIAL Ruby implementation will
use YARV instead of the current interpreter.

What would the result of this be?
- Ruby code runs faster. More often we can write the 'cleanest' code
instead of having to make compromises in order to get better performance.

- People who work on the Ruby implementation have a fairly large chunk of
code they'll need to understand.

.... there's been much discussion over the past couple of years to do with
evaluating Ruby with a VM instead of a straight interpreter (e.g. with
Parrot, Rite etc.) This is an actual "it's probably going to happen, real
soon now" from Matz.

I don't know how YARV relates to Rite ...

If Rite consists of
a) Language changes
b) Implementation changes

then are the changes in the Ruby 1.9 interpreter ideas for Rite, and is YARV
the basis of the Rite implementation?

Or will Rite be another implementation altogether?

 

[Nikolai Weibull]

* Tim Sutherland (Mar 30, 2005 13:40):

I'm surprised no-one has commented on this yet... I was expecting a
chorus of "wow!"s.

wow!

Seriously, though, this is great news. I just figured that everyone
else would fill in the wow!'s for me.

By the end of 2005, Matz thinks/hopes the OFFICIAL Ruby implementation
will use YARV instead of the current interpreter.

Yeah, that would be great. It would hopefully mean that a lot of stuff
that I have currently implemented in C for speed could be moved to a
pure Ruby implementation, which is great, as I am really tired of
writing C code.

I don't know how YARV relates to Rite ...

No, that is perhaps the most interesting question. Will YARV simply be
an intermediate VM for the 1.9 series and 2.0 be a complete rewrite of
everything, or will the codebase of 1.9, including YARV, be modified and
trimmed to become Rite?

I'd argue that 1.9+YARV would have to be transformed quite a bit to meet
the goals set for Rite, but if they work well together, perhaps the
transformation method is better, easier, and faster than a complete
rewrite,
nikolai

 

[Yukihiro Matsumoto]

Hi,

In message "Re: Ruby Weekly News 21st - 27th March 2005"

|> I don't know how YARV relates to Rite ...
|
|No, that is perhaps the most interesting question. Will YARV simply be
|an intermediate VM for the 1.9 series and 2.0 be a complete rewrite of
|everything, or will the codebase of 1.9, including YARV, be modified and
|trimmed to become Rite?

Since YARV engine would achieve what I wanted by complete rewrite,
I'd call YARV merged 1.9 as Rite.

matz.

 

[Bill Guindon]

Hi,

In message "Re: Ruby Weekly News 21st - 27th March 2005"

|> I don't know how YARV relates to Rite ...
|
|No, that is perhaps the most interesting question. Will YARV simply be
|an intermediate VM for the 1.9 series and 2.0 be a complete rewrite of
|everything, or will the codebase of 1.9, including YARV, be modified and
|trimmed to become Rite?

Since YARV engine would achieve what I wanted by complete rewrite,
I'd call YARV merged 1.9 as Rite.

matz.

That confirms what I just read here:
"YARV will become Rite ...if this project succeed"
http://www.atdot.net/yarv/RubyConf2004_YARV_pub.pdf

btw, I did say "Wow!" when I read it. Didn't you hear me?

 

[Glenn Parker]

I'm surprised no-one has commented on this yet... I was expecting a chorus
of "wow!"s.

It is VERY BIG NEWS.

By the end of 2005, Matz thinks/hopes the OFFICIAL Ruby implementation will
use YARV instead of the current interpreter.

Call me when it's ready for testing, then maybe I'll say "wow". At this
point, the news is simply that Matz has endorsed YARV. So, hearty
congrats to the YARV developers!

I asked once before if we would have to wait until Xmas 2005 for a real
YARV-based implementation. Looks like the answer is a tentative "yes",
but there are still seven months to go.

 

[why the lucky stiff]

Since YARV engine would achieve what I wanted by complete rewrite,
I'd call YARV merged 1.9 as Rite.

He has come. Son-shi.

_why (Bow, you idiots!)

 

[Bill Guindon]

He has come. Son-shi.

_why (Bow, you idiots!)

Bow, wow!

good puppy

 

[Robert McGovern]

He has come. Son-shi.

_why (Bow, you idiots!)

Couldn't I just pat them on the back electronically? Its easier on the
knee's you know!

Rob

 

[Navindra Umanee]

Nice! What is involved with this merging?

Also... what happened to Matz's own attempt? Didn't take off?

Cheers,
Navin.

 

[Jay Levitt]

Bow, wow!

good puppy

Rorge!!

(I love it when threads converge.)

 

[Brian Candler]

By the end of 2005, Matz thinks/hopes the OFFICIAL Ruby implementation will
use YARV instead of the current interpreter.

What would the result of this be?
- Ruby code runs faster. More often we can write the 'cleanest' code
instead of having to make compromises in order to get better performance.

- People who work on the Ruby implementation have a fairly large chunk of
code they'll need to understand.

But perhaps also:

- The Ruby source code will be bigger and slower to build
- Scripts will be slower to start up
- Ruby applications may be less reliable initially and/or harder to debug
- Ruby may run on fewer platforms [especially if the VM writes machine-code
directly, or has dependencies on external C compilers, linkers etc]

One of the big advantages of Ruby for me is that it's one-third of the size
of Perl, whilst still being much more feature-complete (e.g. I get OpenSSL,
MD5 and base64 encoding without having to install additional libraries)

Not that I'm saying it's a bad idea... I'm just saying it's not necessarily
going to be better for everyone

Regards,

Brian.

 

[gabriele renzi]

Brian Candler ha scritto:

please excuse me if my questions are dumb..

But perhaps also:

- The Ruby source code will be bigger and slower to build

Sorry, I'm not sure I grok this: it could be probably slower to build,
but why should it become bigger?

- Scripts will be slower to start up

I think the process of compiling ruby in bytecode is quite fast. At
least, I could not notice the slowdown when running scripts with yarv.

- Ruby applications may be less reliable initially and/or harder to debug

why? Is Squeak harder to debug since it's written in
Smalltalk-subset-compilable-in-c ?

- Ruby may run on fewer platforms [especially if the VM writes machine-code
directly, or has dependencies on external C compilers, linkers etc]

well, the 'main' yarv can use direct threading when compiled with gcc
wich is available almost everywhere, but it seem to run faster than
currwent ruby even if compiled withouth it.
OTOH, I think AOT compiling can tuned as to work with any C compiler as
a backend, I don't think there is a real problem with it, though IANAVG[1]
And, on even another hand, many compilers scream when compiling the
current cvs with warnings enabled, but people feel happy anyway

One of the big advantages of Ruby for me is that it's one-third of the size
of Perl, whilst still being much more feature-complete (e.g. I get OpenSSL,
MD5 and base64 encoding without having to install additional libraries)

Yup, but why would this become worst with a new vm, since it already
supports the standard ruby dynamic libraries?




[1]I Am Not A Vm Guru

 

[Brian Candler]

Sorry, I'm not sure I grok this: it could be probably slower to build,
but why should it become bigger?

Well, you have to understand I know nothing about YARV other than what is in
English on the home site - i.e. the RubyConf 2004 presentation at
http://www.atdot.net/yarv/RubyConf2004_YARV_pub.pdf
and the "architecture" which is little more than a list of VM opcodes. And
I've not attempted to install the current code, which I believe is a "work
in progress" anyway, so conclusions taken from that would be invalid.

So I can only really talk about what *might* be the case if you rip out the
existing VM - remember that Ruby *does* have a VM, it's one which uses the
syntax tree as its "machine code" - and replace it with something else.

I can't see that simply replacing one straightforward VM interpreter with
another is likely to make much of a speed improvement in itself; so I'm
assuming the new VM will be performing optimisations (peephole? loop
unrolling? polymorphic inline caches / dynamic rewriting of the byte code
sequences?)

Or, the speed improvements will be obtained by the VM compiling sequences of
code into native machine code (which the diagram in the Rubyconf
presentation implies will happen). That might be quite a hard thing to
achieve, especially allowing for the various different environments it might
run in.

I would assume that this whole environment will be quite a lot more complex
and bigger than the existing evaluator; especially if the old evaluator is
kept as well (e.g. if there are some things that the new VM can't do)

I think the process of compiling ruby in bytecode is quite fast. At
least, I could not notice the slowdown when running scripts with yarv.

I agree that building bytecode from text is not going to be significantly
slower than building a syntax tree. But if you start making external calls
to a C compiler and linker then I would expect startup to be pretty slow,
and the more libraries written in Ruby you pull in, the slower it would be.

why? Is Squeak harder to debug since it's written in
Smalltalk-subset-compilable-in-c ?

I'm thinking of compiling-to-native-machine-code issues. If a piece of
machine code generates a SEGV violation, all you get is a core dump. Of
course, once the whole system is bug-free, this should not happen

- Ruby may run on fewer platforms [especially if the VM writes machine-code
directly, or has dependencies on external C compilers, linkers etc]

well, the 'main' yarv can use direct threading when compiled with gcc
wich is available almost everywhere, but it seem to run faster than
currwent ruby even if compiled withouth it.

Ah, so YARV *links* to the C compiler rather than fork/exec? That could be
quite fast (although the C compiler will still be maintaining its own symbol
tables etc). But then it would be very gcc-specific.

OTOH, I think AOT compiling can tuned as to work with any C compiler as
a backend, I don't think there is a real problem with it, though IANAVG[1]
And, on even another hand, many compilers scream when compiling the
current cvs with warnings enabled, but people feel happy anyway

Yeah, but there's a difference between the ./configure && make && make install
cycle of a standalone C program, and writing a program which dynamically
writes C programs and runs them in real time; especially when it's not
building a single executable but a pile of small object modules which must
be dynamically loaded in, destroyed and recompiled on demand, and all work
together seamlessly.

Besides, if it takes 1 second to compile a C program into an executable,
normally you don't care. But I wouldn't want a 1-second startup overhead on
a Ruby script.

Yup, but why would this become worst with a new vm, since it already
supports the standard ruby dynamic libraries?

Well, if the new VM supports the old C API, so that things like
ext/openssl.so continue to work, that's excellent. I guess it implies that
the main 'VALUE' data structures are unchanged.

I'm just thinking that the whole package *could* end up a lot bigger,
because it's more complex.

But please feel free to prove me wrong. How big is the ruby-1.9 tarball? How
big is ruby-1.9 with YARV?

Cheers,

Brian.

 

[gabriele renzi]

Brian Candler ha scritto:

So I can only really talk about what *might* be the case if you rip out the
existing VM - remember that Ruby *does* have a VM, it's one which uses the
syntax tree as its "machine code" - and replace it with something else.

I can't see that simply replacing one straightforward VM interpreter with
another is likely to make much of a speed improvement in itself; so I'm
assuming the new VM will be performing optimisations (peephole? loop
unrolling? polymorphic inline caches / dynamic rewriting of the byte code
sequences?)

well, the first yarv version I tried did not do any of the above, IIRC,
but it was quite faster trhan ruby anyway. Now it uses inline caches and
does operand unification to avoid dispatching and is even faster.

Anyway, I now understand what you meant, I misunderstood the messsage
previous to your (I thought it was talking about rewriting some of ruby
in ruby itself), so sorry for the noise

Or, the speed improvements will be obtained by the VM compiling sequences of
code into native machine code (which the diagram in the Rubyconf
presentation implies will happen). That might be quite a hard thing to
achieve, especially allowing for the various different environments it might
run in.

as I said above, yarv withouth many optimizations was already much
faster than ruby. The AOT compiler adds even more speed, but it compiles
to C code, so it's not much of a worrying since it is the C compiler
wich takes care of the hard stuff

I would assume that this whole environment will be quite a lot more complex
and bigger than the existing evaluator; especially if the old evaluator is
kept as well (e.g. if there are some things that the new VM can't do)

yes, I see what you meant now, please consider my excuse for
misunderstanding given above

I agree that building bytecode from text is not going to be significantly
slower than building a syntax tree. But if you start making external calls
to a C compiler and linker then I would expect startup to be pretty slow,
and the more libraries written in Ruby you pull in, the slower it would be.

I don't think this is the case. AFAIK, YARV could rely on an external
compiler/linker for aot compiler, but not for normal run.

why? Is Squeak harder to debug since it's written in
Smalltalk-subset-compilable-in-c ?

I'm thinking of compiling-to-native-machine-code issues. If a piece of
machine code generates a SEGV violation, all you get is a core dump. Of
course, once the whole system is bug-free, this should not happen

- Ruby may run on fewer platforms [especially if the VM writes machine-code
directly, or has dependencies on external C compilers, linkers etc]

well, the 'main' yarv can use direct threading when compiled with gcc
wich is available almost everywhere, but it seem to run faster than
currwent ruby even if compiled withouth it.

Ah, so YARV *links* to the C compiler rather than fork/exec? That could be
quite fast (although the C compiler will still be maintaining its own symbol
tables etc). But then it would be very gcc-specific.

no, It is different. What I meant is:
when yarv is compiled with gcc, it uses gcc-specific C extensions wich
allow the vm to be quite faster than using standard C, since it uses
direct jumps
to the next instruction instead of a big switch.

But the role of the C compile ends there.
On the other hand, yarv can be used as a ruby->c compiler, then you can
compile the resulting C files with the standard toolchain, and load them
at runtime as any other C extension. No tight interaction beetween C
toolchain and ruby/yarv at runtime.

OTOH, I think AOT compiling can tuned as to work with any C compiler as
a backend, I don't think there is a real problem with it, though IANAVG[1]
And, on even another hand, many compilers scream when compiling the
current cvs with warnings enabled, but people feel happy anyway

Yeah, but there's a difference between the ./configure && make && make install
cycle of a standalone C program, and writing a program which dynamically
writes C programs and runs them in real time;

here is the point: it is not done at runtime

Well, if the new VM supports the old C API, so that things like
ext/openssl.so continue to work, that's excellent. I guess it implies that
the main 'VALUE' data structures are unchanged.

the main interpreter is exactly the same. Yarv is basically just a patch
to eval.c, that's why it is cool

I'm just thinking that the whole package *could* end up a lot bigger,
because it's more complex.

But please feel free to prove me wrong. How big is the ruby-1.9 tarball? How
big is ruby-1.9 with YARV?

well, "wc -l" says nearly 10.000 lines of C code for yarv, and 13.000
for eval.c.
Yarv has more stuff like script to autogenerate things and is still
incomplete, but I think it won't grow too much. Anyway, Let's hope this

 

[Brian Candler]

On the other hand, yarv can be used as a ruby->c compiler, then you can
compile the resulting C files with the standard toolchain, and load them
at runtime as any other C extension. No tight interaction beetween C
toolchain and ruby/yarv at runtime. ...

here is the point: it is not done at runtime

Ah, so YARV is a *static* compiler, not JIT? Then I misunderstood
completely. I saw JIT->"native code" in the presentation, and I thought it
meant machine-code; I see now it's JIT compilation to YARV byte-code.

Given that Ruby is so dynamic, clearly it's not always going to be able to
compile into C. There's "eval", and there's also the case of external Ruby
libraries:

require 'bar.rb'
Bar.new.run

For Bar to be compiled code, bar.rb would have to be compiled to bar.so
beforehand. Can YARV do that? If so that would be great - it should *reduce*
startup overhead when loading libraries.

[Thinks aloud]. Even without eval, the methods present in an object can
change dynamically often. So given a statement like

a.foo

how does this end up in C? If you're saying that there's no dynamic
recompilation at run-time, then essentially you have to compile

obj = getlocal("a");
fn = find_method1(obj, "foo");
(*fn)(obj);

But then we add some caching. Consider "a = a + 1"; each time round the loop
the object 'a' changes, and we don't want to invalidate the cache each time,
so I realise that actually we should cache on the *class*, not the object.
This should be OK, since any object with its own methods will also have its
own singleton class.

/* a = a + 1 */
/* a = a.+(1) */

obj = getlocal("a");
oc = rb_class_of(obj);
if (oc != oc1235 || mstate != mstate1235) {
fn1235 = find_method2(oc, "+");
oc1235 = oc;
mstate1235 = mstate;
}
obj2 = (*fn1235)(obj, INT2FIX(1));
putlocal("a", obj2);

(where 'mstate' is incremented whenever any new methods or classes are
defined, to invalidate all the caches; I saw something about that in the
Rubyconf presentation)

Then if getlocal / rb_class_of / putlocal are inlined, actually the code
doesn't end up being too bad. In the cache-hit case there's a branch forward
(pipeline bubble) and one indirect function call, to the existing fix_plus()
method.

OK, I think I can see where this is going, and it should eliminate a lot of
interpreter overhead - even if it's never going to be as fast as "a++" in C


Am I guessing along the right lines here?

Cheers,

Brian.

 

[Saynatkari]

On the other hand, yarv can be used as a ruby->c compiler, then you can
compile the resulting C files with the standard toolchain, and load them
at runtime as any other C extension. No tight interaction beetween C
toolchain and ruby/yarv at runtime. ...

here is the point: it is not done at runtime

Ah, so YARV is a *static* compiler, not JIT? Then I misunderstood
completely. I saw JIT->"native code" in the presentation, and I thought it
meant machine-code; I see now it's JIT compilation to YARV byte-code.

Given that Ruby is so dynamic, clearly it's not always going to be able to
compile into C. There's "eval", and there's also the case of external Ruby
libraries:

require 'bar.rb'
Bar.new.run

For Bar to be compiled code, bar.rb would have to be compiled to bar.so
beforehand. Can YARV do that? If so that would be great - it should *reduce*
startup overhead when loading libraries.

[Thinks aloud]. Even without eval, the methods present in an object can
change dynamically often. So given a statement like

a.foo

how does this end up in C? If you're saying that there's no dynamic
recompilation at run-time, then essentially you have to compile

obj = getlocal("a");
fn = find_method1(obj, "foo");
(*fn)(obj);

But then we add some caching. Consider "a = a + 1"; each time round the loop
the object 'a' changes, and we don't want to invalidate the cache each time,
so I realise that actually we should cache on the *class*, not the object.
This should be OK, since any object with its own methods will also have its
own singleton class.

/* a = a + 1 */
/* a = a.+(1) */

obj = getlocal("a");
oc = rb_class_of(obj);
if (oc != oc1235 || mstate != mstate1235) {
fn1235 = find_method2(oc, "+");
oc1235 = oc;
mstate1235 = mstate;
}
obj2 = (*fn1235)(obj, INT2FIX(1));
putlocal("a", obj2);

(where 'mstate' is incremented whenever any new methods or classes are
defined, to invalidate all the caches; I saw something about that in the
Rubyconf presentation)

Then if getlocal / rb_class_of / putlocal are inlined, actually the code
doesn't end up being too bad. In the cache-hit case there's a branch forward
(pipeline bubble) and one indirect function call, to the existing fix_plus()
method.

OK, I think I can see where this is going, and it should eliminate a lot of
interpreter overhead - even if it's never going to be as fast as "a++" in C


Am I guessing along the right lines here?

I think you are reading too much into the preceding statements.
YARV is, specifically, intended to provide a virtual machine for
Ruby and there is not much point in having a 'static' executable
in the sense described above (apart from such situations where
the dynamism is not required).

However, compiling to 'static' native code does not mean that the
Ruby code may not be dynamic! Depending on just what gets compiled,
and it would appear that rather than your Ruby code, the target of
the compilation are the operations that make up your Ruby code, you
may certainly retain dynamism in the _application code_. Just like
you may currently write a C extension and have it behave dynamically.

Perhaps Signor Renzi or Koichi-san himself will further elaborate on this.

Cheers,

Brian.

E