how can we make a ruby compiler

[Tony Arcieri]

[Note: parts of this message were removed to make it a legal post.]

I tend to write Ruby code that avoids the advanced constructs of Ruby,
making translation to C++ easier.

This is pretty much a staple of static type inferencers for dynamic
languages that output C++... many language features simply cannot be
supported using this approach, most notably eval, and eval is pretty much
the most powerful tool for Ruby metaprogramming. Source translation to C++
would preclude many of the things which make Ruby fun and interesting.

 

[Rick DeNatale]

This is pretty much a staple of static type inferencers for dynamic
languages that output C++... many language features simply cannot be
supported using this approach, most notably eval, and eval is pretty much
the most powerful tool for Ruby metaprogramming. Source translation to C++
would preclude many of the things which make Ruby fun and interesting.

This is very true. Dipping into what seems to have been attempted in
this area reveals lots of attempts which solve a small but easy subset
of the problem, like translating a "hello world" program, or a small
numerical problem with some polymorphic numeric values from Ruby or
Python to C or C++, and then stall out either when the hard problems
arise, or when the requirements to qualify for a Master's degree
project are met.

And Mohit's original observation is also important:

If you get to that scale (and are hopefully making money by then or have very large investments that
match your scale of users/ traffic), there is a high chance that you will need to re-architect
or rewrite a bunch of stuff - I don't think source translation will solve the problem.

Languages differ in much more important and subtle ways than their
syntax and surface-level semantics. Those differences lead to very
different architectures/strategies or whatever you want to call it.

Converting an OO program written in, say Ruby, requires adaptation of
not only the language syntax, but the underlying class library, if you
are converting to C++, you need to figure out what to do about the
usage in the Ruby program of base classes, you do probably have
several choices of C++ libraries providing things like collection
classes, all of which will mismatch the interfaces of their Ruby
equivalents, probably in frightening ways.

Replace Smalltalk/Ruby message send semantics with C++ virtual
function calls, and the constraints change. Move the burden of
accounting for and reclaiming the memory used by objects from a GC to
the programmer and things get complicated, and often the performance
of the hand-written memory management (copy constructors, reference
counts ...) starts making the performance 'win' of switching horses
much less certain.

This is Greenspun's tenth rule of programming at work.
http://philip.greenspun.com/research/

--
Rick DeNatale

Blog: http://talklikeaduck.denhaven2.com/
Github: http://github.com/rubyredrick
Twitter: @RickDeNatale
WWR: http://www.workingwithrails.com/person/9021-rick-denatale
LinkedIn: http://www.linkedin.com/in/rickdenatale

 

[William Rutiser]

Lisp, Scheme, Forth and Smalltalk are all compilable so in principle Ruby should be as well. It's just not clear that there's any real gain by doing so outside of very particular applications (i.e. dominated by maths or interpretation performance bottlenecks). Moreover multicore is teasing us towards a world where all processing will be a negligible cost compared to I/O latencies and throughput, much as was the case fifty years ago.

I have just reread all the messages in this thread and decided not to
finish a draft reply that was already too long. In summary, I argued
that possible language implementations lie on a continuum with "pure"
compilers and interpreters at the extremes, and most implementations for
real languages somewhere in the middle. Therefore, it is not very
productive to discuss whether a Ruby compiler is possible, but instead
focus on solutions to the specific problems that cause people to ask
about compilers for Ruby.

In this thread, only Philip Rhoades has mentioned a real application
that he would like to be able to reprogram in Ruby and still have run
times comparable to C. Perhaps he can expand, preferably in a new
thread, on his program and the benefits he hopes to achieve by moving
the work to Ruby.

-- Bill

 

[Rick DeNatale]

In this thread, only Philip Rhoades has mentioned a real application that he
would like to be able to reprogram in Ruby and still have run times
comparable to C. Perhaps he can expand, preferably in a new thread, on his
program and the benefits he hopes to achieve by moving the work to Ruby.

I have an actual existing application (C/C++) for population genetics simulations
which I would dearly love to convert to Ruby.
It was originally all in C and, as a learning exercise, I converted parts of it to C++ - but it was such a pain .
. it would have been so pleasant to re-write in Ruby.
However I would have to get resulting code converted back to C or compiled somehow to
get the performance back to something usable.

It's not clear from this whether he has actually tried to recode the
program in Ruby or this is still a gedanken experiment.

The first step would be to try to re-code in Ruby and see where the
performance bottlenecks resulting from the conversion, if any, were.
Then optimizing these steps either by rewriting the Ruby or converting
selective code to C extensions might get him to the performance he
seeks, or at least acceptable performance.
--
Rick DeNatale

Blog: http://talklikeaduck.denhaven2.com/
Github: http://github.com/rubyredrick
Twitter: @RickDeNatale
WWR: http://www.workingwithrails.com/person/9021-rick-denatale
LinkedIn: http://www.linkedin.com/in/rickdenatale

 

[Eleanor McHugh]

Languages differ in much more important and subtle ways than their
syntax and surface-level semantics. Those differences lead to very
different architectures/strategies or whatever you want to call it.
=20
Converting an OO program written in, say Ruby, requires adaptation of
not only the language syntax, but the underlying class library, if you
are converting to C++, you need to figure out what to do about the
usage in the Ruby program of base classes, you do probably have
several choices of C++ libraries providing things like collection
classes, all of which will mismatch the interfaces of their Ruby
equivalents, probably in frightening ways.
=20
Replace Smalltalk/Ruby message send semantics with C++ virtual
function calls, and the constraints change. Move the burden of
accounting for and reclaiming the memory used by objects from a GC to
the programmer and things get complicated, and often the performance
of the hand-written memory management (copy constructors, reference
counts ...) starts making the performance 'win' of switching horses
much less certain.
=20
This is Greenspun's tenth rule of programming at work.
http://philip.greenspun.com/research/

Funnily enough this is one of the inspirations behind my golightly[0] =
project: if every significant program I write will end up with an ad hoc =
implementation of Lisp (or more often than not Lisp and several other =
languages) then why not just use an abstract platform that allows me to =
plug together Lisp machines, Forth machines, Ruby machines etc. and have =
them all operate on the same underlying type space?

This is subtly different to the traditional virtual machine approach as =
by taking seriously the notion of language agnosticism you end up with =
less a single machine and more a set of components that can be plugged =
together to build an infinite variety of machines that are each better =
suited to the particular tasks they're used for than a general-purpose =
VM. I suspect it's probably similar to the software IC concept you =
mentioned over on Core recently, but with the added wrinkle that I'm =
interested in building in generalised type space transformations if I =
can get my head around them. These would effectively provide a means of =
JIT translation between different language type models, including =
assembler-level type expression where practical. I suspect a big chunk =
of the winter is likely to be spent researching current developments in =
compiler theory in hopes someone else has already solved this problem... =
and if they haven't perhaps it's time I went back to Uni and got my =
Doctorate


Ellie

[0] http://golightly.games-with-brains.net/

Eleanor McHugh
Games With Brains
http://feyeleanor.tel

 

[Rick DeNatale]

On 6 Oct 2010, at 13:11, Rick DeNatale wrote:

This is Greenspun's tenth rule of programming at work.
http://philip.greenspun.com/research/

Funnily enough this is one of the inspirations behind my golightly[0] pro=

ject: if every significant program I write will end up with an ad hoc imple=
mentation of Lisp (or more often than not Lisp and several other languages)=
then why not just use an abstract platform that allows me to plug together=
Lisp machines, Forth machines, Ruby machines etc. and have them all operat=
e on the same underlying type space?

This is subtly different to the traditional virtual machine approach as b=

y taking seriously the notion of language agnosticism you end up with less =
a single machine and more a set of components that can be plugged together =
to build an infinite variety of machines that are each better suited to the=
particular tasks they're used for than a general-purpose VM. I suspect it'=
s probably similar to the software IC concept you mentioned over on Core re=
cently, but with the added wrinkle that I'm interested in building in gener=
alised type space transformations if I can get my head around them. These w=
ould effectively provide a means of JIT translation between different langu=
age type models, including assembler-level type expression where practical.=
I suspect a big chunk of the winter is likely to be spent researching curr=
ent developments in compiler theory in hopes someone else has already solve=
d this problem... and if they haven't perhaps it's time I went back to Uni =
and got my Doctorate

Sounds a bit like several historic efforts like Uncol, IBM's SOM, and
Microsoft's (D)CLR.

Also sounds for a great topic for a Ruby Conf version of the kind of
semi-inebriated late night discussions I used to enjoy at OOPSLA with
the likes of Kent Beck, Ward Cunningham, George Bosworth of Digitalk
(who went on to work on DCLR at MS before giving up on them) and
others back in the heyday.


--=20
Rick DeNatale

Help fund my talk at Ruby Conf 2010:http://pledgie.com/campaigns/13677
Blog: http://talklikeaduck.denhaven2.com/
Github: http://github.com/rubyredrick
Twitter: @RickDeNatale
WWR: http://www.workingwithrails.com/person/9021-rick-denatale
LinkedIn: http://www.linkedin.com/in/rickdenatale

 

[Frank Lesser [LSW]]

Ruby can be dynamically translated ( = jitted ) as Smalltalk is.
A project "Rubinius" exists which goes in that direction.
The problem with ruby is that it is not so easy ( = lot of work ) to write a
ruby to bytecode translator.
Smalltalk & List style languages are almost syntax free - the contrary is
the case for Ruby.
Or Smalltalk VM ( LSW-GVM ) runs Smalltalk at the speed of C/C++ and we plan
to do that also for other dynamic languages ( e.g Haskell, JavaScript also
Ruby ) in the nearest future. The LSW-GVM is currently available for Win32
( deep GUI integration, multithreading ). We are working on a 64 Bit
version.

Frank Lesser

[Note: parts of this message were removed to make it a legal post.]

I tend to write Ruby code that avoids the advanced constructs of Ruby,
making translation to C++ easier.

This is pretty much a staple of static type inferencers for dynamic
languages that output C++... many language features simply cannot be
supported using this approach, most notably eval, and eval is pretty much
the most powerful tool for Ruby metaprogramming. Source translation to C++
would preclude many of the things which make Ruby fun and interesting.

 

[Eleanor McHugh]

Sounds a bit like several historic efforts like Uncol, IBM's SOM, and
Microsoft's (D)CLR.

I don't think anyone's done this particularly well to date, in large =
part because it's a ferociously difficult problem domain to get a decent =
handle on. But I guess that's part of the attraction lol

Also sounds for a great topic for a Ruby Conf version of the kind of
semi-inebriated late night discussions I used to enjoy at OOPSLA with
the likes of Kent Beck, Ward Cunningham, George Bosworth of Digitalk
(who went on to work on DCLR at MS before giving up on them) and
others back in the heyday.

Well I've never been backward in geeking when there's beer on tap, =
though finding anyone who can understand what the hell I'm on about is =
always a bit of a sticking point. They just don't make young geeks like =
they used to...


Ellie

Eleanor McHugh
Games With Brains
http://feyeleanor.tel

 

[Eduardo Blumenfeld]

I may be wrong, but I've read that IronRuby can be set to allow
compilation after x loops through a process, in order to speed up
execution.

Shall that be a (possible) solution to the puzzle?
Specially if that "compiled" code can be saved somewhere?

I'm not that technically savvy, however you can check with the great
guys in the IronRuby forum about it.