Standardizing RPython - it's time.

[John Nagle]

It may be time to standardize "RPython".

There are at least three implementations of "RPython" variants - PyPy,
Shed Skin, and RPython for LLVM. The first two are up and running.
There's a theory paper on the subject:

http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.142.1457&rep=rep1&type=pdf

All three have somewhat different restrictions:

PyPy's Rpython:
http://codespeak.net/pypy/dist/pypy/doc/coding-guide.html

Shed Skin:
file:///C:/Users/nagle/AppData/Local/Temp/shedskin-tutorial-0.3.html

Rpython for LLVM:
http://code.google.com/p/rpython/

So a language standardization effort, independent of CPython,
would be useful.

All attempts to make the dialect defined by CPython significantly
faster have failed. PyPy did not achieve much of a speed
improvement over CPython, and is sometimes slower. Unladen Swallow
has not achieved a significant speedup, and is now a year behind
on quarterly releases, indicating that the project is in serious
trouble. PyPy's RPython, Psyco, and Shed Skin, on the other hand,
all achieved order of magnitude or better speedups over CPython.
So this is the way to go for higher performance.

RPython may be the future of Python, a "golden middle way" between
statically and dynamically typed languages. A basic concept of RPython
is that it is declaration-free, like the CPython dialect, but limits
the ability to change the type of an object at run time.
Both PyPy's RPython and Shed Skin use type inference to determine
the types of objects.

PyPy's RPython has the best defined set of restrictions.
Most of the restrictions are for speed, such as the unique-type
restriction. A few simply reflect unimplemented features,
such as generators. (Generators are not inherently inefficient.)

A significant advantage of having a well defined set of
restrictions is that it encourages writing Python libraries which will
run under both RPython and CPython. This makes migration to a faster
system much easier.

John Nagle

 

[Jed Smith]

file:///C:/Users/nagle/AppData/Local/Temp/shedskin-tutorial-0.3.html

This gives me a 404. Your Web server is broken! Fix it!

Temporarily mirrored: http://jedsmith.org/tmp/shedskin-tutorial-0.5.html

 

[John Nagle]

On 10/11/2010 1:01 PM, John Nagle wrote:
(Correct Shed Skin tutorial link)

 

[Ryan Kelly]

It may be time to standardize "RPython".

There are at least three implementations of "RPython" variants - PyPy,
Shed Skin, and RPython for LLVM. The first two are up and running.
There's a theory paper on the subject:

http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.142.1457&rep=rep1&type=pdf

All three have somewhat different restrictions:

PyPy's Rpython:
http://codespeak.net/pypy/dist/pypy/doc/coding-guide.html

Shed Skin:
file:///C:/Users/nagle/AppData/Local/Temp/shedskin-tutorial-0.3.html

Rpython for LLVM:
http://code.google.com/p/rpython/

So a language standardization effort, independent of CPython,
would be useful.

A similar topic was recently discussed on the pypy-dev mailing list:

http://codespeak.net/pipermail/pypy-dev/2010q3/006170.html


My interpretation is that the pypy devs are -0 on such a standardisation
effort, as it would give them less flexibility in moulding rpython for
their specific needs. Adding features to rpython that make it better as
a general-purpose programming language could actually make it *worse* as
a specialised language for building pypy.

OTOH, there does seem to be a growing interest in using rpython as a
stand-alone language. I've used it for some small projects and it
worked out great.

But the intersection of (people who want rpython as a general-purpose
language) and (people who have the ability to make that happen) seems to
be approximately zero at the moment...


Ryan

--
Ryan Kelly
http://www.rfk.id.au | This message is digitally signed. Please visit
(e-mail address removed) | http://www.rfk.id.au/ramblings/gpg/ for details


-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.10 (GNU/Linux)

iEYEABECAAYFAkyzd8YACgkQfI5S64uP50pozwCgiTc2ZqZGRrrLTAcQf7cBoQvl
mssAn1OcpaKGVIBZVJVGCWgsZsWUpVjk
=xg5a
-----END PGP SIGNATURE-----

 

[John Nagle]

A similar topic was recently discussed on the pypy-dev mailing list:

http://codespeak.net/pipermail/pypy-dev/2010q3/006170.html


My interpretation is that the pypy devs are -0 on such a standardisation
effort, as it would give them less flexibility in moulding rpython for
their specific needs. Adding features to rpython that make it better as
a general-purpose programming language could actually make it *worse* as
a specialised language for building pypy.

OTOH, there does seem to be a growing interest in using rpython as a
stand-alone language. I've used it for some small projects and it
worked out great.

But the intersection of (people who want rpython as a general-purpose
language) and (people who have the ability to make that happen) seems to
be approximately zero at the moment...

With Unladen Swallow looking like a failed IT project, a year
behind schedule and not delivering anything like the promised
performance, Google management may pull the plug on funding.
Since there hasn't been a "quarterly release" in a year, that
may already have effectively happened.

All the schemes to speed up Python as defined by CPython seem to hit
a wall on speed improvement. Some of the numeric benchmarks go faster
on implementations that don't box all numbers, but 2x seems to be about
as good as it gets, even with a JIT compiler.

John Nagle

 

[alex23]

     All the schemes to speed up Python as defined by CPython seem to hit
a wall on speed improvement.  Some of the numeric benchmarks go faster
on implementations that don't box all numbers, but 2x seems to be about
as good as it gets, even with a JIT compiler.

That hasn't been the case with PyPy for quite some time: http://goo.gl/dA7v
(link to specific speed.pypy.org comparison).

Comparing the latest PyPy+JIT to CPython 2.6.2, there are 6 tests that
perform worse and 12 that perform better. Of the 12 improvements, 9
are of an improvement of 3x or better, with one exceeding 12x and one
15x. The django benchmark is 6x faster under PyPy+JIT than CPython.
(The PyPy guys have done a fantastic job with http://speed.pypy.org to
help show the gains that they're getting.)

Don't be so quick to condemn PyPy, especially as they're _actively_
working on this project, not just engaging in endless online rhetoric
about how great life would be if only Python was faster. Achieving
performance gains _without_ sacrificing the power of the language
should be the ultimate goal.

 

[Stefan Behnel]

John Nagle, 11.10.2010 22:01:

It may be time to standardize "RPython".

There are at least three implementations of "RPython" variants - PyPy,
Shed Skin, and RPython for LLVM. The first two are up and running.

The thing is, while RPython can be seen as a general purpose programming
language, it lacks much of what larger applications would need. It doesn't
have access to the CPython ecosystem, for one. Not at all in the case of
Shedskin, and only part of it in the case of PyPy (ironically, excluding
mostly those parts that were written for high performance). Don't know
about the LLVM implementation, which seems to be far from mature anyway.

If you implemented an RPython to CPython extension compiler, I'd expect
that at least some people would use it. Lacking that, however, I don't see
a major interest in writing code in RPython that doesn't have access to all
of the CPython ecosystem. I'm not saying it can't be useful in certain
cases, especially when you can be sure in advance that you won't need that
ecosystem. But I don't really see the general purpose applicability of
RPython to application development.

Stefan

 

[Stefan Behnel]

Stefan Behnel, 12.10.2010 09:18:

If you implemented an RPython to CPython extension compiler, [...]

BTW, if anyone wanted to do that, it might be a good idea to start with
Cython, adapt its type inference layer and add the few missing Python
language features (or pay the core developers to do it for you). That would
likely be the fastest way to get this rolling.

Stefan

 

[Carl Friedrich Bolz]

Hi John,

All attempts to make the dialect defined by CPython significantly
faster have failed. PyPy did not achieve much of a speed
improvement over CPython, and is sometimes slower.

This is not true. While PyPy is indeed sometimes slower than CPython, there is a
significant speedup for many synthetic and real-life benchmarks. PyPy with a JIT
is up to 20 times faster than CPython for synthetic benchmarks like richards,
but still something like 6 times faster for Django. It also beats Psyco in
almost all benchmarks. For detailed measurements, please visit our benchmarking
site: http://speed.pypy.org

All these numbers are also only the current state, PyPy is still continuously
improving.

Cheers,

Carl Friedrich