Greenlets: where are they now???

[David Pokorny]

Hi,

A few weeks ago, I discovered Armin Rigo's greenlets and thought they were
brilliant. I'm seriously thinking about using them, and at least someone
else is

http://www.eby-sarna.com/pipermail/peak/2004-August/001720.html

Currently, I understand that Greenlets are more-or-less hidden from the
public eye in the Stackless CVS, even though they compile and work like a
charm with regular CPython. Given all the desire for full coroutines that I
hear on Python-dev, greenlets strike me as genuine "hidden treasure." Here
is my point:

A) Why aren't greenlets being considered as a solution to coroutines? (Do
people feel that "hacking the C stack is bad" is a compelling argument? It
strikes me that any coroutine implementation in CPython will have to do
this.)

B) Will a later version of CPython support greenlets natively? (in
particular by adding the appropriate entries to PyThreadState)

C) Will greenlets ever be released? Is there a ToDo list that prevents their
release?

D) How did he come up with a cool name like "Greenlet"?

Cheers,
David

 

[Peter Hansen]

A) Why aren't greenlets being considered as a solution to coroutines?

Maybe because coroutines aren't a problem that needs to be solved.
Seriously though, what is the actual problem to which you feel
greenlets are a good solution? Answering that might reveal that
relatively few people actually face the problem described,
thus few people bother with its solution...

D) How did he come up with a cool name like "Greenlet"?

"Green threads" is a term that is fairly widely used for
non-native (non-OS) threads, though I have no idea of the
origin. Presumably these are a lightweight type of non-
native thread, thus greenlet.

-Peter

 

[David Pokorny]

Seriously though, what is the actual problem to which you feel
greenlets are a good solution?

Greenlets are about 90% of a solution (which I've thought hard about) to the
problem of interrupting an execution after 50 byte-code, while retaining the
ability to resume the execution (for, say, another 50 byte codes).

The main requirements of the solution are:

- it work with regular CPython with minimal tinkering (some tinkering of
ceval.c is necessary).

- it not do any "reimplementing eval," i.e. I'm not in the business of
second-guessing PyEval_EvalFrame.

....

In other words, my message may have implied that I'm concerned with other
people's uses for coroutines. I'm actually much more interested in hearing
Armin Rigo comment on the development status of Greenlets.

BTW, the code that ultimately will get interrupted every 50 byte codes is
not my own---it will be Python bots, written by users... ...friendly
users... (<suspicious glance at deprecated rexec>)

David

 

[Peter Hansen]

Greenlets are about 90% of a solution (which I've thought hard about) to the
problem of interrupting an execution after 50 byte-code, while retaining the
ability to resume the execution (for, say, another 50 byte codes).

Python's regular threads already allow this, so there must be
something more to it.

-Peter

 

[Michael Hudson]

Hi,

A few weeks ago, I discovered Armin Rigo's greenlets and thought they were
brilliant. I'm seriously thinking about using them, and at least someone
else is

http://www.eby-sarna.com/pipermail/peak/2004-August/001720.html

Currently, I understand that Greenlets are more-or-less hidden from the
public eye in the Stackless CVS, even though they compile and work like a
charm with regular CPython. Given all the desire for full coroutines that I
hear on Python-dev, greenlets strike me as genuine "hidden treasure." Here
is my point:

A) Why aren't greenlets being considered as a solution to coroutines? (Do
people feel that "hacking the C stack is bad" is a compelling argument? It
strikes me that any coroutine implementation in CPython will have to do
this.)

I think the answer to your parenthetical question is "yes".

B) Will a later version of CPython support greenlets natively? (in
particular by adding the appropriate entries to PyThreadState)

Pass.

"It is tough to make predictions, especially about the future."
-- Yogi Berra

C) Will greenlets ever be released? Is there a ToDo list that prevents their
release?

I think lack of tuits is the main thing. I don't know if there are
obscure bugs or something like that.

D) How did he come up with a cool name like "Greenlet"?

Like Peter said, I presume it's a play on "green threads".

I'm fairly sure that (a) the only person who can really answer your
questions is Armin himself and (b) Armin doesn't read
comp.lang.python. I'll try to catch him on IRC and point him here,
but you might want to email him yourself...

Cheers,
mwh

 

[David Pokorny]

Python's regular threads already allow this, so there must be
something more to it.

I haven't the faintest idea how to gain fine control over threads. In
particular, suppose I have 10 of these "executions" going (lets call them
"steplets") and I want to run each of them for 50 byte-codes, then switch
back to the controlling thread and do lots of bookkeeping (I don't control
the code that the steplets run). I also have a bunch of other processes that
I don't want to take part in this "cycle rationing" but rather run as honest
threads.

David

 

[Michele Simionato]

"It is tough to make predictions, especially about the future."
-- Yogi Berra

I think I saw this attributed to Nils Bohr. Can somebody check?

Michele Simionato

 

[Peter Hansen]

I haven't the faintest idea how to gain fine control over threads.

Not sure what you mean by "fine control". I was reacting to the
"run for 50 bytecodes" part, which is something all threads do,
based on the current value of sys.setcheckinterval() (which defaults
to 100 now, if I recall correctly).

In
particular, suppose I have 10 of these "executions" going (lets call them
"steplets") and I want to run each of them for 50 byte-codes, then switch
back to the controlling thread and do lots of bookkeeping (I don't control
the code that the steplets run).

Not exactly something regular threads let you do under direct control,
though of course the main thread will run in due course anyway.

> I also have a bunch of other processes that
I don't want to take part in this "cycle rationing" but rather run as honest
threads.

It almost sounds like you want to do this to prevent these other
threads from being able to take up too much CPU time. I don't
believe the greenlets approach would necessarily guarantee that,
either, since there are bytecodes that can consume unbounded
amounts of time.

You've described the characteristics that you think your
solution requires, but still not the actual problem. I suspect
the application you have is similar to ones where the usual
response in this newsgroup has been to point out that ultimately
only running this stuff in another process will provide the
guarantees you might need. (Though this "fine-grained control
with bookkeeping" thing is a little different from the usual
requests.)

-Peter

 

[Alan Kennedy]

[Michael Hudson]
[Michele Simionato]

I think I saw this attributed to Nils Bohr. Can somebody check?

Or was it Enrico Fermi? Winston Churchill? Vint Cerf? Groucho Marx?

http://www.larry.denenberg.com/predictions.html

regards,

 

[Michael Hudson]

I think I saw this attributed to Nils Bohr. Can somebody check?

Hmm. I already had but googling just now found this page:

http://www.larry.denenberg.com/predictions.html

which attributes variations on the above sentiment to just about
everyone you've ever heard of.

The phrasing above is certainly attributed to Yogi Berra pretty often.
Bohr is more likely to have the wildly different phrase:

"Prediction is very difficult, especially about the future."

attributed to him

It's a fairly obvious thought, after all.

Cheers,
mwh

 

[John Lenton]

I haven't the faintest idea how to gain fine control over threads. In
particular, suppose I have 10 of these "executions" going (lets call them
"steplets") and I want to run each of them for 50 byte-codes, then switch
back to the controlling thread and do lots of bookkeeping (I don't control
the code that the steplets run). I also have a bunch of other processes that
I don't want to take part in this "cycle rationing" but rather run as honest
threads.

wouldn't calling setcheckinterval with some ridiculously large
argument on entering the loop and setting it back to 100 (or 50) on
exit be enough?

--
John Lenton ([email protected]) -- Random fortune:
Q: How many Marxists does it take to screw in a light bulb?
A: None: The light bulb contains the seeds of its own revolution.

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.2.4 (GNU/Linux)

iD8DBQFBRuw3gPqu395ykGsRAp9XAJ9MxuidjMLf4DYh8T6N8dknfsNBgQCgwRPf
/to9f2mephasn7V56+U2r7M=
=nIMh
-----END PGP SIGNATURE-----

 

[David Pokorny]

I don't
believe the greenlets approach would necessarily guarantee that,
either, since there are bytecodes that can consume unbounded
amounts of time.

You've described the characteristics that you think your
solution requires, but still not the actual problem.

Here's the idea: the executions I want to step through are bots that are
competing, 1 on 1, in a mathematical game (or otherwise, perhaps simulated
tag). Here is a good (and old) example

http://soda.csua.berkeley.edu/~davebrok/knobeln.txt

To answer your first point, although I do not write the bot code, anyone can
look at it, and in particular anyone can verify that no "cheating" occurs
along the lines of performing serious number crunching in a single bytecode
(via a C extension---AFAIK this is the main loophole in the
unbounded-execution-time-for-a-single-bytecode department).

David