Question about bytecode and yield

[Daniel Brodie]

Hello,

1) I am trying to understand how yield works in a bytecode way. How
exactly does the YIELD_VALUE instruction work?
Does the function 'change' to return a generator at compile time
(which, from my understanding, not much happens at that stage in
python) or at the YIELD_VALUE bytecode?

Any pointer to documentation about this would be appriciated, I have
tried, but havn't really found anything.

2) This is the reason I am asking #1. Is it possible to, at runtime,
change the bytecode of a function so that to turn it into a generator
from a normal, average, function? Would just putting in a YIELD_VALUE
(in a sane way, obvioussly) work?

3) yield does some saving of the current frame, and so forth. Does
this happen, burried underneath, or would it be possible to do
something like that (saving the frame, local variables, etc... and
resuming it later) on my own, even as a C module.

Once again, any documentation (in CVS, or something like that) about
this would be appriciated.

Thank You,
Daniel Brodie

 

[Tim Peters]

[Daniel Brodie]

1) I am trying to understand how yield works in a bytecode way. How
exactly does the YIELD_VALUE instruction work?
Does the function 'change' to return a generator at compile time
(which, from my understanding, not much happens at that stage in
python) or at the YIELD_VALUE bytecode?

At compile time. The code object's co_flags member has to have the
CO_GENERATOR flag set in order for generator setup to occur correctly.
That is, the runtime must know that a function is a generator before
it can invoke it correctly.

Any pointer to documentation about this would be appriciated, I have
tried, but havn't really found anything.

PEP 255 documents the semantic of generators, The implementation in
CPython is, of course, wholly defined by the source code. See
ceval.c. The implementation of generators is pretty simple (but
because Python always had heap-allocated "stack frames").

2) This is the reason I am asking #1. Is it possible to, at runtime,
change the bytecode of a function so that to turn it into a generator
from a normal, average, function?
No.

Would just putting in a YIELD_VALUE (in a sane way, obvioussly) work?
No.

3) yield does some saving of the current frame, and so forth.

The primary thing it does is *refrain* from decrementing the refcount
on the frame object. That's what keeps the locals and "instruction
pointer" etc alive for resumption.

Does this happen, burried underneath, or would it be possible to do
something like that (saving the frame, local variables, etc... and
resuming it later) on my own, even as a C module.

Anything crossing the C-Python boundary requires entirely different
machinery. See

http;//www.stackless.com/

for fancier stuff.