Python bytecode STORE_NAME

[schwarz]

As part of some research I am doing a Python Virtual Machine in Java,
and the exact semantics of the STORE_NAME bytecode is unclear to be,
so I was hoping somebody here could clarify it.
The STORE_NAME bytecode is supposed to set a value for a name in the
current scope. However, the following piece of code:

def hello(who):
print "Hello", who
return hello(who)
print "Say:"
hello("World")

Results in this bytecode for the top level:
1, LOAD_CONST, 1
4, MAKE_FUNCTION, 0
7, STORE_NAME, 0
10, LOAD_CONST, 2
13, PRINT_ITEM, None
14, PRINT_NEWLINE, None
15, LOAD_NAME, 0
18, LOAD_CONST, 3
21, CALL_FUNCTION, 1
24, POP_TOP, None
25, LOAD_CONST, 0
28, RETURN_VALUE, None

And this bytecode for the hello function:
1, LOAD_CONST, 1
4, PRINT_ITEM, None
5, LOAD_FAST, 0
8, PRINT_ITEM, None
9, PRINT_NEWLINE, None
10, LOAD_GLOBAL, 1
13, LOAD_FAST, 0
16, CALL_FUNCTION, 1
19, RETURN_VALUE, None
20, LOAD_CONST, 0
23, RETURN_VALUE, None

The first column are the byte numbers, and the last column contains
the arguments to the byte codes if they take any.

The function is stored using STORE_NAME with offset 0 in the module
scope, but it is loaded from inside the hello method using LOAD_GLOBAL
with offset 1. My questions are: Does STORE_NAME add things to the
global scope when used top level? And why is the offset different?

The documentation contains nothing usable:
http://www.python.org/doc/2.5.2/lib/bytecodes.html

regards,
Mathias

 

[Peter Otten]

As part of some research I am doing a Python Virtual Machine in Java,
and the exact semantics of the STORE_NAME bytecode is unclear to be,
so I was hoping somebody here could clarify it.
The STORE_NAME bytecode is supposed to set a value for a name in the
current scope. However, the following piece of code:

def hello(who):
print "Hello", who
return hello(who)
print "Say:"
hello("World")

Results in this bytecode for the top level:
1, LOAD_CONST, 1
4, MAKE_FUNCTION, 0
7, STORE_NAME, 0
10, LOAD_CONST, 2
13, PRINT_ITEM, None
14, PRINT_NEWLINE, None
15, LOAD_NAME, 0
18, LOAD_CONST, 3
21, CALL_FUNCTION, 1
24, POP_TOP, None
25, LOAD_CONST, 0
28, RETURN_VALUE, None

And this bytecode for the hello function:
1, LOAD_CONST, 1
4, PRINT_ITEM, None
5, LOAD_FAST, 0
8, PRINT_ITEM, None
9, PRINT_NEWLINE, None
10, LOAD_GLOBAL, 1
13, LOAD_FAST, 0
16, CALL_FUNCTION, 1
19, RETURN_VALUE, None
20, LOAD_CONST, 0
23, RETURN_VALUE, None

The first column are the byte numbers, and the last column contains
the arguments to the byte codes if they take any.

The function is stored using STORE_NAME with offset 0 in the module
scope, but it is loaded from inside the hello method using LOAD_GLOBAL
with offset 1. My questions are: Does STORE_NAME add things to the
global scope when used top level? And why is the offset different?

Every code object has its own co_names attribute (a tuple). The arguments
are offsets into that tuple.

Using Python 2.5 I can't reproduce your example, I get 0 offsets in both
cases. Here's a simpler one:
.... x
.... y
........ y
.... 2 0 LOAD_GLOBAL 0 (x)
3 POP_TOP

3 4 LOAD_GLOBAL 1 (y)
7 POP_TOP
8 LOAD_CONST 0 (None)
11 RETURN_VALUE 2 0 LOAD_GLOBAL 0 (y)
3 POP_TOP
4 LOAD_CONST 0 (None)
7 RETURN_VALUE('y',)

Peter

 

[schwarz]

Every code object has its own co_names attribute (a tuple). The arguments
are offsets into that tuple.

Using Python 2.5 I can't reproduce your example, I get 0 offsets in both
cases. Here's a simpler one:


...     x
...     y
...>>> def g():

...     y
...>>> dis.dis(f)

  2           0 LOAD_GLOBAL              0 (x)
              3 POP_TOP

  3           4 LOAD_GLOBAL              1 (y)
              7 POP_TOP
              8 LOAD_CONST               0 (None)
             11 RETURN_VALUE>>> dis.dis(g)

  2           0 LOAD_GLOBAL              0 (y)
              3 POP_TOP
              4 LOAD_CONST               0 (None)
              7 RETURN_VALUE>>> f.func_code.co_names
('x', 'y')

('y',)

Peter

Ok, thanks a lot. That helped me understand the offsets. Your
disassembly misses the code in the global scope that creates the two
methods from the code objects. That code looks like this for your
example (dis won't give you this. I use a modified version of the
byteplay disassembler which can disassemble a module without loading
it):
1, LOAD_CONST, 1 //Loads the code object for function f
4, MAKE_FUNCTION, 0 //Creates a function from the code object
7, STORE_NAME, 0 //Stores it as 'f' using STORE_NAME
10, LOAD_CONST, 2 //Loads the code object for function g
13, MAKE_FUNCTION, 0 //Creates the function
16, STORE_NAME, 1 //Stores it as 'g' using STORE_NAME
19, LOAD_CONST, 0 //Loads None
22, RETURN_VALUE, None //Exists the program with status None

The two last bytecodes are there because I didn't use the interactive
mode. I guess I narrowed down my other question to whether the
semantics are that stuff saved using STORE_NAME in the global scope
can be loaded everywhere else using LOAD_GLOBAL. What baffled me was
that there is actually a STORE_GLOBAL byte code.

 

[Peter Otten]

Ok, thanks a lot. That helped me understand the offsets. Your
disassembly misses the code in the global scope that creates the two
methods from the code objects. That code looks like this for your
example (dis won't give you this. I use a modified version of the
byteplay disassembler which can disassemble a module without loading
it):
1, LOAD_CONST, 1 //Loads the code object for function f
4, MAKE_FUNCTION, 0 //Creates a function from the code object
7, STORE_NAME, 0 //Stores it as 'f' using STORE_NAME
10, LOAD_CONST, 2 //Loads the code object for function g
13, MAKE_FUNCTION, 0 //Creates the function
16, STORE_NAME, 1 //Stores it as 'g' using STORE_NAME
19, LOAD_CONST, 0 //Loads None
22, RETURN_VALUE, None //Exists the program with status None

You can get it with standard tools, too:

$ cat tmp.py
def f():
x
y

def g():
x
$ python -c'import tmp'
$ python
Python 2.5.1 (r251:54863, Jul 31 2008, 23:17:43)
[GCC 4.1.3 20070929 (prerelease) (Ubuntu 4.1.2-16ubuntu2)] on linux2
Type "help", "copyright", "credits" or "license" for more information.

import marshal
module = marshal.loads(open("tmp.pyc").read()[8:])
import dis
dis.dis(module)

1 0 LOAD_CONST 0 (<code object f at
0x2b3389f3d648, file "tmp.py", line 1>)
3 MAKE_FUNCTION 0
6 STORE_NAME 0 (f)

5 9 LOAD_CONST 1 (<code object g at
0x2b3389f3d828, file "tmp.py", line 5>)
12 MAKE_FUNCTION 0
15 STORE_NAME 1 (g)
18 LOAD_CONST 2 (None)
21 RETURN_VALUE

I just didn't deem it relevant.

The two last bytecodes are there because I didn't use the interactive
mode. I guess I narrowed down my other question to whether the
semantics are that stuff saved using STORE_NAME in the global scope
can be loaded everywhere else using LOAD_GLOBAL. What baffled me was
that there is actually a STORE_GLOBAL byte code.

If you are really interested in the subtleties you have to dive into the
source. After a quick look into

http://svn.python.org/view/python/trunk/Python/ceval.c?rev=63675&view=markup

I think STORE_NAME works with the module startup code because global and
local namespace are identical. Within a function (with distinct global and
local namespaces) you'd have to use STORE_GLOBAL.

Peter