YIELD_VALUE Byte Code

A

Andreas Löscher

Hi,
I am not sure if this is the right newsgroup, so if not don't hesitate
to tell me.

I am developed a Python to C compiler, so that Byte Code files
automatically can be translated into C Extension Modules. (And it works
pretty well --> http://www.coremountains.com/products/bytecoat/)

While developing, I found something strange concerning the YIELD_VALUE
OpCode.

Since Python Version 2.5 it behaves the following:
1. pop yield value from stack and return it to
a former gen_send_ex() call from Objects/genobject.c

2. push the yield value on the stack in gen_send_ex() and return it

3. when the generator is executed again, the yield value is
'poped' from the stack again with the POP_TOP opcode

Now I found that a little strange:
1. why is the value removed from the stack, than pushed on the
stack to remove it finally again? the combination of
YIELD_VALUE and TOP_POP seems hard coded in compile.c
which means that a TOP_POP follows every YIELD_VALUE
TOP_POP

2. If the semantic of the YIELD_VALUE OpCode has changed, why
is this reached by using another function? Such thing
should be done in the OpCode.

(e.a.: instead of retval = POP()
--> retval = TOP(); Py_INCREF(retval); )


I am a little confused about this.

Best,
Andreas
 
T

Terry Reedy

Andreas said:
Hi,
I am not sure if this is the right newsgroup, so if not don't hesitate
to tell me.

Since there is no CPython internals list, this is the right place to
start, even though you might end up having to post a slightly off-topic
query to python-devel just to get the attention of whoever has the
answer. But there are a few deveoopers who read and post here also.
I am developed a Python to C compiler, so that Byte Code files
automatically can be translated into C Extension Modules. (And it works
pretty well --> http://www.coremountains.com/products/bytecoat/)

While developing, I found something strange concerning the YIELD_VALUE
OpCode.

Since Python Version 2.5 it behaves the following:
1. pop yield value from stack and return it to
a former gen_send_ex() call from Objects/genobject.c

2. push the yield value on the stack in gen_send_ex() and return it

3. when the generator is executed again, the yield value is
'poped' from the stack again with the POP_TOP opcode

Now I found that a little strange:
1. why is the value removed from the stack, than pushed on the
stack to remove it finally again? the combination of
YIELD_VALUE and TOP_POP seems hard coded in compile.c
which means that a TOP_POP follows every YIELD_VALUE
TOP_POP

2. If the semantic of the YIELD_VALUE OpCode has changed, why
is this reached by using another function? Such thing
should be done in the OpCode.

(e.a.: instead of retval = POP()
--> retval = TOP(); Py_INCREF(retval); )

I am a little confused about this.

I suspect that what you see reflects the 2.5 change of 'yield x' from a
statement to an expression to enable gen.send(). It is possible that
someone did the simplest thing that worked, rather than properly
rewriting the code. Or maybe the trickery is needed. You can always try
a simplyfing patch against the test suite and even submit it if it passes.

Terry Jan Reedy
 
K

Kee Nethery

I've been looking for examples of how to run unit tests on a CGI.
Didn't find any so I came up with this pared down sample. Don't know
where to post example code so I figured I'd just email it to this
list. Perhaps it will be findable by others in the future.

This CGI with unit test examples takes three separate files to make it
work. Copy the example code into the files with the names provided and
put them all into the same directory.

In my current project, I have lots of unit tests for all the functions
I have created, but the main CGI code itself had no unit tests and it
really needed them. In this example, the CGI itself is a handful of
lines that should never change and the bulk of the processing is in a
function. This allows me to create unit tests on everything except the
few unchanging lines that are the generic CGI in the cgiRunner.py file.

Kee Nethery

---------- cgiRunner.py ------------
#!/usr/bin/env python

# 2009-11-17

# this is a simple CGI that is structured to allow for unit tests.

# this CGI responds to a GET or a POST
# send it anything and it will parrot it back along with client
# browser info.

# This sample has three files:
# cgiRunner.py this file
# cgiFunctions.py does the bulk of the CGI processing
# cgiUnitTest.py the unittest file for this CGI

# In this example, the bulk of the CGI processing occurs in a
# function. The function is in a separate file so that the
# unit tests can be run against it. All this main does is
# convert the URL parameters into a dictionary and
# feed that dictionary to the CGI function formatClientData()

# This is written for Python 2.6.x I'm using 2.6.4 and am trying to use
# code that works with Python 3.x and up.

import cgi ## so that I can be a web server CGI
import cgiFunctions ## where formatClientData() is located

def main():
# give me a list of all the user inputs posted to the cgi
formPostData = cgi.FieldStorage()
# and turn it into a key value pair dictionary
# this uses a list comprehension which I modelled after others
# examples. Normally I'd do a for loop but figured this
# should be as fast as possible.
formPostDict = dict((keyValue,formPostData[keyValue].value) \
for keyValue in formPostData)

# this is the call to the actual CGI code.
cgiResponseList = cgiFunctions.formatClientData(formPostDict)

# I like to return a list from functions so that I can put
# validity tests in the function. This one basically returns
# True if the GET or POST had any parameters sent to the
# CGI, False if none. Just an example.
if cgiResponseList[1] == True:
cgiResponseData = cgiResponseList[0]
else:
# if no parameters were sent to the CGI, the response
# describes how to add parameters to the URL that
# triggers this CGI.
cgiResponseData = cgiResponseList[0] + '\n\n' + 'The POST \
or GET you submitted had no user supplied parameters so other than \
client data. The only data that can be returned is the data \
provided by your web client. Most CGIs accept parameters and \
return results based upon those parameters. To provide GET data \
try adding a parameter like "sample=some_text". A URL with \
"sample" as a parameter (and sample2 as a second parameter) might \
look like \n"http://machine.domain.com/cgi-bin/SimpleCgiUnittest.\
py?sample=some_text&sample2=more_text".'

# Python 3.x style print statement that works in Python 2.6.x
print('Content-type: text/html')
print('')
print(cgiResponseData)


main()


---------- cgiFunctions.py ------------
#!/usr/bin/env python

# 2009-11-17

import os ## gets the CGI client values like IP and URL

def formatClientData(formPostDict):
"""
This function contains the guts of the CGI response.
It is designed as a function so that I can use
unittest to test it.
"""
# create the output variable and then add stuff to it
cgiResponseData = ''

# I like to return lists from functions so that more than
# some kind of validity checking comes back from the
# function. In this case it's a lame validity check but
# it's an example.
hasFormPostData = (len(formPostDict) > 0)

# for something interesting to return, CGI client data
cgiResponseData = cgiResponseData + 'from client browser:\n'
for cgiKey in list(os.environ.keys()):
# for each client data value, add a line to the output
cgiResponseData = cgiResponseData + \
str(cgiKey) + ' = ' + os.environ[cgiKey] + '\n'

cgiResponseData = cgiResponseData + '\n\nfrom the URL \
POST or GET:\n\n'
# cycle through and output the POST or GET inputs
for keyValuePair in formPostDict:
cgiResponseData = cgiResponseData + \
keyValuePair + ' = ' + formPostDict[keyValuePair] + '\n'

return [cgiResponseData, hasFormPostData]




---------- cgiUnitTest.py ------------
#!/usr/bin/env python

import cgiFunctions
import unittest

class cgiTests(unittest.TestCase):

## For debugging of unit tests, to get one unit test
## to run first, UPPERCASE the first char of the unittext
## name (after "test_") so that it runs first then lower
## case it when it should be part of the group.

def setUp(self):
# this is for setting external stuff before unit tests
# this gets called before EVERY test
pass

def test_formatClientData_With(self):
inputDict = {'sample': 'test_text', 'drink': 'tea'}
cgiResponse = '''from the URL POST or GET:

sample = test_text
drink = tea
'''
expected0 = True
expected1 = True
outputList = cgiFunctions.formatClientData(inputDict)
outputBoolean0 = (cgiResponse in outputList[0])
self.assertEqual(outputBoolean0,expected0)
self.assertEqual(outputList[1],expected1)

def test_formatClientData_Without(self):
inputDict = {}
expected1 = False
outputList = cgiFunctions.formatClientData(inputDict)
self.assertEqual(outputList[1],expected1)

if __name__ == '__main__':
testSuite=unittest.TestLoader().loadTestsFromTestCase(cgiTests)
unittest.TextTestRunner(verbosity=2).run(testSuite)
 
G

greg

Andreas said:
Since Python Version 2.5 it behaves the following:
1. pop yield value from stack and return it to
a former gen_send_ex() call from Objects/genobject.c

2. push the yield value on the stack in gen_send_ex() and return

Are you sure about that? I would expect the value pushed to
be the one sent back into the generator using a send() call
if any, or None if the generator is resumed using next()
rather than send().
3. when the generator is executed again, the yield value is
'poped' from the stack again with the POP_TOP opcode

This will happen if the result of the yield expression is
not used for anything in the Python code.
 

Ask a Question

Want to reply to this thread or ask your own question?

You'll need to choose a username for the site, which only take a couple of moments. After that, you can post your question and our members will help you out.

Ask a Question

Members online

Forum statistics

Threads
473,995
Messages
2,570,231
Members
46,820
Latest member
GilbertoA5

Latest Threads

Top