[ANN] pyasm 0.2 - dynamic x86 assembler for python

[Grant Olson]

PyASM by Grant Olson <olsongt at verizon.net>
=============================================

PyASM is a dynamic x86 assembler for python. By "dynamic", I mean that it
can be used to generate inline assembly functions in python at runtime
without requiring object file generation or linkage.

New in version 0.2
------------------

+ Linux Support. Will work in Linux environments as well as Windows.
+ Simplified Interface. You only need to use one function to generate
code.
+ Preliminary Documentation.

More information and downloads are available on my homepage:

http://mysite.verizon.net/olsongt


#################################
## helloWorld.py
## assembly hello world script
#################################

 

[olsongt]

#################################
## helloWorld.py
## assembly hello world script
#################################

DOH! The example file got truncated. Here it is.

#################################
## helloWorld.py
## assembly hello world script
#################################

from pyasm import pyasm

pyasm(globals(),r"""
!CHARS hello_str 'Hello world!\n\0'

!PROC hello_world PYTHON
!ARG self
!ARG args

PUSH hello_str
CALL PySys_WriteStdout
ADD ESP, 0x4
MOV EAX,PyNone
ADD [EAX],1
!ENDPROC
""")

hello_world()

 

[Stefan Behnel]

Hi!

What about an interface like this:

------------------------------
@pyasm
def hello_world(*some_args):
"""
!CHARS hello_str 'Hello world!\n\0'

!PROC hello_world PYTHON
!ARG self
!ARG args

PUSH hello_str
CALL PySys_WriteStdout
ADD ESP, 0x4
MOV EAX,PyNone
ADD [EAX],1
!ENDPROC
"""

hello_world(1,2,3)
------------------------------

Meaning: Put the assembler into the doc-string of a function. Then use a
decorator to run the assembler on the function's __doc__ string and build an
assembly function that takes the same arguments to make the assembly function
directly callable.

Maybe the decorator line has to look like this:
@pyasm(globals())
or something like that, I can't tell. I don't think it would be much work to
implement this.

Stefan

 

[JanC]

Stefan Behnel schreef:

Meaning: Put the assembler into the doc-string of a function. Then
use a decorator to run the assembler on the function's __doc__ string
and build an assembly function that takes the same arguments to make
the assembly function directly callable.

That would 'disable' a programmer's ability to add useful documentation
in the comments, but I played a bit further with the decorator idea.

The code below makes it possible to write assembler code for different
architectures (in case pyasm ever supports that and also a Python
code version to use when running on a system where no assembler code
can be used. It prints:

Hello x86 world!
Hello python world!
Hello i386 world!

Note: there is no serious error checking in this code and it's maybe
not the best possible solution (this is my first decorator code), but
it seems to work...

------------------------------
# supported_architectures should be defined in pyasm
# this is just to show how it could work...
supported_architectures = ['x86','i386']

# asm optimization decorator
def asm_optimise(*architectures):
arch_iter = architectures.__iter__()
try:
arch = arch_iter.next()
while arch not in supported_architectures:
arch = arch_iter.next()

def decorate(f):
# x86 only at this moment ;-)
def optimized(self):
from pyasm.x86asm import codePackageFromFile
from pyasm.x86cpToMemory import CpToMemory
from pyasm.pythonConstants import PythonConstants
import cStringIO

arch_asm = eval('self._asm_%s' % arch)
cp = codePackageFromFile(cStringIO.StringIO(arch_asm),PythonConstants)
mem = CpToMemory(cp)
mem.MakeMemory()
mem.BindPythonFunctions(globals())
return __call__()
return optimized
except StopIteration:
# no supported architecture found
def decorate(f):
return f

return decorate


class hello_world(object):
"""Hello world with assembler optimisation for x86."""

_asm_x86 = r"""
!CHARS hello_str 'Hello x86 world!\n\0'

!PROC __call__ PYTHON
!ARG self
!ARG args
PUSH hello_str
CALL PySys_WriteStdout
ADD ESP, 0x4
MOV EAX,PyNone
ADD [EAX],1
!ENDPROC
"""

@asm_optimise('x86')
def __call__(self):
print 'Hello python world!'

test = hello_world()
test()


class hello_world2(object):
"""Hello world with assembler optimisation for unknown architecture."""

# fake unknown architecture asm
_asm_unknown = r"""
!CHARS hello_str 'Hello unknown world!\n\0'

!PROC __call__ PYTHON
!ARG self
!ARG args
PUSH hello_str
CALL PySys_WriteStdout
ADD ESP, 0x4
MOV EAX,PyNone
ADD [EAX],1
!ENDPROC
"""

@asm_optimise('unknown')
def __call__(self):
print 'Hello python world!'

test = hello_world2()
test()


class hello_world3(object):
"""Hello world with assembler optimisation for x86."""

_asm_x86 = r"""
!CHARS hello_str 'Hello x86 world!\n\0'

!PROC __call__ PYTHON
!ARG self
!ARG args
PUSH hello_str
CALL PySys_WriteStdout
ADD ESP, 0x4
MOV EAX,PyNone
ADD [EAX],1
!ENDPROC
"""

_asm_i386 = r"""
!CHARS hello_str 'Hello i386 world!\n\0'

!PROC __call__ PYTHON
!ARG self
!ARG args
PUSH hello_str
CALL PySys_WriteStdout
ADD ESP, 0x4
MOV EAX,PyNone
ADD [EAX],1
!ENDPROC
"""

# i386 will be tried first, then x86
@asm_optimise('i386','x86')
def __call__(self):
print 'Hello python world!'

test = hello_world3()
test()

 

[Michael Spencer]

[an example of using decorators to control pyasm]

Another (perhaps wacky) approach would be to change the assembler source syntax
enough to make it legal Python - in particular, this means parenthesizing the
arguments - then it can just be stored in-line with other Python source. This
has the additional benefit that one could write support functions to enable the
source to be executed interactively in Python.

The following example uses the CPython opcodes, represented as Python functions.
Python control structures 'while' and 'if' are used as assembler directives
for flow.

Michael

>>> import ackermann
>>> Ackermann = assemble(ackermann.ackSrc) [snip assembler output]
>>> Ackermann

>>> Ackermann(3,8)

2045


# ackermann.py --------------------------------------------------

def ackSrc(m,n):
"Compute Ackermann's function on a stack"
# Can be assembled to Python bytecode, or (not implemented yet)
# executed in Python with suitable support functions


LOAD_CONST("Return")
LOAD_FAST(m)
LOAD_FAST(n)

while condition(ROT_TWO(), DUP_TOP(), LOAD_CONST("Return"), COMPARE_OP("!=")):

if condition(POP_TOP(), DUP_TOP(), LOAD_CONST(0), COMPARE_OP("==")):
POP_TOP()
POP_TOP()
LOAD_CONST(1)
BINARY_ADD()

else:
if condition(POP_TOP(), ROT_TWO(), DUP_TOP(), LOAD_CONST(0),
COMPARE_OP("==")):
POP_TOP()
POP_TOP()
LOAD_CONST(1)
BINARY_SUBTRACT()
LOAD_CONST(1)
else:
POP_TOP()
DUP_TOP()
LOAD_CONST(1)
BINARY_SUBTRACT()
ROT_THREE()
POP_TOP()
DUP_TOP()
LOAD_CONST(1)
BINARY_SUBTRACT()
ROT_THREE()
ROT_TWO()
POP_TOP()
POP_TOP()
return

# ByteCode.py --------------------------------------------------

"""Python ByteCode Assembler

Author: Michael Spencer
Version: 0 - Experiment
3/11/2005

Example usage:

>>> import ackermann
>>> Ackermann = assemble(ackermann.ackSrc) [snip assembler output]
>>> Ackermann

>>> Ackermann(3,8)

2045
"""


import dis
import compiler
import compiler.ast as ast
opmap = dis.opmap
import new
import inspect


class AbstractVisitor(object):
"""Standard depth-first AST walker - dispatches to methods
based on Node class name"""
def __init__(self):
self._cache = {} # dispatch table

def visit(self, node,**kw):
#print "Visiting: %s" % node.__class__

if node is None: return None
cls = node.__class__
meth = self._cache.setdefault(cls,
getattr(self,'visit'+cls.__name__,self.default))
return meth(node, **kw)

def default(self, node, **kw):
for child in node.getChildNodes():
self.visit(child, **kw)
visitExpression = default


class Assembler(AbstractVisitor):
"""Python bytecode assembler"""

def __init__(self):
self._cache = {} # dispatch table
self.i = 0 # Bytecode instruction counter

self.co_varnames = []
self.co_consts = []
self.jumptable = {}
self.co_codelst = []

def emit(self, funcname, arg = None):
i = self.i
try:
opcode = opmap[funcname]
except KeyError:
raise SyntaxError, "Unknown operation: %s" % funcname
self.co_codelst.append(opcode)
if opcode > dis.HAVE_ARGUMENT:
print "%4s %4s %s %s" % (i, opcode, funcname.ljust(20), arg)
self.co_codelst.extend(self._getlohi(arg))
self.i = i + 3
else:
print "%4s %4s %s" % (i, opcode, funcname.ljust(20))

self.i = i + 1

def getcodestring(self):
self._resolvejumps()
return "".join(map(chr, self.co_codelst))

def getcode(self):
return new.code(self.co_argcount, # argcount
self.co_argcount, # nlocals
10000, # stacksize
67, # flags
self.getcodestring(), # codestring
tuple(self.co_consts), # constants
tuple(self.co_varnames), # names
tuple(self.co_varnames), # varnames
"assembly", # filename
self.co_name, # name
0, # firstlineno
"" # lnotab
)

def _getlohi(self, arg):
if isinstance(arg, int):
return arg % 256, arg / 256
else:
return None,None

def _resolvejumps(self):
for origin, dest in self.jumptable.iteritems():
self.co_codelst[origin+1rigin+3] = self._getlohi(dest - origin - 3)

def visitFunction(self, node, **kw):

self.co_name = node.name
self.co_argcount = len(node.argnames)
self.co_varnames.extend(node.argnames)

print "def %s(%s)" % (node.name, node.argnames)
self.visit(node.code)



def visitCallFunc(self,node,**kw):
funcname = node.node.name
if funcname == 'COMPARE_OP': # Special case
try:
comptype = node.args[0].value
arg = self.comparisons.index(comptype)
except ValueError:
raise SyntaxError, "Unknown comparison %s" % comptype
else:
args = [self.visit(arg) for arg in node.args]
if args:
arg = args[0]
else:
arg = None

if funcname != "condition": # special case, emits no code
self.emit(funcname, arg)

comparisons = list(dis.cmp_op)

def visitConst(self, node, **kw):
val = node.value
try:
return self.co_consts.index(val)
except ValueError:
self.co_consts.append(val)
return len(self.co_consts)-1

def visitName(self, node, **kw):
name = node.name
try:
return self.co_varnames.index(name)
except ValueError:
self.co_varnames.append(name)
return len(self.co_consts)-1

def visitIf(self, node, **kw):
"node.tests = [(condition, stmt),...] node.else_ = stmt"

self.visit(node.tests[0][0]) # Get the predicate suite

jumpbase = self.i
self.emit("JUMP_IF_FALSE")
self.visit(node.tests[0][1])

if node.else_:
elsebase = self.i
self.emit("JUMP_FORWARD")
self.jumptable[jumpbase] = self.i
print ">> comefrom %s" % jumpbase

self.visit(node.else_)
self.jumptable[elsebase] = self.i

print ">> comefrom %s" % elsebase
else:
self.jumptable[jumpbase] = self.i
print ">> comefrom %s" % jumpbase

def visitReturn(self, node, **kw):
self.emit("RETURN_VALUE")


def visitWhile(self, node, **kw):

loopstart = self.i
self.visit(node.test)
looptest = self.i
self.emit("JUMP_IF_FALSE")

self.visit(node.body)
self.emit("JUMP_ABSOLUTE",loopstart)
print ">> comefrom %s" % looptest
self.jumptable[looptest] = self.i


def assemble(source):
if type(source) == type(assemble):
source = inspect.getsource(source)

tree = compiler.parse(source)
bc = Assembler()
bc.visit(tree)
return new.function(bc.getcode(),globals())

 

[olsongt]

[JanC]

The code below makes it possible to write assembler code for different
architectures (in case pyasm ever supports that and also a Python
code version to use when running on a system where no assembler code
can be used. It prints:
[Michael]
Another (perhaps wacky) approach would be to change the assembler
source syntax
enough to make it legal Python - in particular, this means parenthesizing
the
arguments - then it can just be stored in-line with other Python source. This
has the additional benefit that one could write support functions to enable
the
source to be executed interactively in Python.

The following example uses the CPython opcodes, represented as Python

functions.
Python control structures 'while' and 'if' are used as assembler directives
for flow.

I've been trying to come up with responses to these two posts, but I'm
just plain speechless. That should be taken as a complement.