Recompile AST?

[chrisspen]

Is it possible to recompile the AST generated by compiler.parse, back
into code or an executable code object? My aim here is to allow a
script to manipulate Python code as elements within a list. However, it
doesn't look like the module has any native methods designed to do
this.

Chris

 

[=?iso-8859-1?Q?Fran=E7ois?= Pinard]

[[email protected]]

Is it possible to recompile the AST generated by compiler.parse, back
into code or an executable code object? My aim here is to allow
a script to manipulate Python code as elements within a list. However,
it doesn't look like the module has any native methods designed to do
this.

Pynits does something like this for pretty-printing Python statements,
from within Vim. I did not touch this project for a while, yet I still
intend to revisit it in a few weeks or months (for an actual need). You
may peek at:

http://fp-etc.progiciels-bpi.ca/pynits.html

 

[Bengt Richter]

Is it possible to recompile the AST generated by compiler.parse, back
into code or an executable code object? My aim here is to allow a
script to manipulate Python code as elements within a list. However, it
doesn't look like the module has any native methods designed to do
this.

It is possible, but if you want code from a particular python source
string that you are giving to compiler.parse, why not use compile?
Or do you want to modify the AST and then compile? This can be done too,
but modifying the AST can be tricky.

Based on the starting point providided by
http://groups.google.com/group/comp...read/thread/5fa80186d9f067f4/7a2351b221063a8c

I've done a few experiments, e.g.,
http://groups.google.com/group/comp.lang.python/msg/9a3f473ae5c23751

I've also posted sporadic musings about the possibilities for AST-transforming
custom import functions to do optimizations and macros and special forms etc.,
but no one seemed much interested (or maybe they quietly went off to do something
of their own ;-)

I've seen some references to new AST stuff in the latest version, but I haven't
got the new version. I really must clean up my disk for space ;-/

Regards,
Bengt Richter

 

[Paul Boddie]

I've also posted sporadic musings about the possibilities for AST-transforming
custom import functions to do optimizations and macros and special forms etc.,
but no one seemed much interested (or maybe they quietly went off to do
something of their own ;-)

For an example of AST transformations, take a look at the
libxml2macro.py script in the libxml2dom distribution [1]: it contains
code which transforms ASTs based on simple rules (changing method calls
to function calls) as well as the magic sequence of instructions to
generate .pyc files from ASTs. Originally, the analysis distribution
[2] also compiled modified ASTs to Python bytecode, but such activities
didn't really fit in with the goals of that particular project.

Paul

[1] http://www.python.org/pypi/libxml2dom
[2] http://www.python.org/pypi/analysis

 

[Leif K-Brooks]

Is it possible to recompile the AST generated by compiler.parse, back
into code or an executable code object?

Into a bytecode object:
42

Into Python source code: <http://ecritters.biz/asttosource.py>.

 

[Chris Spencer]

Into a bytecode object:

42

Into Python source code: <http://ecritters.biz/asttosource.py>.

Thanks, that's exactly what I was looking for. I had almost figured that
out, but I had overlooked the need for set_filename.

Chris

 

[Bengt Richter]

I've also posted sporadic musings about the possibilities for AST-transforming
custom import functions to do optimizations and macros and special forms etc.,
but no one seemed much interested (or maybe they quietly went off to do
something of their own ;-)

For an example of AST transformations, take a look at the
libxml2macro.py script in the libxml2dom distribution [1]: it contains
code which transforms ASTs based on simple rules (changing method calls
to function calls) as well as the magic sequence of instructions to
generate .pyc files from ASTs. Originally, the analysis distribution
[2] also compiled modified ASTs to Python bytecode, but such activities
didn't really fit in with the goals of that particular project.

Paul

[1] http://www.python.org/pypi/libxml2dom
[2] http://www.python.org/pypi/analysis

Thanks for the links. I downloaded the tgzs but it will be a while
before I get into anything ;-)

Regards,
Bengt Richter