Accessing Python parse trees

[Manlio Perillo]

Hi.
With module parser it is possible to access Python parse trees.
But this only works for 'external' source.
I would like to known if, at least in theory, it can be possible to
access Python parse trees from 'inside' a script.


As a simple example:


def on_parsing(ast):
...


@parsing -> on_parsing
some statement



This will call on_parsing function with the AST object generated from
the next statement:

on_parsing(parser.suit(somestatement))



Thanks and regards Manlio Perillo

 

[Lonnie Princehouse]

No. I don't think it's possible to read the parse tree used by the
interpreter, especially as it is being created. Here are a couple of
kludgy ideas that might come close, though:

1. Use introspection to have your on_parsing function locate the and
parse the code being executed. I'm not sure if you'll be able to
figure out _exactly_ where on_parsing is being called from (e.g. you
can probably only narrow it down to the statement, not the expression),
but you should be able to come close enough to be useful. The inspect
module and sys._getframe will come in handy. Note that this won't work
if you can't retrieve the source (as is the case with exec/eval)

2. Or, using the code module, start up your own interpreter inside of
the interpreter. Intercept and parse lines of source as they are fed
to this interpreter.

On a side note, check out the compiler module. You might find it to be
friendlier and more useful than parser.

 

[Manlio Perillo]

No. I don't think it's possible to read the parse tree used by the
interpreter, especially as it is being created. Here are a couple of
kludgy ideas that might come close, though:

Is this a 'limitation' of the current version or it is impossible for
the architecture of CPython?
What about pypy?

On a side note, check out the compiler module. You might find it to be
friendlier and more useful than parser.

Thanks for the hint. It is what I want.
Unfortunately is seem to be not well documented.


Anyway, here is an example of what I would like to do:

#begin
def foo(**kwargs): print kwargs

foo(a = 1, b = 2, c = 3)
#end


In the current implementation kwargs is a dict, but I need to have the
keyword argument sorted.
Unfortunately subclassing fron dict and installing the class in the
__builtin__ module (with the name 'dict') does not work, CPython uses
only builtin types.

With the compiler module I can obtain the keyword arguments in the
order the were specified.
The problem is how to do this for every call to foo!




Thanks and regards Manlio Perillo

 

[Kent Johnson]

Anyway, here is an example of what I would like to do:

#begin
def foo(**kwargs): print kwargs

foo(a = 1, b = 2, c = 3)
#end


In the current implementation kwargs is a dict, but I need to have the
keyword argument sorted.
Unfortunately subclassing fron dict and installing the class in the
__builtin__ module (with the name 'dict') does not work, CPython uses
only builtin types.

With the compiler module I can obtain the keyword arguments in the
order the were specified.
The problem is how to do this for every call to foo!

Why not just pass the kind of argument you want? What is it you really need to do?

def foo(kwds): print kwds

foo(MyDict(a = 1, b = 2, c = 3))

Kent

 

[Steve Holden]

Is this a 'limitation' of the current version or it is impossible for
the architecture of CPython?
What about pypy?




Thanks for the hint. It is what I want.
Unfortunately is seem to be not well documented.


Anyway, here is an example of what I would like to do:

#begin
def foo(**kwargs): print kwargs

foo(a = 1, b = 2, c = 3)
#end


In the current implementation kwargs is a dict, but I need to have the
keyword argument sorted.
Unfortunately subclassing fron dict and installing the class in the
__builtin__ module (with the name 'dict') does not work, CPython uses
only builtin types.

With the compiler module I can obtain the keyword arguments in the
order the were specified.
The problem is how to do this for every call to foo!

The nature of the interpreter is that the C code implementing function
calls specifically uses a dict created in C rather than using the
mechanisms that would be used to create a dict from within a python
program, so you have no way to "hook" your own implementation in to the
interpreter without modifying the C code.

Introspection does have its limits, and unfortunately that's one of
them. Sine the code that interprets the byte codes is pretty much all
written in C, there's no way to affect something that does not already
have Python run-time hooks provided (such as the __add__ method that
allows you to implement a specific response to the binary "+" operator).

What you probably need is a specific magic hook for the "**" operator,
but I'm not sure that's going to happen any time soon ...

regards
Steve

 

[Manlio Perillo]

Why not just pass the kind of argument you want? What is it you really need to do?

def foo(kwds): print kwds

foo(MyDict(a = 1, b = 2, c = 3))

Kent

I don't understand your code.
Here an example using OrderedDict from twisted:
{'a': 1, 'c': 3, 'b': 2}


Simply I can't use a dict.

I have to do, as an example example:
foo('a', 1, 'b', 2, 'c', 3)

or

foo(['a', 'b', 'c'], [1, 2, 3])


Thanks and regards Manlio Perillo