Compiler, ast and forwards/backwards compatibility

[Orestis Markou]

Hello,

I'm the developer of PySmell ( http://github.com/orestis/pysmell ), a
static analysis/intellisense provider for Python. I am targeting
Python 2.4 code so I'm using the compiler package.

I've been toying around yesterday with the ast module in Python 2.6
and it seems much more cleaner. One thing I don't understand is how
should one handle backwards and forwards compatibility.

The documentation for the ast module states that it "helps to find out
programmatically what the current grammar looks like". I can't find
any reference (even when reading the code) on how you should go about
this, other than checking the sys.version number and reading up on the
changes.

My understanding is that there is no way to write, say, an ast visitor
that runs under Python 3.0 that targets 2.4 because the ast has
changed, and there's no way to indicate that you want to parse another
version.

I guess that Python 2.6 can target Python 2.3-6, and with specific
compiler flags it can also target 3.0, so it seems that the correct
thing to do is to use that.

Am I correct? Am I seriously confused? Please help!

Thanks,
Orestis

 

[Martin v. Löwis]

The documentation for the ast module states that it "helps to find out

programmatically what the current grammar looks like". I can't find
any reference (even when reading the code) on how you should go about
this, other than checking the sys.version number and reading up on the
changes.

Not sure what "this" is, but if you mean what you quoted - what does
that have to do with version numbers?

To find out what the grammar looks like, just inspect the classes in
the _ast module, e.g.

py> _ast.For._fields
('target', 'iter', 'body', 'orelse')
py> _ast.For._attributes
['lineno', 'col_offset']

In any case, you shouldn't look at sys.version, but at _ast.__version__

To see the source code version of that, look at Python/Parser.asdl.

My understanding is that there is no way to write, say, an ast visitor
that runs under Python 3.0 that targets 2.4 because the ast has
changed, and there's no way to indicate that you want to parse another
version.

I wouldn't say that. The writer might not be trivial, but should be
fairly simple. It can't be 1:1, because, as you say, the AST has
changed.

I guess that Python 2.6 can target Python 2.3-6, and with specific
compiler flags it can also target 3.0, so it seems that the correct
thing to do is to use that.

Depends on what you want to do. To transform source code so that
people can still read and understand it, the _ast module might be
inappropriate, as it drops all comments.

For code-rewriting applications, look at lib2to3 instead.

Am I correct? Am I seriously confused? Please help!

I think you are a little confused.

Regards,
Martin