Idea for pure-python templates using AST.

P

Paul Wray

Hello all

Ive had what I think is a great idea for pure-python templates (I can almost
hear the groans, bear with me...)

For the impatient, proof of concept is at http://pastie.org/2379978
demonstrating simple substitution, balanced tags using context manager,
subtemplates, and template inheritance.

I'm posting here to get opinions on:
* the merits of the idea, (or otherwise!)
* whether there are any established/mature templating systems that use this
approach, or whether its come up before,
* ideas for improvements and advice on other aspects such as sandboxing
* of course, to share the idea in case others want to use it

Background: Ive been working on an application that recursively renders
html/xml documents from heterogenoeus trees, with separate classes for each
document component. First I built my own renderer but was dissatisfied with
the repetitive code. Then looked at Mako and Jinja, and used Jinja but was
still disatisfied, because I still needed a render method in each class to
do preparation, and also the template which was centrally managed by the
Jinja loader and environment. I found a way to call templates recursively
via Jinja filters, but was not sure that it wouldnt blow up in my face, so I
also had separate calls to render the children of each node, and then feed
the value to the parent template. I kept thinking that there must be a way
to get the brevity and clarity of declarative templates, plus the simplicity
of pure python loops, tests and function calls.

The idea:
Python syntax allows a statement to be a bare literal or identifier. These
have no effect on the program.

So the function below is legal python:

def myFunc():
'a'
x = 45
'b'; 'c'; x

So is this (within the appropriate class context of course):

def body(self, r):
'<h1>'; self.heading; '</h1>'
'<ul>'
for itm in self.items:
'<li>'; itm; '</li>'
'</ul>'

The idea is simply to use python ASTs to transform this code so that it
accumulates the values of the bare expressions.

I think this give the best of both worlds - declarative syntax for the
template literals, but in a pure python context, giving you all the power of
python control statements, classes etc.

For application contexts outside pure python code (ie user-created
templates) , it would be simple to extend the technique to creating template
functions from strings, and insert the resulting methods into a namespace
for execution.)

I think, given the technique is already working with the AST, that
sandboxing should not be too hard either - advice on this aspect would be
appreciated.

Paul Wray
 
A

aspineux

Hello all

Ive had what I think is a great idea for pure-python templates (I can almost
hear the groans, bear with me...)

For the impatient, proof of concept is athttp://pastie.org/2379978
demonstrating simple substitution, balanced tags using context manager,
subtemplates,  and template inheritance.

You code fail, see below for other comment

Traceback (most recent call last):
File "Download/pastie-2379978.rb", line 108, in <module>
make_template(template1)
File "Download/pastie-2379978.rb", line 60, in make_template
ast.fix_missing_locations(astFromSrc)
File "/usr/lib/python2.6/ast.py", line 133, in fix_missing_locations
_fix(node, 1, 0)
File "/usr/lib/python2.6/ast.py", line 132, in _fix
_fix(child, lineno, col_offset)
File "/usr/lib/python2.6/ast.py", line 132, in _fix
_fix(child, lineno, col_offset)
File "/usr/lib/python2.6/ast.py", line 121, in _fix
if 'lineno' in node._attributes:
AttributeError: 'arguments' object has no attribute '_attributes'



I'm posting here to get opinions on:
* the merits of the idea, (or otherwise!)
* whether there are any established/mature templating systems that use this
approach, or whether its come up before,
* ideas for improvements and advice on other aspects such as sandboxing
* of course, to share the idea in case others want to use it

This is very original ! First time I see it. I like it.
But how to debug large template ?
How to find/detect a missing </TAG> ?
This is very important. This is one big advantage of Genshi over Kid
How to report the correct error at the correct line ?
How to find/get nice editor to edit large template ?

Background: Ive been working on an application that recursively renders
html/xml documents from heterogenoeus trees, with separate classes for each
document component. First I built my own renderer but was dissatisfied with
the repetitive code. Then looked at Mako and Jinja, and used Jinja but was
still disatisfied, because I still needed a render method in each class to
do preparation, and also the template which was centrally managed by the
Jinja loader and environment. I found a way to call templates recursively
via Jinja filters, but was not sure that it wouldnt blow up in my face, so I
also had separate calls to render the children of each node, and then feed
the value to the parent template. I kept thinking that there must be a way
to get the brevity and clarity of declarative templates, plus the simplicity
of pure python loops, tests and function calls.

The idea:
Python syntax allows a statement to be a bare literal or identifier. These
have no effect on the program.

So the function below is legal python:

def myFunc():
    'a'
    x = 45
    'b'; 'c'; x

So is this (within the appropriate class context of course):

def body(self, r):
        '<h1>'; self.heading; '</h1>'
        '<ul>'
        for itm in self.items:
            '<li>'; itm; '</li>'
        '</ul>'

The idea is simply to use python ASTs to transform this code so that it
accumulates the values of the bare expressions.

You could call it PHP :)
I think this give the best of both worlds - declarative syntax for the
template literals, but in a pure python context, giving you all the powerof
python control statements, classes etc.

For application contexts outside pure python code (ie user-created
templates) , it would be simple to extend the technique to creating template
functions from strings, and insert the resulting methods into a namespace
for execution.)

I think, given the technique is already working with the AST, that
sandboxing should not be too hard either - advice on this aspect would be
appreciated.


Maybe a good idea.
But still a lot of work to get the level of existing libraries.

Maybe better if mixed with other tools like Genshi to create widget.
Maybe next toscawidget could use such a technique.


Thanks for sharing
 
T

Terry Reedy

Hello all

Ive had what I think is a great idea for pure-python templates (I can
almost hear the groans, bear with me...)

For the impatient, proof of concept is at http://pastie.org/2379978
demonstrating simple substitution, balanced tags using context manager,
subtemplates, and template inheritance.

I'm posting here to get opinions on:
* the merits of the idea, (or otherwise!)
* whether there are any established/mature templating systems that use
this approach, or whether its come up before,
* ideas for improvements and advice on other aspects such as sandboxing
* of course, to share the idea in case others want to use it

Background: Ive been working on an application that recursively renders
html/xml documents from heterogenoeus trees, with separate classes for
each document component. First I built my own renderer but was
dissatisfied with the repetitive code. Then looked at Mako and Jinja,
and used Jinja but was still disatisfied, because I still needed a
render method in each class to do preparation, and also the template
which was centrally managed by the Jinja loader and environment. I found
a way to call templates recursively via Jinja filters, but was not sure
that it wouldnt blow up in my face, so I also had separate calls to
render the children of each node, and then feed the value to the parent
template. I kept thinking that there must be a way to get the brevity
and clarity of declarative templates, plus the simplicity of pure python
loops, tests and function calls.

The idea:
Python syntax allows a statement to be a bare literal or identifier.
These have no effect on the program.

More generally, Python has expression statements, with the result of the
expression ignored. These are usually function calls with side-effects.
"print('x')" has no effect on the program and the return value is
usually ignored.
So the function below is legal python:

def myFunc():
'a'
x = 45
'b'; 'c'; x

So is this (within the appropriate class context of course):

def body(self, r):
'<h1>'; self.heading; '</h1>'
'<ul>'
for itm in self.items:
'<li>'; itm; '</li>'
'</ul>'

The idea is simply to use python ASTs to transform this code so that it
accumulates the values of the bare expressions.

Interesting idea, though I have no experience for comparison.
 
C

Chris Angelico

The idea is simply to use python ASTs to transform this code so that it
accumulates the values of the bare expressions.

That'd be similar to what the interactive loop does. Are you aware,
though, that docstrings are bare expressions? You may have a syntactic
collision there.

ChrisA
 
A

anand jeyahar

Hi all,
I did it. Finally managed to port mysqltuner.pl to python. Was a
real pain in the butt doing it from bottom up manually, without ever
really learing perl syntax. But i finally got it done. Now i need help
testing it. find it here.
(e-mail address removed):anandjeyahar/mysqlDbAdmin-python.git.

Also i never really thought about design. Just blindly/mechanically,
translated from perl to python. So criticize and let me know how i can
improve this.

Thanks and Regards,
==============================================
Anand Jeyahar
https://sites.google.com/site/anandjeyahar
==============================================
The man who is really serious,
with the urge to find out what truth is,
has no style at all. He lives only in what is.
                  ~Bruce Lee

Love is a trade with lousy accounting policies.
                 ~Aang Jie
 
I

Irmen de Jong

The idea:
Python syntax allows a statement to be a bare literal or identifier.
These have no effect on the program.

So the function below is legal python:

def myFunc():
'a'
x = 45
'b'; 'c'; x

So is this (within the appropriate class context of course):

def body(self, r):
'<h1>'; self.heading; '</h1>'
'<ul>'
for itm in self.items:
'<li>'; itm; '</li>'
'</ul>'

Looks very similar to PTL what Quixote uses:
http://www.quixote.ca/overview/paper.html

(never used it though, and I doubt Quixote used ASTs)

Irmen
 
P

Paul Wray

That'd be similar to what the interactive loop does. Are you aware,
though, that docstrings are bare expressions? You may have a syntactic
collision there.

ChrisA

Thanks yes ama aware of docstrings but did not consider.
They are easy to strip out though.
 
C

Chris Angelico

Thanks yes ama aware of docstrings but did not consider.
They are easy to strip out though.

Maybe. You'd have to take notice of what's a docstring and what's the
first element to be outputted. Or alternatively, just forbid
docstrings on those functions.

ChrisA
 
P

Paul Wray

Looks very similar to PTL what Quixote uses:http://www.quixote.ca/overview/paper.html

(never used it though, and I doubt Quixote used ASTs)

Irmen

Thanks. Yes it looks much the same, and it does use AST. This from the
ptl_compile.py module:

"""Compile a PTL template.

First template function names are mangled, noting the template type.
Next, the file is parsed into a parse tree. This tree is converted
into
a modified AST. It is during this state that the semantics are
modified
by adding extra nodes to the tree. Finally bytecode is generated
using
the compiler package.
"""
 
P

Paul Wray

You code fail, see below for other comment
Traceback (most recent call last):
  File "Download/pastie-2379978.rb", line 108, in <module>
    make_template(template1)
  File "Download/pastie-2379978.rb", line 60, in make_template
    ast.fix_missing_locations(astFromSrc)
  File "/usr/lib/python2.6/ast.py", line 133, in fix_missing_locations
    _fix(node, 1, 0)
  File "/usr/lib/python2.6/ast.py", line 132, in _fix
    _fix(child, lineno, col_offset)
  File "/usr/lib/python2.6/ast.py", line 132, in _fix
    _fix(child, lineno, col_offset)
  File "/usr/lib/python2.6/ast.py", line 121, in _fix
    if 'lineno' in node._attributes:
AttributeError: 'arguments' object has no attribute '_attributes'

Sorry, should have specified Python 2.7.
Also Pastie thinks its ruby code so the donwloaded file is confusingly
___.rb
 

Ask a Question

Want to reply to this thread or ask your own question?

You'll need to choose a username for the site, which only take a couple of moments. After that, you can post your question and our members will help you out.

Ask a Question

Members online

Forum statistics

Threads
473,982
Messages
2,570,190
Members
46,736
Latest member
zacharyharris

Latest Threads

Top