On python syntax...

[Steve H]

I'm writing a library in which I have a stream object supporting a
Write mehod, and I require to provide support for writing arbitrary
strings in arbitrary nested contexts. A simplified example solution
might look like this.

--
# Simplified matching marker calls.
def Open(s) :
s.write("(")
def Close(s) :
s.write(")")

#Main body
def Square(s) :
s.Open()
for i in range(100) :
s.open
for j in range(100) :
s.write("{%s,%s}"%i,j)
s.close
s.close
--

While this solution would work, I'm less than happy that the
programmer is left to ensure calls to Open are matched with calls to
close. It is significant to note that every call to open is at the
beginning of a nested context, and every call to close is at the
corresponing end of the same nested context.

Having glanced through the python manual I notice that the C++ trick
of using an object with a destructor to manage this sort of behaviour
is inappropriate for a phython script (as __del__ may be called at any
time once the ref-count for an object is 0.) I wonder, is there a
better approach to this problem than the solution above (maybe using
lambda functions?) I'd like a main body of the following form to
generate the same result:

def Square(s) :
ManageContext(s)
for i in range(100) :
ManageContext(s)
for j in range(100) :
s.write("{%s,%s}"%i,j)

Any suggestions?

 

[Jp Calderone]

[snip]

Having glanced through the python manual I notice that the C++ trick
of using an object with a destructor to manage this sort of behaviour
is inappropriate for a phython script (as __del__ may be called at any
time once the ref-count for an object is 0.) I wonder, is there a
better approach to this problem than the solution above (maybe using
lambda functions?) I'd like a main body of the following form to
generate the same result:

def Square(s) :
ManageContext(s)
for i in range(100) :
ManageContext(s)
for j in range(100) :
s.write("{%s,%s}"%i,j)

Any suggestions?

from StringIO import StringIO
import types

def managedIterator(s, i):
s.write("(")
for e in i:
if isinstance(e, types.GeneratorType):
managedIterator(s, e)
else:
s.write(e)
s.write(")")


def square():
def outer():
for i in range(100):
yield inner(i)
def inner(i):
for j in range(100):
yield "{%s,%s}" % (i, j)
return outer()

s = StringIO()
managedIterator(s, square())
print s.getvalue()


In 2.4, it seems, you will even be able to rewrite square() as:

def square():
return (("{%s,%s}" % (i, j) for j in range(100)) for i in range(100))

Jp

 

[John Roth]

I'm writing a library in which I have a stream object supporting a
Write mehod, and I require to provide support for writing arbitrary
strings in arbitrary nested contexts. A simplified example solution
might look like this.

--
# Simplified matching marker calls.
def Open(s) :
s.write("(")
def Close(s) :
s.write(")")

#Main body
def Square(s) :
s.Open()
for i in range(100) :
s.open
for j in range(100) :
s.write("{%s,%s}"%i,j)
s.close
s.close
--

While this solution would work, I'm less than happy that the
programmer is left to ensure calls to Open are matched with calls to
close. It is significant to note that every call to open is at the
beginning of a nested context, and every call to close is at the
corresponing end of the same nested context.

Having glanced through the python manual I notice that the C++ trick
of using an object with a destructor to manage this sort of behaviour
is inappropriate for a phython script (as __del__ may be called at any
time once the ref-count for an object is 0.) I wonder, is there a
better approach to this problem than the solution above (maybe using
lambda functions?) I'd like a main body of the following form to
generate the same result:

def Square(s) :
ManageContext(s)
for i in range(100) :
ManageContext(s)
for j in range(100) :
s.write("{%s,%s}"%i,j)

Any suggestions?

If I wanted things to happen automatically, I'd do one
of two things.

A. Build a stack so that I could insert ending syntax
when I popped something off the stack

B. Use an object oriented approach and build a
tree. Then I could simply tell the tree to write
itself, and each object could handle the ending
syntax.

Also: please don't use tabs for indentation when you're
posting code snippets. There are a number of e-mail and
news readers that simply ignore tabs, causing your
entire program to be neatly left justified, and making
it difficult to read. I've replaced your tabs with spaces
in this reply.

John Roth

 

[Peter Otten]

I'm writing a library in which I have a stream object supporting a
Write mehod, and I require to provide support for writing arbitrary
strings in arbitrary nested contexts. A simplified example solution
might look like this.

--
# Simplified matching marker calls.
def Open(s) :
s.write("(")
def Close(s) :
s.write(")")

#Main body
def Square(s) :
s.Open()
for i in range(100) :
s.open
for j in range(100) :
s.write("{%s,%s}"%i,j)
s.close
s.close
--

While this solution would work, I'm less than happy that the
programmer is left to ensure calls to Open are matched with calls to
close. It is significant to note that every call to open is at the
beginning of a nested context, and every call to close is at the
corresponing end of the same nested context.

Having glanced through the python manual I notice that the C++ trick
of using an object with a destructor to manage this sort of behaviour
is inappropriate for a phython script (as __del__ may be called at any
time once the ref-count for an object is 0.) I wonder, is there a
better approach to this problem than the solution above (maybe using
lambda functions?) I'd like a main body of the following form to
generate the same result:

def Square(s) :
ManageContext(s)
for i in range(100) :
ManageContext(s)
for j in range(100) :
s.write("{%s,%s}"%i,j)

Any suggestions?

Below is a Context class with a random mix of ways to tackle your problem.
Instead of

ctx.open()
yourfunc(ctx, arg1, ..., argN)
ctx.close()

just write

ctx.wrap(yourfunc, arg1, ..., argN)

While in doubt about the wrap() and wrapItems() methods, I think that the
level-counting approach is worth consideration. Store the level at the
begin of a function and assert that it has not changed at the end, putting
it into a try ... finally statement for functions with more then one exit
point (or just query it with your debugger).

class Context:
def __init__(self, write=None):
if write is None:
import sys
write = sys.stdout.write
self.write = write
self.level = 0
def open(self):
self.write("(")
self.level += 1
def close(self):
assert self.level > 0
self.write(")")
self.level -= 1
def done(self):
assert self.level == 0
def wrap(self, fun, *args):
self.open()
fun(self, *args)
self.close()
def wrapItems(self, seq, mapfunc=str):
for item in seq:
self.open()
self.write(mapfunc(item))
self.close()

def b(ctx, i, j):
ctx.write("%s %s" % (i, j))

def square(ctx, n, m) :
for i in range(n):
ctx.open()
for j in range(m) :
ctx.wrap(b, i, j)
ctx.close()
ctx.write("\n")

ctx = Context()
ctx.wrap(square, 5, 4)
ctx.write("\n")
ctx.wrapItems(range(6), lambda x: str(x*x))
ctx.done()

Peter

 

[Steve H]

Peter Otten <[email protected]> and others posted some interesting
replies:

Thanks very much for the various suggestions.

John: I'd thought of building a tree then calling write on it
myself... in one sense I find that approach neat, however it has a
significant disadvantage when it comes to writing long streams - it
needs the whole object allocated before I start writing - hence
introducing a potentially large latency and also, potentially,
requiring many costly memory allocations - which may be a significant
barrier to performance when used on 'longer' streams.
The "use a stack" idea is OK - in so far as it solves the problem of
matching up the write closing token (should I have many different tags
- say, with XML) but doesn't address my main concern - i.e. that I
don't want to have to write explicit code to output the close-context
marker - whatever it is.

Jp: Very neat Python 2.4 answer - I like it... but it doesn't really
address my current concerns... it's a touch too clever for me as it
relies upon details of the specific example I posted which don't hold
in my 'real-world' examples. I'm also concerned that I'd need to
allocate the whole string before writing it - which would cause high
latency and significant resource usage in a streaming environment.

Peter: This is far more like the sort of answer I'd hoped for... I
think something like wrap() might be OK - I hadn't realised I could
pass arbitrarily many arguments like that. One 'down' side to this is
that I'd have prefered not to have to define a new function for each
of my contexts, though I suppose this might not be too much of a
hardship. I'd have preferred to be able to have many different nested
contexts in a single function (to share local variables etc) - though
I guess that this isn't going to be possible. The wrapitems()
implementation is neat - I like it... but I don't think it would apply
to my real world problem - where I expect to do more complex things
than just iterate a fixed loop - I envision other "contexts" for
example: conditionals, while loops, functions etc.

I still have a feeling that some way to hook the close of an execution
context would be extremely useful... though I'm beginning to believe
that this isn't currently supported. I'd be completely satisfied with
wrap() if I could define the wrapped function inline somehow - yet
still execute it within wrap. I'd want the wrapped function to be
anonymous and to have access to all the local variables available to
wrap() - without having to explicity pass each one to the wrapped
function... I guess the obvious lambda args:... approach isn't going
to be effective because I'd still need to specify every argument
separately.

Steve

 

[Diez B. Roggisch]

While this solution would work, I'm less than happy that the

programmer is left to ensure calls to Open are matched with calls to
close. It is significant to note that every call to open is at the
beginning of a nested context, and every call to close is at the
corresponing end of the same nested context.

Maybe you can take advantage of python beeing an interpreted language and
try to analyse the code fragment your programmer gives to you. As long as
he doesn't do any really nasty stuff, you could try to analyse the parse
tree and find blocks finishing without a close call. The parser module
would make this possible. I'm not sure if its possible to get a parse tree
out of a compiled function, but somewhere the source code must be, so you
should be able to access it.

Mind you, I never worked with that stuff before - I think it could be done,
but I'm not sure if its worth the effort

Regards,

Diez