A faster method to generate a nested list from a template?

[DJTB]

Hi all,

I'm new to Python. I'm trying to create a fast function to do the following:

t = [['a1','a2'],['b1'],['c1'],['d1']]
l = [1,2,3,4,5][[1, 2], [3], [4], [5]]

t is some sort of template. This is what I have now:

def create_nested_list(template,l_orig):
'''Uses a template to create a new list

t = [['a1','a2'],['b1'],['c1'],['d1']]
l = [1,2,3,4,5] [[1, 2], [3], [4], [5]]
'''

tl = map(len, template)
# Input size check
if reduce(lambda x,y: x+y, tl) != len(l_orig):
raise "Wrong input size"

l = l_orig
new_nested_list = []
for x in tl:
q = []
i = 0
while i < x:
q.append(l.pop(0))
i += 1
new_nested_list.append(q)
return new_nested_list

I'd like to know if it is possible to make this faster (using Python magic I
don't know of yet), because this function will be called a lot
('constantly').

Thanks in advance,
Stan.

 

[Jeff Epler]

Are there only a few, unchanging templates? If so, (dynamiclly) create
a function for each template. This will be nearly the fastest you can
go in Python, excluding the time to create and byte-compile the nesting
function.

# This code is in the public domain
def make_nesting_expression(l, s):
result = []
for c in l:
if isinstance(c, list):
sub, s = make_nesting_expression(c, s)
result.append(sub)
else:
result.append("l[%d]" % s)
s += 1
print "make_nesting_expression", l, result
return "[" + ",".join(result) + "]", s

def make_nesting_function(l):
return eval("lambda l: %s" % make_nesting_expression(l, 0)[0])

t = [['a1','a2'],['b1'],['c1'],['d1']]
l = [1, 2, 3, 4, 5]
f = make_nesting_function(t)
print f(l)

Jeff

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.2.1 (GNU/Linux)

iD8DBQFCeNpeJd01MZaTXX0RAn4vAJ4+7ZFhwCYh+QSSqeUoNph/frm32QCggwkt
hOuLLmswcC3Fz4g47NdlJVk=
=ZyPe
-----END PGP SIGNATURE-----

 

[jasondrew72]

Are there only a few, unchanging templates? If so, (dynamiclly) create
a function for each template. This will be nearly the fastest you can
go in Python, excluding the time to create and byte-compile the nesting
function.

# This code is in the public domain
def make_nesting_expression(l, s):
result = []
for c in l:
if isinstance(c, list):
sub, s = make_nesting_expression(c, s)
result.append(sub)
else:
result.append("l[%d]" % s)
s += 1
print "make_nesting_expression", l, result
return "[" + ",".join(result) + "]", s

def make_nesting_function(l):
return eval("lambda l: %s" % make_nesting_expression(l, 0)[0])

t = [['a1','a2'],['b1'],['c1'],['d1']]
l = [1, 2, 3, 4, 5]
f = make_nesting_function(t)
print f(l)

Jeff

Nice approach! Though I'm wondering, because the "template" is so close
to the function you're trying to create, why not just create the
function directly? As in:

f5 = lambda p: (p[0:2], p[2], p[3], p[4])
f5(('a', 'b', 'c', 'd', 'e', 'f'))

(('a', 'b'), 'c', 'd', 'e')

Or if you need to be able to remember what the template looks like
(e.g. reading/writing it as a string in a file), you can always create
the function by evaluating a (properly constructed) template string:

f5 = eval("lambda p: (p[0:2], p[2], p[3], p[4])")
f5(('a', 'b', 'c', 'd', 'e', 'f'))

(('a', 'b'), 'c', 'd', 'e')

If you want to avoid lambdas, the same ideas can be used with short
defs, e.g.:
def f5(p): return (p[0:2], p[2], p[3], p[4])

Incidentally, these solutions raise an IndexError if your list is too
short, and silently drop the end of a list that's too long.

That's my keep-it-simple-I-might-understand-it approach.

Jason