A query about list

[Santanu Chatterjee]

Hello everybody,

I am very new to python. I have a query about
list in python.

Suppose I have a list
a = [1,[2,3,4],5,6,7,[8,9,10],11,12]
I want to know if there is any simple python
facility available that would expand the above list
to give
a = [1,2,3,4,5,6,7,8,9,10,11,12]

I know I can do that with a type() and a for/while loop,
but is there any simpler way?

Regards,
Santanu

 

[Dave Benjamin]

I am very new to python. I have a query about
list in python.

Suppose I have a list
a = [1,[2,3,4],5,6,7,[8,9,10],11,12]
I want to know if there is any simple python
facility available that would expand the above list
to give
a = [1,2,3,4,5,6,7,8,9,10,11,12]

I know I can do that with a type() and a for/while loop,
but is there any simpler way?

Normally, I'd suggest "reduce(operator.add, ...)" to flatten a list, but
since you've got some "naked" entries, that won't work...

I don't know if you consider this simpler, but you could define a reduction
function that checks the type of the second argument, and use it like this:
.... if type(y) is type([]): return x + y
.... return x + [y]
....

reduce(merge, a, [])

[1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12]

Dave

 

[=?ISO-8859-1?Q?Gerhard_H=E4ring?=]

Hello everybody,

I am very new to python. I have a query about
list in python.

Suppose I have a list
a = [1,[2,3,4],5,6,7,[8,9,10],11,12]
I want to know if there is any simple python
facility available that would expand the above list
to give
a = [1,2,3,4,5,6,7,8,9,10,11,12]

I know I can do that with a type() and a for/while loop,
but is there any simpler way?

Why do you have such a data structure in the first place? Can't it be
avoided?

If for example you made the mistake of .append()-ing lists to a list,
then you can .extend() it instead.

-- Gerhard

 

[Santanu Chatterjee]

Suppose I have a list
a = [1,[2,3,4],5,6,7,[8,9,10],11,12]
I want to know if there is any simple python facility available that
would expand the above list to give
a = [1,2,3,4,5,6,7,8,9,10,11,12]

I know I can do that with a type() and a for/while loop, but is there
any simpler way?

I don't know if you consider this simpler, but you could define a
reduction function that checks the type of the second argument, and use it
like this:
... if type(y) is type([]): return x + y
... return x + [y]
...

reduce(merge, a, [])

[1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12]

Thanks for the quick reply. Yes, this is similar to what I would do
using for loop.

But by 'simpler' what I really meant was that manipulating
the list 'a' itself without creating a new list object.
I read that creating new objects is time consuming (and that it is
better to use fill a list and then 'join' to create a string, than to
increase a string using += ). Sorry I forgot to mention it before.

I was trying something like a.insert(1,a.pop(1)) but I need to
modify a.pop(1) somehow so that the brackets vanish ...you know
what I mean. Is that possible ?

Regards,
Santanu

 

[Dave Benjamin]

I was trying something like a.insert(1,a.pop(1)) but I need to
modify a.pop(1) somehow so that the brackets vanish ...you know
what I mean. Is that possible ?

Ahh, I see what you mean, now. You probably want slice assignment.

Try, for starters:
a[1:2] = a[1]

You'll probably still need to use type().

Dave

 

[Dave Benjamin]

I was trying something like a.insert(1,a.pop(1)) but I need to
modify a.pop(1) somehow so that the brackets vanish ...you know
what I mean. Is that possible ?

Ahh, I see what you mean, now. You probably want slice assignment.

Try, for starters:
a[1:2] = a[1]

You'll probably still need to use type().

This *seems* to work, but I have a sneaking feeling there's a bug in here as
a result of the slice assignment. I haven't been able to find a boundary
case that breaks this, but it seems like the slice assignment would mess up
the indexes in the loop. Can anyone find a way to either break or improve
this?

a = [1, [2, 3, 4], 5, 6, 7, [8, 9, 10], 11, 12]
def flatten_sublists(lst):

.... for i, item in enumerate(lst):
.... if type(item) is type([]):
.... lst[i:i+1] = lst
....[1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12]

 

[Santanu Chatterjee]

I was trying something like a.insert(1,a.pop(1)) but I need to modify
a.pop(1) somehow so that the brackets vanish ...you know what I mean. Is
that possible ?

Ahh, I see what you mean, now. You probably want slice assignment.

Try, for starters:
a[1:2] = a[1]

You'll probably still need to use type().

Yes, this is what I wanted.
Thanks for this simple solution.

Regards,
Santanu

 

[Santanu Chatterjee]

Why do you have such a data structure in the first place? Can't it be
avoided?

Well, maybe, but I was trying to get an entire file into a list with
the ASCII values of the characters in binary form (not actually 'binary'
but as a list containing 8 1s and 0s) as its contents, then expand the
lists in place to get a long stream of 1s and 0s for some mathematical
operation.
Dave provided a simple solution which should have occurred to me in the
first place

Thanks, anyway.

Regards,
Santanu

 

[Terry Reedy]

This *seems* to work, but I have a sneaking feeling there's a bug in here as
a result of the slice assignment. I haven't been able to find a boundary
case that breaks this, but it seems like the slice assignment would mess up
the indexes in the loop. Can anyone find a way to either break or improve
this?

I believe enumerate is a generator which makes one pair at a time, as
needed. If so, then you are iterating over inserted items after the
first, which may or may not be what is wanted. I believe an empty
sublist would be deleted, shifting remainder to left and skipping next
item. Try [1, [], [2,3,5]] and also [[[1,2],3], [4,[5,6]]]

a = [1, [2, 3, 4], 5, 6, 7, [8, 9, 10], 11, 12]
def flatten_sublists(lst):

... for i, item in enumerate(lst):
... if type(item) is type([]):
... lst[i:i+1] = lst
...[1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12]

If my hypothesis is correct, I might rewrite with while i loop to
either iterate over either all (for multilevel flatten) or none (for
two-level flatten) of inserted items.

Terry J. Reedy

 

[Peter Otten]

I was trying something like a.insert(1,a.pop(1)) but I need to
modify a.pop(1) somehow so that the brackets vanish ...you know
what I mean. Is that possible ?

Ahh, I see what you mean, now. You probably want slice assignment.

Try, for starters:
a[1:2] = a[1]

You'll probably still need to use type().

This *seems* to work, but I have a sneaking feeling there's a bug in here
as a result of the slice assignment. I haven't been able to find a
boundary case that breaks this, but it seems like the slice assignment
would mess up the indexes in the loop. Can anyone find a way to either
break or improve this?

a = [1, [2, 3, 4], 5, 6, 7, [8, 9, 10], 11, 12]
def flatten_sublists(lst):

... for i, item in enumerate(lst):
... if type(item) is type([]):
... lst[i:i+1] = lst
...[1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12]

Here's a variant that might interest you. It entirely omits indices and
slices and can deal with recursively nested sequences.
You need not create a list at all, if you want to iterate over the elements
one at a time.

<code>
def flatten(seq):
""" Flatten a sequence of sequences of sequences...
"""
# avoid infinite recursion with strings
if isinstance(seq, basestring):
yield seq
else:
try:
for i in seq:
for k in flatten(i):
yield k
except TypeError:
yield seq

# seems to work with the borderline cases
assert list(flatten([1,[2,[3,4]],5,6,7,[8,9,10],11,12])) == range(1, 13)
assert "ABCDEFGHIJKL" == "".join(
flatten(["A", ["B", ["C", "D"]], "E", "F", "G", ["H", "I", "J"], "K",
"L"]))
assert list(flatten([1, [], [2,3,4,5]])) == range(1, 6)
assert list(flatten([[[1,2],3], [4,[5,6]]])) == range(1, 7)
assert list(flatten([[]])) == []
assert list(flatten([])) == []

# but beware of infinite iterators anywhere in your tree...
#from itertools import count
#for i in flatten(count()): print i
</code>

Peter

 

[Dave Benjamin]

Hi, Peter!

Here's a variant that might interest you. It entirely omits indices and
slices and can deal with recursively nested sequences.
You need not create a list at all, if you want to iterate over the elements
one at a time.

def flatten(seq):
""" Flatten a sequence of sequences of sequences...
"""
...

Nice work! I've occasionally used a flatten after a bunch of list processing
because it made a sequence of not-quite-parallel operations seem more
parallel. I saw it as unfortunate that I was creating all of these extra
lists, but it seemed a fair tradeoff between clarity and efficiency. I think
your example shows that if I had only understood laziness better (hehe) I
might not have had to make a tradeoff at all.

Thanks,
Dave

 

[Peter Hansen]

Thanks for the quick reply. Yes, this is similar to what I would do
using for loop.

But by 'simpler' what I really meant was that manipulating
the list 'a' itself without creating a new list object.
I read that creating new objects is time consuming (and that it is
better to use fill a list and then 'join' to create a string, than to
increase a string using += ). Sorry I forgot to mention it before.

Executing _any_ statement is time consuming. The question is
how much time it consumes, though, right? Certainly creating
a list is a fairly simple operation in Python, and I think
you should focus more on readability and functionality (making
it work) rather than worrying about performance issues right
now. Also, modifying something in-place tends to be more
error-prone and dangerous than building a new item from pieces
of the old (e.g. when doing multi-threaded stuff, you could
access an inconsistent view of something). Better just to
go with the typical Python meaning of "simpler", which has
nothing to do with whether something creates new objects.

-Peter

 

[Alex Martelli]

Dave Benjamin wrote:
...

Normally, I'd suggest "reduce(operator.add, ...)" to flatten a list, but

It would be an utter waste anyway, in Python 2.3. The sum() builtin
is quite a bit faster, as well as obviously simpler.

since you've got some "naked" entries, that won't work...

Yeah, there's the rub, of course. But I don't think I have anything
to add to the many ideas you and others have proposed on this;
just wanted to point out sum's existence and preferability;-).


Alex

 

[Dave Benjamin]

Yeah, there's the rub, of course. But I don't think I have anything
to add to the many ideas you and others have proposed on this;
just wanted to point out sum's existence and preferability;-).

Hey, I didn't even think of that. Lots of little goodies in 2.3. =)