List Problem

[jimbo1qaz]

I have a nested list. Whenever I make a copy of the list, changes in one affect the other, even when I use list(orig) or even copy the sublists one by one. I have to manually copy each cell over for it to work.
Link to broken code: http://jimbopy.pastebay.net/1090401

 

[jimbo1qaz]

I have a nested list. Whenever I make a copy of the list, changes in one affect the other, even when I use list(orig) or even copy the sublists one by one. I have to manually copy each cell over for it to work.

Link to broken code: http://jimbopy.pastebay.net/1090401

No, actually that's the OK code. http://jimbopy.pastebay.net/1090494 is the broken one.

 

[Chris Angelico]

No, actually that's the OK code. http://jimbopy.pastebay.net/1090494 is the broken one.

The first thing I'd change about that code is the whole thing of using
try/exec/except to suppress IndexError. Definitely not good code. I'm
not wholly certain, but I think you might run into weird issues with
negative OOBounds indices (since Python treats a negative list index
as counting from the far end).

This is nothing to do with your originally requested issue, which I
can't see the cause of in your script there. But when you assign a
list, you just get another reference to the same list.

ChrisA

 

[Dave Angel]

No, actually that's the OK code. http://jimbopy.pastebay.net/1090494 is the broken one.

I also would prefer an inline posting of the code, but if it's too big
to post here, it's probably too big for me to debug here.

The usual reason for such a symptom is a nested list, where you have
multiple references to the same inner list inside the outer. When you
change one of those, you change all of them.

alist = [1, 2, 3]
blist = [alist, alist, alist] # or blist = alist * 3
print blist
alist.append(49)
print blist

davea@think:~/temppython$ python jimbo.py
[[1, 2, 3], [1, 2, 3], [1, 2, 3]]
[[1, 2, 3, 49], [1, 2, 3, 49], [1, 2, 3, 49]]

Solution to this is to make sure that only copies of alist get into
blist. One way is

blist = [alist[:], alist[:], alist[:]]

More generally, you can get into this type of trouble whenever you have
non-immutable objects inside the list.

Understand, this is NOT a flaw in the language. It's perfectly
reasonable to be able to do so, in fact essential in many cases, when
you want it to be the SAME item.

 

[Steven D'Aprano]

I have a nested list. Whenever I make a copy of the list, changes in one
affect the other,

Then you aren't making a copy.

py> first_list = [1, 2, 3]
py> second_list = first_list # THIS IS NOT A COPY
py> second_list.append(9999)
py> print first_list
[1, 2, 3, 9999]

even when I use list(orig)

Nonsense. Either you are confused, or there is something you aren't
telling us. Calling list *does* make a copy:

py> first_list = [1, 2, 3]
py> second_list = list(first_list)
py> second_list.append(9999)
py> print first_list
[1, 2, 3]

What aren't you telling us? My guess is that there are TWO lists
involved, and you're only copying ONE:


py> a = [1, 2, 3]
py> a.append(["ham", "spam", "cheese"])
py> print a
[1, 2, 3, ['ham', 'spam', 'cheese']]
py> b = list(a) # make a copy of a, but not the contents of a
py> b.append(99) # change b
py> b[-1].append("tomato")
py> print a
[1, 2, 3, ['ham', 'spam', 'cheese', 'tomato']]

Notice that b is a copy of a: changing b does not change a. But the
embedded list within a is *not* copied, so whether you append to that
list via a or b is irrelevant, both see the same change because there is
only one inner list in two places.

It might be more obvious if you give the shared sublist a name:

c = ['ham', 'spam', 'tomato']
a = [1, 2, 3, c]
b = [1, 2, 3, c] # a copy of a, but not a copy of c

Now it should be obvious that any changes to c will show up in both a and
b, regardless of how you change it. All three of these will have the
exact same effect:

a[-1].append('eggs')
b[-1].append('eggs')
c.append('eggs')


The way to copy lists, and all their sublists, and their sublists, and so
on all the way down, is with the copy module:

import copy
b = copy.deepcopy(a)

but if you are doing this a lot, (1) your code will be slow, and (2) you
can probably redesign your code to avoid so many copies.

By the way, instead of dumping lots of irrelevant code on us, please take
the time to narrow your problem down to the smallest possible piece of
code. We're volunteers here, and you are not paying us to wade through
your code trying to determine where your problem lies. That is up to you:
you narrow down to the actual problem, then ask for help.

Please read http://sscce.org/ for more details of how, and why, you
should do this.


Thank you.

 

[Chris Angelico]

blist = [alist, alist, alist] # or blist = alist * 3

(Minor point: I think you mean this.)
# or blist = [alist] * 3

Understand, this is NOT a flaw in the language. It's perfectly
reasonable to be able to do so, in fact essential in many cases, when
you want it to be the SAME item.

And this is the real part. There's no other way to handle complex
objects that makes as much sense. PHP's system of references and
copy-on-write assignment doesn't truly cover all cases, and it can
make operations unexpectedly run vastly faster or slower depending on
external circumstances, which gets annoying (the first write to an
assigned array has to copy the array). C simply doesn't let you move
arrays around, only pointers to them, so semantics are actually pretty
similar to high level languages, only in a completely different way.

These sorts of issues only ever seem to crop up with nested arrays,
which strengthens this next point: Deep copying is a really REALLY
hairy concept. It seems so simple at first:

a = [[1, 2, 3], [4, 5, 6], [7, 8, 9]]
b = deepcopy(a) # b is a new list with three new sublists

But then it gets messy.

a = [[1, 2, 3]]*2 + [[7, 8, 9]]

It's much better to make copying a very explicit thing; it's so
expensive that you really should make it very clear in your code when
this happens.

ChrisA

 

[Chris Angelico]

a = [[1, 2, 3], [4, 5, 6], [7, 8, 9]]
b = deepcopy(a) # b is a new list with three new sublists

Oops, this should be:

b = copy.deepcopy(a)

as in Steven's post. And in case I wasn't clear about it, the
deepcopy() function does deal with the issue I mention, but that's
part of why it's an expensive operation.

ChrisA

 

[Steven D'Aprano]

C simply doesn't let you move
arrays around, only pointers to them, so semantics are actually pretty
similar to high level languages, only in a completely different way.

I once dated a girl who looked exactly like Scarlett Johannsen only
completely different.

Pascal let's you pass arrays around either by value (which copies them,
and may be expensive) or by reference (which doesn't), neither of which
is what Python (or Java, Ruby, etc.) do. Passing pointers by value is not
the same as Python object semantics.

 

[jimbo1qaz]

I have a nested list. Whenever I make a copy of the list, changes in one affect the other, even when I use list(orig) or even copy the sublists one by one. I have to manually copy each cell over for it to work.

Link to broken code: http://jimbopy.pastebay.net/1090401

OK, deepcopy fixed it!
And I fixed the catch indexerror thing too.

 

[Chris Angelico]

I once dated a girl who looked exactly like Scarlett Johannsen only
completely different.

Pascal let's you pass arrays around either by value (which copies them,
and may be expensive) or by reference (which doesn't), neither of which
is what Python (or Java, Ruby, etc.) do. Passing pointers by value is not
the same as Python object semantics.

If your arrays are allocated on the heap, you can pretty much treat
the pointer like a HLL list object. The only real difference is that
C, by default, won't garbage-collect your heap allocations, so you'll
leak memory. The behaviours you observe will be pretty similar,
though.

ChrisA

 

[Littlefield, Tyler]

No, actually that's the OK code. http://jimbopy.pastebay.net/1090494 is the broken one.

I've not been following this thread fully, but why not just use
x=list(y) to copy the list?
The issue is that when you assign i=[1,2,3] and then j = i, j is just a
reference to i, which is why you change either and the other changes.

--
Take care,
Ty
http://tds-solutions.net
The aspen project: a barebones light-weight mud engine:
http://code.google.com/p/aspenmud
He that will not reason is a bigot; he that cannot reason is a fool; he that dares not reason is a slave.

 

[Chris Angelico]

I've not been following this thread fully, but why not just use x=list(y) to
copy the list?
The issue is that when you assign i=[1,2,3] and then j = i, j is just a
reference to i, which is why you change either and the other changes.

The problem is with lists as elements of that list, so the key is deepcopy.

ChrisA