S
Steven D'Aprano
Yes. But this isn't going to cost any more time than figuring out
whether or not the list multiplication is going to cause quirks, itself.
Human psychology *tends* (it's a FAQ!) to automatically assume the
purpose of the list multiplication is to pre-allocate memory for the
equivalent (using lists) of a multi-dimensional array. Note the OP even
said "4d array".
I'm not entirely sure what your point is here. The OP screwed up -- he
didn't generate a 4-dimensional array. He generated a 2-dimensional
array. If his intuition about the number of dimensions is so poor, why
should his intuition about list multiplication be treated as sacrosanct?
As they say, the only truly intuitive interface is the nipple. There are
many places where people's intuition about programming fail. And many
places where Fred's intuition is the opposite of Barney's intuition.
Even more exciting, there are places where people's intuition is
*inconsistent*, where they expect a line of code to behave differently
depending on their intention, rather than on the code. And intuition is
often sub-optimal: e.g. isn't it intuitively obvious that "42" + 1 should
give 43? (Unless it is intuitively obvious that it should give 421.)
So while I prefer intuitively obvious behaviour where possible, it is not
the holy grail, and I am quite happy to give it up.
The OP's original construction was simple, elegant, easy to read and
very commonly done by newbies learning the language because it's
*intuitive*. His second try was still intuitive, but less easy to read,
and not as elegant.
Yes. And list multiplication is one of those areas where intuition is
suboptimal -- it produces a worse outcome overall, even if one minor use-
case gets a better outcome.
I'm not disputing that [[0]*n]*m is intuitively obvious and easy. I'm
disputing that this matters. Python would be worse off if list
multiplication behaved intuitively.
An analogy: the intuitively obvious thing to do with a screw is to bang
it in with a hammer. It's long, thin, has a point at the end, and a flat
head that just screams "hit me". But if you do the intuitive thing, your
carpentry will be *much worse* than the alternatives -- a hammered in
screw holds much less strongly than either a nail or a screwed in screw.
The surface area available for gripping is about 2% compared to a nail
and about 0.01% compared to a screw used correctly.
Having list multiplication copy has consequences beyond 2D arrays. Those
consequences make the intuitive behaviour you are requesting a negative
rather than a positive. If that means that newbie programmers have to
learn not to hammer screws in, so be it. It might be harder, slower, and
less elegant to drill a pilot hole and then screw the screw in, but the
overall result is better.
* Consistency of semantics is better than a plethora of special
cases. Python has a very simple and useful rule: objects should not
be copied unless explicitly requested to be copied. This is much
better than having to remember whether this operation or that
operation makes a copy. The answer is consistent:
Bull. Even in the last thread I noted the range() object produces
special cases.range(1, 3)range(0,5)[1] 1
range(0,5)[1:3]
What's the special case here? What do you think is copied?
You take a slice of a tuple, you get a new tuple.
You take a slice of a list, you get a new list.
You take a slice of a range object, you get a new range object.
I'm honestly not getting what you think is inconsistent about this.
The principle involved is that it gives you what you *usually* want;
Who is the "you" that decides what "you" usually want? And how do they
know what is "usual"?
Two-dimensional arrays in Python using lists are quite rare. Anyone who
is doing serious numeric work where they need 2D arrays is using numpy,
not lists. There are millions of people using Python, so it's hardly
surprising that once or twice a year some newbie trips over this. But
it's not something that people tend to trip over again and again and
again, like C's "assignment is an expression" misfeature.
I read some of the documentation on why Python 3 chose to implement it
this way.
What documentation is this? Because this is a design decision that goes
all the way back to at least Python 1.5:
[steve@ando ~]$ python1.5
Python 1.5.2 (#1, Aug 27 2012, 09:09:18) [GCC 4.1.2 20080704 (Red Hat
4.1.2-52)] on linux2
Copyright 1991-1995 Stichting Mathematisch Centrum, Amsterdam
[[0, 99, 0, 0, 0], [0, 99, 0, 0, 0], [0, 99, 0, 0, 0]]x = [[0]*5]*3
x[0][1] = 99
x
So I expect the design decision for Python 3 was "we made the right
decision before, there's no need to change it".
(pardon me for belabouring the point here)
Q: Does [0]*10 make ten copies of the integer object? A: No, list
multiplication doesn't make copies of elements.
Neither would my idea for the vast majority of things on your first
list.
Um, yes? The point is that "vast majority" is not "everything". Hence,
your suggested behaviour is inconsistent.
Q: What about [[]]*10?
A: No, the elements are never copied.
YES! For the obvious reason that such a construction is making mutable
lists that the user wants to populate later. If they *didn't* want to
populate them later, they ought to have used tuples -- which take less
overhead. Who even does this thing you are suggesting?!
Who knows? Who cares? Nobody does:
n -= n
instead of just n=0, but that doesn't mean that we should give it some
sort of special meaning different from n -= m. If it turns out that the
definition of list multiplication is such that NOBODY, EVER, uses [[]]*n,
that is *still* not a good reason for special-casing it. All it means is
that this will be a less-obscure example of the billions of things which
can be done in Python but nobody wants to.
You have quoted from the Zen of Python a few times in this post. Perhaps
you missed one of the most critical ones?
Special cases aren't special enough to break the rules.
There are perfectly good ways to generate a 2D array out of lists, and
even better reasons not to use lists for that in the first place. (Numpy
arrays are much better suited for serious work.)
They aren't list multiplication compatible in any event! It's a total
nonsense objection.
I'm afraid you've just lost an awful lot of credibility there.
py> x = [{}]*5
py> x
[{}, {}, {}, {}, {}]
py> x[0]['key'] = 1
py> x
[{'key': 1}, {'key': 1}, {'key': 1}, {'key': 1}, {'key': 1}]
And similarly for any other mutable object.
If you don't understand that lists can contain other mutable objects
apart from lists, then you really shouldn't be discussing this issue.
RARE!!!! You are NUTS!!!!
Yes, rare. I base that on about 15 years of Python coding and many
thousands (tens of thousands?) of hours on Python forums like this one.
What's your opinion based on?
List multiplication is rare enough, but when it is used, it is usually
used to generate a 1D array like this:
values = [None]*n # or 0 is another popular starting value
Using it twice to generate a 2D array is even rarer.
Give an example usage of why someone would want to do this. Then we can
discuss it.
Proxying objects is hardly a rare scenario. Delegation is less common
since you can subclass built-ins, but it is still used. It is a standard
design pattern.
Please link to the objection being proposed to the developers, and their
reasoning for rejecting it.
I think you are exaggerating.
Python is a twenty year old language. Do you really think this is the
first time somebody has noticed it?
It's hard to search for discussions on the dev list, because the obvious
search terms bring up many false positives. But here are a couple of bug
reports closed as "won't fix":
http://bugs.python.org/issue1408
http://bugs.python.org/issue12597
I suspect it is long past time for a PEP so this can be rejected once and
for all.
No it isn't efficient. It's *slow* when done as in your example.List multiplication [x]*n is conceptually equivalent to: <snip>
This is nice and simple and efficient.
Well of course it is slow*er* when you move it from low-level C to high
level Python, but it is still fast.
I don't think so -- again, look at range(); it was made to work
inconsistent for a "common" case.
You mentioned range before, but it isn't clear to me what you think is
inconsistent about it.
Besides, 2D arrays are *not* rare and people *have* to copy internals of
them very often.
So you say.
The copy speed will be the same or *faster*, and the typing less -- and
the psychological mistakes *less*, the elegance more.
You think that it is *faster* to copy a list than to make a new pointer
to it? Your credibility is not looking too good here.
It's hardly going to confuse anyone to say that lists are copied with
list multiplication, but the elements are not.
Well, that confuses me. What about a list where the elements are lists?
Are they copied?
What about other mutable objects? Are they copied?
What about mutable objects which are uncopyable, like file objects?
Every time someone passes a list to a function, they *know* that the
list is passed by value -- and the elements are passed by reference.
And there goes the last of your credibility. *You* might "know" this, but
that doesn't make it so.
Python does not use either call-by-value or call-by-reference, and it
certainly doesn't use different calling conventions for different
arguments or parts of arguments. Everything is passed using the same
calling convention. Start here:
http://mail.python.org/pipermail/tutor/2010-December/080505.html
People in Python are USED to lists being "the" way to weird behavior
that other languages don't do.
Python's calling behaviour is identical to that used by languages
including Java (excluding unboxed primitives) and Ruby, to mention only
two.
You're starting to shout and yell, so perhaps it's best if I finish this
here.