list.clear() missing?!?

P

Peter Hansen

Fredrik said:
let's see. lists are introduced on page 19, a more extensive discussion of lists is
found on page 33, the del statement appears on page 34, and the dir() function
is introduced on page 46.

You're spending a lot of time trying to convince me I'm wrong, yet until
Georg did something about it just now, nowhere in those sections did
it talk about [:] specifically, which is sort of the whole point. It is
*not* obvious that although [x:] and [:y] and [x:y] take subsets of
things, that leaving them both out is either legal or useful.

Thankfully (to Georg), it now is. (Well, I haven't read it yet, but
I'll take his word for it.)

-Peter
 
F

Fredrik Lundh

Peter said:
You're spending a lot of time trying to convince me I'm wrong

no, I'm just posting observable facts in response to various "I'm too
lazy to look this up, but I'll assume that things are this way" posts.
Thankfully (to Georg)

it's fixed in the wiki too...

talking about the wiki, are you and rurpy the same person, btw? the
implied "I'm going to pretend that you've never done anything to im-
prove the situation" insinuations are getting a bit tiresome...

</F>
 
M

Mystilleef

I agree. Lists should have a clear method. But what's shocking is that
it doesn't seem obvious to others. list.clear() is a whole lot more
readable, intuitive, "flowable" and desirable than del list. Or maybe I
haven't had enough coffee this morning. I'd go as far as saying all
container objects should have a clear method. This is the reason why
those Rubyist whine about Python being inconsistent and not being "OO
enough."
 
R

Richie Hindle

[Mystilleef]
Lists should have a clear method. But what's shocking is that
it doesn't seem obvious to others. list.clear() is a whole lot more
readable, intuitive, "flowable" and desirable than [the alternatives]

+1 to all of that.
 
G

gry

A perspective that I haven't seen raised here is inheritance.
I often say
mylist = []
if I'm done with the current contents and just want a fresh list.

But the cases where I have really needed list.clear [and laboriously
looked for it and ended up with
del l[:]
were when the object was my own class that inherits from list, adding
some state and other functionality. Of course I *could* have added my
own 'clear' function member, but I *wanted* it to behave like a
standard
python list in it's role of maintaining a sequence of processing steps.
So, I end up doing
del current_process[:]
which, IMO, looks a bit queer, especially when current_process
object is a fairly elaborate data object.
 
S

Steven Bethard

Raymond said:
[Steven Bethard]
I think these are all good reasons for adding a clear method, but being
that it has been so hotly contended in the past, I don't think it will
get added without a PEP. Anyone out there willing to take out the best
examples from this thread and turn it into a PEP?

Something this small doesn't need a PEP. I'll just send a note to
Guido asking for a pronouncement.

Thanks. It'd be nice to have something to refer to the next time this
comes up.
Here's a draft list of pros and cons (any changes or suggestions are
welcome):

Pros:
-----

* s.clear() is more obvious in intent

* easier to figure-out, look-up, and remember than either s[:]=[] or
del s[:]

* parallels the api for dicts, sets, and deques (increasing the
expecation that lists will too)

* the existing alternatives are a bit perlish

I'd snip the one below. It doesn't really contribute anything, and was
sarcastic in the first place. ;)
* the OP is shocked, SHOCKED that python got by for 16 years without
list.clear()


Cons:
-----

* makes the api fatter (there are already two ways to do it)

* expanding the api makes it more difficult to write classes that can
be polymorphically substituted for lists

* learning slices is basic to the language (this lesson shouldn't be
skipped)

* while there are valid use cases for re-using lists, the technique is
already overused for unsuccessful attempts to micro-optimize (creating
new lists is surprisingly fast)

In fairness, if we're going to drop the "SHOCKED" comment, we should
drop the first two clauses of the point below (but the "relevant idiom"
part is clearly a good point).
* the request is inane, the underlying problem is trivial, and the
relevant idiom is fundamental (api expansions should be saved for rich
new functionality and not become cluttered with infrequently used
redundant entries)

Other than those minor edits, I think you've pretty much gotten
everything. I look forward to a pronouncement.

Thanks again!

STeVe
 
F

Fredrik Lundh

Mel said:
Both extend and append have one-line slice equivalents,
except that the equivalents have to keep referring to
the length of the list.. (have to keep finding the
len function.)

fwiw, the *tutorial* defines append and extend in terms of slicing...

</F>
 
R

Raymond Hettinger

* the request is inane, the underlying problem is trivial, and the
Is this sort of editorialising fair, or just a way of not-so-subtly
encouraging Guido to reject the whole idea, now and forever?

Bah. Guido is not stupid, nor easily misled. Both the pros and cons
were quipped with abrupt perjoratives so the bullet points could be
stated succinctly and with a bit of levity. The translation to
verbose, soft, politically correct statements is self-evident.

"request is inane" --> "A generation of python programmers has found
list clearing to be like other parts of the language that you get used
to very quickly and do not prove to be a problem in practice. The
request is in the same category as others which challenge api choices
made 16 years ago; in particular, the decision to have compact APIs
where the named methods do not duplicate functionality provided by
syntax using operators and keywords. The request is less of a bug
report and more a rejection of Guido's sense of design and his
subsequent experience using his own language."

"underlying problem is trivial" --> "Books such as the Python Pocket
Reference or Python in a Nutshell are able to cover this idiom with
just a single sentence. Once known and understood, the application of
the current-way-to-do-it is immediate, compact, and effective."

"the existing alternatives are a bit perlish" --> "Both alternatives
involve a bit of inventiveness in combining two ideas (either the del
keyword and its interaction with slicing notation or the assignment of
an emtpy list to a slice). Each approach has a visual appearance of
being a syntax trick. The effect contrasts with much of the rest of
the language where it is usually possible to write code is a way that
can be read and understood by non-python programmers. The existing
solution trades readability for the succinctness of a compact
syntactical idiom."


A list.clear method will make deleting items from a list more OO,
consistent with almost everything else you do to lists, and less
procedural. This is especially true if clear() takes an optional index (or
two), allowing sections of the list to be cleared, not just the entire
list.

Don't shoot yourself in the foot here. If you want to advocate
list.clear(), then you're hurting your chances by pushing for it to
take an optional argument. Essentially, this amounts to an
unwillingness to use the del-keyword and to duplicate its functionality
with a named method.

Also, in the python-dev world, making something "more OO" is neither a
virtue nor a vice. It is better to argue for rich functionality,
succeptibility to errors, or dramatic improvements of existing
real-world code.
 
F

Fredrik Lundh

Raymond said:
Also, in the python-dev world, making something "more OO" is neither a
virtue nor a vice.

except that arguments along the line of "if the syntax is not obj.method(),
it's not OO enough" are likely to be mostly ignored.

(nobody's going to be impressed by yet another "len(obj) isn't OO" variant)

</F>
 
S

Sion Arrowsmith

Fredrik Lundh said:
except that arguments along the line of "if the syntax is not obj.method(),
it's not OO enough" are likely to be mostly ignored.

(nobody's going to be impressed by yet another "len(obj) isn't OO" variant)

Does that suggest that what's needed is clear(obj) and __clear__
methods? :cool:
 
R

Raymond Hettinger

[Dan Christensen]
It's true that this runs at the same speed as the del variants on my
machine. That's not too surprising to me, but I still don't
understand why the del variants are more than 5% faster than the first
version.

Understanding it involves looking at implementation specific details
such as the overallocation scheme for growing lists and the performance
of your machine's underlying memory allocator.

Once this is understood, is it something that could be optimized?
It's pretty common to rebind a variable to a new value, and if
this could be improved 5%, that would be cool.

Sorry, that's not how the language works. Something like
"a=range(100000)" means:
* build a NEW list for 100000 elements
* THEN assign it to the variable "a"
* which THEN reduces the ref count to the previous binding for "a"
* and THEN IF the ref count is zero, free the list previously bound to
"a"

In other words, if "a" is already bound to a large list, then the above
assignment necessarily creates a second, non-overlapping list in
memory.

However, if you write "a=None;a=range(100000)", then the original list
gets freed BEFORE the new list is created and the system has a chance
to re-use that large, contiguous block of memory.
 
S

Sergei Organov

Peter Hansen said:
Then it's a good reason we had this thread, so you could learn something
*crucial* to understanding Python and writing non-buggy code: name
binding versus variables which occupy fixed memory locations like in
some other languages. This has to be by far the most frequent area that
newcomer's trip up. But that's another story...

I, as a newcomer, don't have much trouble understanding the binding vs
the assignment by themselves. What does somewhat confuse is dual role of
the "=" operator, -- sometimes it means "bind" and other times it means
"assign", right? For me it seems that the language would be less
error-prone and easier to grok if these two operations had different
names/syntax (thinking about lisp "let" vs "set"), though it seems to be
too drastic change even for Python3000.

-- Sergei.
 
F

Felipe Almeida Lessa

Em Sex, 2006-04-14 às 09:17 +0400, Sergei Organov escreveu:
I, as a newcomer, don't have much trouble understanding the binding vs
the assignment by themselves. What does somewhat confuse is dual role of
the "=" operator, -- sometimes it means "bind" and other times it means
"assign", right? For me it seems that the language would be less
error-prone and easier to grok if these two operations had different
names/syntax (thinking about lisp "let" vs "set"), though it seems to be
too drastic change even for Python3000.

The "=" operator *always* binds.
 
D

Dennis Lee Bieber

I, as a newcomer, don't have much trouble understanding the binding vs
the assignment by themselves. What does somewhat confuse is dual role of
the "=" operator, -- sometimes it means "bind" and other times it means
"assign", right? For me it seems that the language would be less

It always means bind... But if the LHS is a mutable object, AND you
have specified a component of that object, it is the component that is
being rebound...

lst[:] = []

is rebinding the elements inside the list "lst", and not rebinding the
name "lst". Essentially, once you add any "selector" to the name
(object[...]= or object.xxx=) you are going "inside" the object, and
manipulating (rebinding) what is inside. If the name is used "pure"
(object=), you are rebinding the /name/ to a different object.

--
 
S

Sergei Organov

Dennis Lee Bieber said:
I, as a newcomer, don't have much trouble understanding the binding vs
the assignment by themselves. What does somewhat confuse is dual role of
the "=" operator, -- sometimes it means "bind" and other times it means
"assign", right? For me it seems that the language would be less

It always means bind... But if the LHS is a mutable object, AND you
have specified a component of that object, it is the component that is
being rebound...

lst[:] = []

is rebinding the elements inside the list "lst", and not rebinding the
name "lst". Essentially, once you add any "selector" to the name
(object[...]= or object.xxx=) you are going "inside" the object, and
manipulating (rebinding) what is inside. If the name is used "pure"
(object=), you are rebinding the /name/ to a different object.

Me gets corrected, thanks. Now I need to unroll my mind somewhat back to
figure out when and why I started to believe it sometimes assigns ;)

-- Sergei.
 
B

Ben C

Dennis Lee Bieber said:
It always means bind... But if the LHS is a mutable object, AND you
have specified a component of that object, it is the component that is
being rebound...

lst[:] = []
[...]
Me gets corrected, thanks. Now I need to unroll my mind somewhat back to
figure out when and why I started to believe it sometimes assigns ;)

I used to think it assigned with things like integers, because if you
write:

a = 5
b = a
b += 1
print a

a is still 5. So it looked like a and b stored values and b got a "copy"
of a's value. But this is the wrong interpretation,

b += 1

is really b = b + 1, and rebinds b.

You can see what's really going on if you use the id() function on a and
b during these operations.

The other reason for the confusion is that I think in Java a variable
either stores a value, in the case of numbers, or a reference in the
case of objects (or a copy-on-write reference, which behaves like a
value, in the case of strings). In Python it's better to think of it as
always a reference, and to think in terms of immutable vs. mutable
objects that are referred to.

If it weren't for the id() function I think the difference between
"variable stores value", "variable stores immutable reference" and
"variable stores copy-on-write reference" would be implementation detail
and never visible to the programmer. That's why it's easy to be
"confused"-- most of the time these interpretations are equivalent, so
it doesn't matter which you work with.
 
S

Steven D'Aprano

Raymond, I suspect we're not seeing eye to eye on this issue, but I do
appreciate you taking the time and effort. Thank you. My comments follow.

On Thu, 13 Apr 2006
Both the pros and cons
were quipped with abrupt perjoratives so the bullet points could be
stated succinctly and with a bit of levity.

Humour so often doesn't come across in text -- I didn't see the levity you
intended.

"request is inane" --> "A generation of python programmers has found
list clearing to be like other parts of the language that you get used
to very quickly and do not prove to be a problem in practice. The
request is in the same category as others which challenge api choices
made 16 years ago; in particular, the decision to have compact APIs
where the named methods do not duplicate functionality provided by
syntax using operators and keywords. The request is less of a bug
report and more a rejection of Guido's sense of design and his
subsequent experience using his own language."

Of course it isn't a bug report. It's a feature request.

As for Guido's sense of design, in *this particular instance* I think he
got it wrong. That's hardly a rejection of the man's overall design skills.

In any case, as the creator of Python, Guido has never seen the language
with the eyes of a Python beginner. All the more credit to him for
creating a language which is simultaneously friendly to beginners and
powerful for experts. But he isn't superhuman, his experience is not the
only experience. (True, as BDFL, his experience has the final vote, but
that's a whole different kettle of fish.)

A bit of searching on Google finds this issue coming up frequently. It
seems to me that there is a steady stream of Python programmers asking
"where's the list clear method?". Perhaps it isn't a monthly occurrence,
but it is still very common. That surely suggests that, as powerful
as slicing is, people aren't getting the connection between slicing and
emptying a list, and even when they do get it, many dislike it.

Since there is no evidence that these people are universally stupid, it
suggests strongly that the connection is too subtle. Some may protest that
the idiom is in the tutorial, that it is obvious once you know slicing and
del, but the fact is that many people aren't finding it obvious or
self-evident at all -- not even those who have read the tutorial.

Here is another possible solution: have the list.clear method be defined
this way:

def clear(self):
print "Haven't you read the tutorial? Use del self[:] or self[:] = []"

That at least will stop the procession of people asking how to clear a
list, and Fredrik won't have to spend his time telling them to read the
tutorial.

(Hey, if it's good enough for quit and exit... *wink*)

Don't shoot yourself in the foot here. If you want to advocate
list.clear(), then you're hurting your chances by pushing for it to
take an optional argument. Essentially, this amounts to an
unwillingness to use the del-keyword and to duplicate its functionality
with a named method.

Deleting names from namespaces is conceptually distinct from deleting
components of compound objects like lists, and should be handled
differently: objects know their own contents, and can operate on
themselves, but they don't know what namespace they live in or what
name(s) they are known by. Deleting items from a list should be a case of
"object, operate on yourself", just like remove, append and so on, and
hence should be specified by an object method, not a keyword like del.

So you are right: I am uncomfortable having del do double duty to both
delete names and operate on the internals of compound objects. It feels
wrong to me. If insert were a keyword (as in "insert list[i:j], obj") it
would feel wrong to me too, and for the same reason.

I haven't been using Python for sixteen years, but at six years and
counting I'm not exactly a newbie. At first I accepted the del obj[:]
idiom implicitly -- the whole language was new to me, and it was no more
strange to me than any other Python syntax or idiom. But as I've got more
familiar with Python, the sense of two distinct concepts being
artificially forced into the same syntax has just gotten stronger.

As a pragmatist, I'd be happy to see a bare list.clear() method; that
would at least have the advantages of being easy to find with dir(list)
and accessible with help(). But as a purist, I really think it needs
optional arguments. Make of that what you will.
 
V

Ville Vainio

Duncan said:
Looking in the 'obvious' place in the Tutorial, section 5.1 'More on
Lists' I found in the immediately following section 5.2 'The del
statement':

I read the tutorial 6 years ago, and don't read it regularly. What's in
the tutorial is not really important, what can be easily looked up in
library reference or interctive prompt is.

There is a bit of "elitism" regarding the defense of del lst[:]. Many
python programmers are "casual" programmers (was it Tim Berners-Lee
that explained how great python is for casual users?) who know the
language but don't remember the way everything is done. Of course
someone that uses python 8 hours a day instantly recall lst[:], but a
casual one will more probably launch an ipython prompt and do:

[ipython]|1> l = []
[ipython]|2> l.
l.__add__ l.__getslice__ l.__ne__ l.append
l.__class__ l.__gt__ l.__new__ l.count
l.__contains__ l.__hash__ l.__reduce__ l.extend
l.__delattr__ l.__iadd__ l.__reduce_ex__ l.index
l.__delitem__ l.__imul__ l.__repr__ l.insert
l.__delslice__ l.__init__ l.__reversed__ l.pop
l.__doc__ l.__iter__ l.__rmul__ l.remove
l.__eq__ l.__le__ l.__setattr__ l.reverse
l.__ge__ l.__len__ l.__setitem__ l.sort
l.__getattribute__ l.__lt__ l.__setslice__
l.__getitem__ l.__mul__ l.__str__
[ipython]|2> # wtf?
 
F

Fredrik Lundh

Ben C said:
I used to think it assigned with things like integers, because if you
write:

a = 5
b = a
b += 1
print a

a is still 5. So it looked like a and b stored values and b got a "copy"
of a's value. But this is the wrong interpretation,

b += 1

is really b = b + 1, and rebinds b.

almost: "b += obj" is really "b = b.__iadd__(obj)" with "b = b + obj"
as a fallback if __iadd__ isn't supported by "b".
If it weren't for the id() function I think the difference between
"variable stores value", "variable stores immutable reference" and
"variable stores copy-on-write reference" would be implementation
detail and never visible to the programmer.

except when they are:
>>> a = [1, 2, 3]
>>> b = a
>>> b += [4]
>>> a [1, 2, 3, 4]
>>> b
[1, 2, 3, 4]

</F>
 

Ask a Question

Want to reply to this thread or ask your own question?

You'll need to choose a username for the site, which only take a couple of moments. After that, you can post your question and our members will help you out.

Ask a Question

Members online

No members online now.

Forum statistics

Threads
474,293
Messages
2,571,501
Members
48,189
Latest member
StaciLgf76

Latest Threads

Top