Proposal: [... for ... while cond(x)]

[Eighty]

I suggest a new extension of the list comprehension syntax:

[x for x in xs while cond(x)]

which would be equivalent to

list(itertools.takewhile(cond, xs))

+ Since Python favors list comprehensions over map, filter, and reduce,
this would be the preferred way to do this
+ "Takewhile operations" occur often, at least for me
+ I don't think it would break any existing syntax

An analogous syntax for dropwhile would be nice, but I can't think of
one.

This is not a PEP because it's a very simple idea and probably not just
anyone (read: me) can write and submit one. If there has been a PEP for
this, I've missed it; if not, it would be nice if someone wrote one.

Discuss.

 

[Duncan Booth]

I suggest a new extension of the list comprehension syntax:

[x for x in xs while cond(x)]

which would be equivalent to

list(itertools.takewhile(cond, xs))

What would this syntax offer that:

[x for x in takewhile(cond, xs)]

doesn't currently offer? (Apart, that is, from saving you 3 characters of
typing)

 

[Eighty]

I suggest a new extension of the list comprehension syntax:

[x for x in xs while cond(x)]

which would be equivalent to

list(itertools.takewhile(cond, xs))

What would this syntax offer that:

[x for x in takewhile(cond, xs)]

doesn't currently offer? (Apart, that is, from saving you 3 characters of
typing)

The same thing that [f(x) for x in xs] offers that map(f, xs) doesn't,
and the same thing that [x for x in xs if f(x)] offers that filter(f,
xs) doesn't. It's more "pythonic". You can use an expression for cond
instead of a lambda.

 

[Duncan Booth]

I suggest a new extension of the list comprehension syntax:

[x for x in xs while cond(x)]

which would be equivalent to

list(itertools.takewhile(cond, xs))

What would this syntax offer that:

[x for x in takewhile(cond, xs)]

doesn't currently offer? (Apart, that is, from saving you 3
characters of typing)

The same thing that [f(x) for x in xs] offers that map(f, xs) doesn't,
and the same thing that [x for x in xs if f(x)] offers that filter(f,
xs) doesn't. It's more "pythonic". You can use an expression for cond
instead of a lambda.

No, the list comprehension lets you write an expression directly avoiding a
function call, and it also allows you to add in a condition which can be
used to filer the sequence. Your proposal adds nothing.

 

[Slawomir Nowaczyk]

On Sun, 06 Aug 2006 18:59:39 +0000 (GMT)

#> >> > I suggest a new extension of the list comprehension syntax:
#> >> >
#> >> > [x for x in xs while cond(x)]
#> >> >
#> >> > which would be equivalent to
#> >> >
#> >> > list(itertools.takewhile(cond, xs))
#> >>
#> >> What would this syntax offer that:
#> >>
#> >> [x for x in takewhile(cond, xs)]
#> >>
#> >> doesn't currently offer?
#> >
#> > The same thing that [f(x) for x in xs] offers that map(f, xs) doesn't,
#> > and the same thing that [x for x in xs if f(x)] offers that filter(f,
#> > xs) doesn't. It's more "pythonic". You can use an expression for cond
#> > instead of a lambda.
#> >
#> No, the list comprehension lets you write an expression directly
#> avoiding a function call, and it also allows you to add in a
#> condition which can be used to filer the sequence.

I am not sure if I understand you correctly, but... Does it?

a = [0,1,2,3,7,8,9]
[x for x in takewhile(lambda x: x in a, range(10))] [0, 1, 2, 3]
[x for x in takewhile(x in a, range(10))]

Traceback (most recent call last):
File "<stdin>", line 1, in ?
TypeError: 'bool' object is not callable

Did I miss something? Notice that using "if" gives different result:

[x for x in range(10) if x in a]

[0, 1, 2, 3, 7, 8, 9]

#> Your proposal adds nothing.

Well, I am not sure how useful the proposal really is, but it seems to
add *something* if it would allow for things like:
[x for x in range(10) while x in a]

--
Best wishes,
Slawomir Nowaczyk
( (e-mail address removed) )

Women who seek to be equal to men lack ambition.

 

[Diez B. Roggisch]

No, the list comprehension lets you write an expression directly avoiding a

function call, and it also allows you to add in a condition which can be
used to filer the sequence. Your proposal adds nothing.

It does. Consider this:

whatever = [x for x in xrange(1000000000) while x < 10]


That would run only in a splitsecond of what the whole listcomp would.


Yet I still fail to see that this is a really useful use-case. takewhile
is easily abstracted away as map and reduce and filter are using lambda
- which I'm a modest supporter of.

But functions that are monotonic such that the takewhile really offers a
advantage over the much more general listcomp + if are IMHO way to
seldom to justify a new syntax.

The overall complexity is still O(n).

Regards,

Diez

 

[Duncan Booth]

No, the list comprehension lets you write an expression directly
avoiding a function call, and it also allows you to add in a
condition which can be used to filer the sequence. Your proposal adds
nothing.

It does. Consider this:

whatever = [x for x in xrange(1000000000) while x < 10]


That would run only in a splitsecond of what the whole listcomp would.

Except that the comparable listcomp today is:

whatever = [x for x in takewhile(lambda x: x < 10, xrange(1000000000))]

which also runs in a split second.

Actually, the OP was correct, it does add something: it removes the need
for a function or lambda in the takewhile just as the original listcomp
removes a function or lambda compared with the map version.

 

[Duncan Booth]

#> No, the list comprehension lets you write an expression directly
#> avoiding a function call, and it also allows you to add in a
#> condition which can be used to filer the sequence.

I am not sure if I understand you correctly, but... Does it?

a = [0,1,2,3,7,8,9]
[x for x in takewhile(lambda x: x in a, range(10))] [0, 1, 2, 3]
[x for x in takewhile(x in a, range(10))]

Traceback (most recent call last):
File "<stdin>", line 1, in ?
TypeError: 'bool' object is not callable

Did I miss something?

Yes, you missed out a lambda (so I was wrong, your suggestion would
actually gain you more than 3 characters of typing)

Try:

a = [0,1,2,3,7,8,9]
[x for x in takewhile(lambda x:x in a, range(10))]

[0, 1, 2, 3]

For this particular expression you could also write:

[x for x in takewhile(a.__contains__, range(10))]

[0, 1, 2, 3]

or with Python 2.5 we can avoid referencing __contains__ with the following
variant:

from itertools import takewhile
from functools import partial
from operator import contains
a = [0,1,2,3,7,8,9]
[x for x in takewhile(partial(contains,a), range(10))] [0, 1, 2, 3]

 

[Rick Zantow]

No, the list comprehension lets you write an expression directly
avoiding a function call, and it also allows you to add in a
condition which can be used to filer the sequence. Your proposal adds
nothing.

It does. Consider this:

whatever = [x for x in xrange(1000000000) while x < 10]


That would run only in a splitsecond of what the whole listcomp

would.

Except that the comparable listcomp today is:

whatever = [x for x in takewhile(lambda x: x < 10, xrange (1000000000))]

which also runs in a split second.

Actually, the OP was correct, it does add something: it removes the need
for a function or lambda in the takewhile just as the original listcomp
removes a function or lambda compared with the map version.

Consider how it would be if the situation were reversed, and
whatever = [x for x in xrange(1000000000) while x < 10] was the
convention today. What advantage would there be to replacing it with
whatever = [x for x in takewhile(lambda x: x < 10, xrange(1000000000))]?

As a newcomer to Python, I'd find the first syntax far more readily
graspable, and I'd have to wonder why I'd ever need takewhile and lambda
just to do what appears to be straightforward conditioning of a loop.

I'm not a newcomer to Python, and I wonder about that anyway.

I also note this, using Python 2.4.2 on win32:

whatever = [x for x in takewhile(lambda x: x < 10, xrange

(1000000000))]
Traceback (most recent call last):
File "<stdin>", line 1, in ?
NameError: name 'takewhile' is not defined

So in addition to the two functions, I need an import statement. It
looks like the argument can certainly be made that simplifying the
syntax and lightening the call load bring some advantage to the table.
There are other arguments to be made against the proposed syntax, I'm
sure.

 

[Eighty]

I suggest a new extension of the list comprehension syntax:

[x for x in xs while cond(x)]

which would be equivalent to

list(itertools.takewhile(cond, xs))

+ Since Python favors list comprehensions over map, filter, and reduce,
this would be the preferred way to do this
+ "Takewhile operations" occur often, at least for me
+ I don't think it would break any existing syntax

An analogous syntax for dropwhile would be nice, but I can't think of
one.

This is not a PEP because it's a very simple idea and probably not just
anyone (read: me) can write and submit one. If there has been a PEP for
this, I've missed it; if not, it would be nice if someone wrote one.

Discuss.

So does no one have a comment on this? The one objection I can come up
with is that this would change the set builder notation semantics too
much, but since the iteration order in a list comprehension is already
well defined, I don't think that would be the case. However, for this
reason, it wouldn't fit in a dict comprehension, if that would ever be
made, perhaps making the syntax inconsistent?

 

[Terry Reedy]

I suggest a new extension of the list comprehension syntax:

[x for x in xs while cond(x)]

This does not work.

e(x) for x in xs if cond(x)

is an abbreviation of

for x in xs:
if cond(x)
yield e(x)

and similarly with more for and if clauses,
whereas the analogous expansion of your proposal

for x in xs:
while cond(x):
yield e(x)

is an infinite loop and not at all what you mean.

And what would

x for x in xs while cond(x) if blah(x)
x for x in xs if blah(x) while cond(x)
x*y for x in xs while cond(x) for y in ys

mean? The ability to mix and match clauses after the first for clause is
an important part of comprehension syntax.

So keep using the function provided. I am pretty sure takewhile is rare
for most other people.

So does no one have a comment on this?

Ain't gonna happen.

The one objection I can come up with
is that this would change the set builder notation semantics too much

Yes, to the point where you are hijacking the syntax, for no useful gain,
more than extending it ;-).

Terry Jan Reedy

 

[Eighty]

whereas the analogous expansion of your proposal

for x in xs:
while cond(x):
yield e(x)

is an infinite loop and not at all what you mean.

You're right. The syntax is ambiguous. I agree it's not a good idea,
now.

x for x in xs while cond(x) if blah(x)
x for x in xs if blah(x) while cond(x)
x*y for x in xs while cond(x) for y in ys

These wouldn't be a problem.
"... for x in xs while cond(x) ..." would be transformed into "... for
x in takewhile(cond, xs) ..."
which could be applied to an if thingy if you first transform
"... for x in xs if cond(x) ..." into "... for x in filter(cond, xs)
....".

 

[Neil Hodgson]

Eighty:

So does no one have a comment on this?

Similar proposals have appeared before (such as on python-dev about
a year ago) and haven't attracted much positive comment. The benefits
appear to be small compared to the cost of making the language larger.
You could try the formal route of writing a PEP so that a detailed
proposal and its reasons for (probable) rejection would be available for
others.

Neil

 

[Nick Mellor]

Hi all,

It might be a risk that we create some hairy or subtle semantics, but perhaps this idea has legs. The 'pythonic' syntax is very attractive.

# skip the silence at the start
data = (amp for amp in signal from abs(amp) > 0.05)

# drop files that are less than 600 bytes
# (already sorted into ascending order of size)
files = (file for file in filelist from os.stat(file).st_size > 600)

# Analyse bunch of log lines up to a certain timestamp
cutoff = datetime.datetime.now() - datetime.timedelta(hours=1)
checklogs = (entry for entry in log while entry.timestamp <= cutoff)

In use it feels along the same lines as else: clauses in try/finally and for loops, both of which I like-- it seems natural in Python, keeps the meaning clear and consistent, avoids complicated, iterative, inexpressive code.

You're right. The syntax is ambiguous. I agree it's not a good idea,

now.

When while, if and from are made mutually exclusive, not mixed, on a comprehension clause, it looks to me like the ambiguity disappears:

x for x in xs while cond(x) if blah(x)

not legal Python-- if and while are mutually exclusive on the same comprehension clause. On the one hand it's like writing x for x in xs if f(x) < 10 if g(x) > 100, the illegality of which no-one is complaining about. Just write:

x for x in xs while cond(x) and cond2(x)

or

x for x in xs if cond(x) and cond2(x)

If there is a need to filter using if as well as while, then use two comprehensions:

intermediate = [x for x in xs while cond(x)]
final = [x for x in intermediate if cond2(x)]

or the same comprehension applied to itself:

y for y in x for x in xs if cond(x) while cond2(y)

The if and the while operation *should* be kept separate-- they're different stages of processing the data.

legal Python-- the 'if cond(y)' unambiguously refers to the inner comprehension.

x for x in xs if blah(x) while cond(x)

not legal Python-- if and while are mutually exclusive in the same comprehension clause.

The objection to "while" as an abbreviation:

e(x) for x in xs if cond(x)

is an abbreviation of

for x in xs:
if cond(x)
yield e(x)

and similarly with more for and if clauses,
whereas the analogous expansion of your proposal

for x in xs:
while cond(x):
yield e(x)

is an infinite loop and not at all what you mean.

That's because while is being used here with a different semantic to the while loop. It actually means:

for x in xs:
if not cond(x):
break
yield e(x)

and for 'from':


emit = False
for x in xs:
if not emit:
if cond(x):
emit = True
else:
yield e(x)


Cheers,

Nick

 

[Nick Mellor]

On Friday, 27 September 2013 00:39:30 UTC+10, Nick Mellor wrote:

[snip]

for x in xs:
if not emit:
if cond(x):
emit = True
else:
yield e(x)

Oops!

for x in xs:
if not emit:
if cond(x):
emit = True
yield e(x)
else:
yield e(x)