Filter versus comprehension (was Re: something about split()???)

[Terry Reedy]

for each item in the iterable.

A case of premature optimisation?

No, as regards my post. I simply made a factual statement without
advocating a particular action.

filter(lambda x: <expr>, iterable)
(x for x in iterable if <expr>)

both create iterators that produce the items in iterable such that

"ranger=range(0)")
0.83

"ranger=range(20)")
2.60

Simply keeping true items is faster with filter -- at least on my
particular machine with 3.3.0b2.
1.03

Filter is also faster if the expression is a function call.
f=lambda i: False")
2.5033614114454394f=lambda i: False")
3.2394095327040304

---
Perhaps or even yes as regards the so-called rule 'always use
comprehension'. If one prefers filter as more readable, if one only
wants to keep true items, if the expression is a function call, if
evaluating the expression takes much more time than the extra function
call so the latter does not matter, if the number of items is few enough
that the extra time does not matter, then the rule is not needed or even
wrong.

So I think PyLint should be changed to stop its filter fud.

 

[Ramchandra Apte]

for each item in the iterable.






No, as regards my post. I simply made a factual statement without

advocating a particular action.



filter(lambda x: <expr>, iterable)

(x for x in iterable if <expr>)



both create iterators that produce the items in iterable such that

bool(<expr>) is true. The following, with output rounded, shows

something of the effect of the extra function call.




"ranger=range(20)")

2.60



Simply keeping true items is faster with filter -- at least on my

particular machine with 3.3.0b2.




1.03



Filter is also faster if the expression is a function call.




f=lambda i: False")

2.5033614114454394


f=lambda i: False")

3.2394095327040304



---

Perhaps or even yes as regards the so-called rule 'always use

comprehension'. If one prefers filter as more readable, if one only

wants to keep true items, if the expression is a function call, if

evaluating the expression takes much more time than the extra function

call so the latter does not matter, if the number of items is few enough

that the extra time does not matter, then the rule is not needed or even

wrong.



So I think PyLint should be changed to stop its filter fud.

When filtering for true values, filter(None,xxx) can be used
Your examples with lambda i:False are unrealistic - you are comparing `if False` vs <lambda function>(xx) - function call vs boolean check

 

[Ramchandra Apte]

for each item in the iterable.






No, as regards my post. I simply made a factual statement without

advocating a particular action.



filter(lambda x: <expr>, iterable)

(x for x in iterable if <expr>)



both create iterators that produce the items in iterable such that

bool(<expr>) is true. The following, with output rounded, shows

something of the effect of the extra function call.




"ranger=range(20)")

2.60



Simply keeping true items is faster with filter -- at least on my

particular machine with 3.3.0b2.




1.03



Filter is also faster if the expression is a function call.




f=lambda i: False")

2.5033614114454394


f=lambda i: False")

3.2394095327040304



---

Perhaps or even yes as regards the so-called rule 'always use

comprehension'. If one prefers filter as more readable, if one only

wants to keep true items, if the expression is a function call, if

evaluating the expression takes much more time than the extra function

call so the latter does not matter, if the number of items is few enough

that the extra time does not matter, then the rule is not needed or even

wrong.



So I think PyLint should be changed to stop its filter fud.

When filtering for true values, filter(None,xxx) can be used
Your examples with lambda i:False are unrealistic - you are comparing `if False` vs <lambda function>(xx) - function call vs boolean check

 

[Terry Reedy]

Your mail agent in inserting blank lines in quotes -- google?
See if you can turn that off.

Your examples with lambda i:False are unrealistic - you are comparing
`if False` vs <lambda function>(xx) - function call vs boolean check

That is exactly the comparison I wanted to make. The iteration + boolean
check takes .37 for 20 items, the iteration + call 1.77.

 

[Dennis Lee Bieber]

Your mail agent in inserting blank lines in quotes -- google?
See if you can turn that off.

It appears to be a change Google made in the last month or two... My
hypothesis is that they are replacing hard EOL found in inbound NNTP
with an HTML <p>, and then on outgoing replacing the <p> with a pair of
NNTP line endings. In contrast, text composed on Google is coming in as
long single lines (since quoting said text in a response produces on a
">" at the start of the paragraph.

 

[Walter Hurry]

It appears to be a change Google made in the last month or two... My
hypothesis is that they are replacing hard EOL found in inbound NNTP
with an HTML <p>, and then on outgoing replacing the <p> with a pair of
NNTP line endings. In contrast, text composed on Google is coming in as
long single lines (since quoting said text in a response produces on a
">" at the start of the paragraph.

Google Groups sucks. These are computer literate people here. Why don't
they just use a proper newsreader?

 

[Dennis Lee Bieber]

Google Groups sucks. These are computer literate people here. Why don't
they just use a proper newsreader?

Probably because their ISP doesn't offer a free server <G>

 

[Terry Reedy]

Probably because their ISP doesn't offer a free server <G>

Python lists are available on the free gmane mail-to-news server.

 

[Emile van Sebille]

Probably because their ISP doesn't offer a free server <G>

Python lists are available on the free gmane mail-to-news server.[/QUOTE]

I'm getting high load related denials with the gmane connections a lot
recently so I'm open to alternatives.

Suggestions or recommendations?


Emile

 

[Mark Lawrence]

Python lists are available on the free gmane mail-to-news server.

I don't think the core-mentorship list is available on gmane. Have I
missed it, has nobody asked for it to go on there or what?

 

[Ned Deily]

On 8/24/2012 3:03 PM Terry Reedy said...
I'm getting high load related denials with the gmane connections a lot
recently so I'm open to alternatives.

The high load denials should be a thing of the past as the gmane NNTP
server was very recently upgraded to use SSDs instead of standard disks.

 

[Ned Deily]

I don't think the core-mentorship list is available on gmane. Have I
missed it, has nobody asked for it to go on there or what?

core-mentorship is a closed list so it would not be appropriate for it
to be mirrored anywhere.

http://mail.python.org/mailman/listinfo/core-mentorship

 

[Walter Hurry]

Probably because their ISP doesn't offer a free server <G>

There are plenty of free Usenet providers.

 

[David Robinow]

Google Groups sucks. These are computer literate people here. Why don't
they just use a proper newsreader?

I haven't used a newsreader in over a decade. I'm quite happy with a
mailing list. Am I missing something?

 

[Tim Golden]

I haven't used a newsreader in over a decade. I'm quite happy with a
mailing list. Am I missing something?

Not really. I'm the same; it just means you can skip over the occasional
ggroups-newsreader discussion threads which pop up
about 3 times a year on average.



TJG