map/filter/reduce/lambda opinions and background unscientificmini-survey

[Ron Adam]

How about just getting rid of del? Removal from collections could be
done with a method call, and i'm not convinced that deleting variables
is something we really need to be able to do (most other languages
manage without it).

Since this is a Python 3k item... What would be the consequence of
making None the default value of an undefined name? And then assigning
a name to None as a way to delete it?

Some benefits
=============

*No more NamesError exceptions!

print value
value = 25
print value
value = None #same as 'del value'


*No initialization needed for a while loop!

While not something:
if <condition>:
something = True


*Test if name exists without using a try-except!

if something == None:
something = value


*And of course one less keyword!


Any drawbacks?


Cheers,
Ron

PS... not much sleep last night, so this may not be well thought out.

 

[Dan Sommers]

Since this is a Python 3k item... What would be the consequence of
making None the default value of an undefined name? And then assigning
a name to None as a way to delete it?

[ ... ]

Any drawbacks?

Lots more hard-to-find errors from code like this:

filehandle = open( 'somefile' )
do_something_with_an_open_file( file_handle )
filehandle.close( )

Regards,
Dan

 

[Steven Bethard]

And a syntax just occured to me -- what about this:

[y*x for x,y]

?

(that is:

[<expression> for <argument list>]

If you haven't already, see:

http://wiki.python.org/moin/AlternateLambdaSyntax

for other similar proposals.

STeVe

 

[Steven Bethard]

Yes, I think a different key word would help. My current favorite
alternative is to put it in parentheses similar to list comprehensions
and use "let".

(let x,y return x+y)

If you haven't already, see:

http://wiki.python.org/moin/AlternateLambdaSyntax

for other similar proposals.

STeVe

 

[Duncan Booth]

Arguing the case for del: how would I, in doing automated testing,
ensure that I've returned everything to a "clean" starting point in all
cases if I can't delete variables? Sometimes a global is the simplest
way to do something... how do I delete a global if not with "del"?

I generally find that unit tests force me to structure the code in a
cleaner manner, e.g. to not use globals as much, but if you do need to
delete a global you do it in exactly the same way as you delete anything:
use the "del" statement:
global x
del x


Traceback (most recent call last):
File "<pyshell#7>", line 1, in -toplevel-
x
NameError: name 'x' is not defined
Where I have used 'del' in a unit test it has been to delete local
variables rather than globals. Specifically I wanted to ensure that some
data structures were being torn down properly, so the test went something
like this:

setup: creates a weakref dictionary.

teardown: asserts that the weakref dictionary is empty.

then each test does:

try:
create something
add it to the weakref dictionary
then test it
finally:
use del to remove local variables
force a garbage collection

Without the del, when a test fails you get two failures, because the
traceback information keeps the variables alive.

 

[Steven D'Aprano]

How about just getting rid of del? Removal from collections could be done
with a method call,

Which would be called object.del() I presume. And that opens a big
can of worms.

Suppose we have a list L = [4, 3, 2, 1, 0], what should L.del(1) do?

It looks like it should result in L becoming [4, 3, 2, 0]. An easy mistake
to make, if you forget that the argument is (presumably) an index.
You could make it clear by insisting on L.del[1] but that requires a big
change in Python's syntax.

What should L.del() do, with no arguments? Raise an error?

Now, you have something like this:

class thing:
pass
obj = thing()
obj.alpha = [4, 3, 2, 1, 0]
obj.beta = 5

Python's object model suggests that obj.alpha.del() should call
alpha's del method, in the same way that obj.alpha.append() would call
alpha's append method.

So how do you delete obj.alpha? obj.del("alpha") might work. But what if
obj itself is a mapping, with a key "alpha" as well as an attribute alpha.
Which one should obj.del("alpha") delete?

Now, if you said that L.del() should raise an exception earlier, what
about obj.beta.del()?


Presumably every object automatically has a del method, so you
don't have to program a del method yourself. obj.del is a method object.
So it has a del method. (Yes, sometimes you want to delete methods.
Functions are first class objects in Python.) Which has a del method.
Which has a del method.

What should obj.del.del.del.del.del.del.del.del.del() do?

and i'm not convinced that deleting variables is
something we really need to be able to do (most other languages manage
without it).

Most other languages don't have namespaces that can get polluted, or
on-the-fly creation of variables. Most other languages don't consider
variables to be simply attributes of a module object. And most other
languages don't allow you to run interactive sessions where it is easy to
mistakenly make variables you don't want.

py> x = 1
py> u = x+2 # oops, typo, meant y not u
py> del u # prevent confusion in the programmer's mind

It is also useful sometimes to delete a module object from the top level
namespace before re-importing it, rather than merely reloading it. That
requires being able to delete a variable.

In summary: del being a keyword works. del() being an object method is
unclear, confusing and complicated.

 

[Ron Adam]

Since this is a Python 3k item... What would be the consequence of
making None the default value of an undefined name? And then assigning
a name to None as a way to delete it?

[ ... ]

Any drawbacks?

Lots more hard-to-find errors from code like this:

filehandle = open( 'somefile' )
do_something_with_an_open_file( file_handle )
filehandle.close( )

Regards,
Dan

If do_something_with_an_open_file() is not defined. Then you will get:

TypeError: 'NoneType' object is not callable


If "file_handle" (vs "filehandle") is None. Then you will still get an
error as soon as you tried to use the invalid file handle.

AttributeError: 'NoneType' object has no attribute 'read'


If the error was filehundle.close() you will get:

AttributeError: 'NoneType' object has no attribute 'close'


I don't think any of those would be hard to find.


Cheers,
Ron

 

[Devan L]

Here's a couple of examples from my own code:

# from a Banzhaf Power Index calculator
# adds things that aren't numbers
return reduce(operator.add,
(VoteDistributionTable({0: 1, v: 1}) for v in electoral_votes))

return sum([VoteDistributionTable({0:1, v:1} for v in
electoral_votes],VoteDistributionTable({}))
Any time you use operator.add, you can probably use
sum(sequence,initialvalue)

# from a custom numeric class
# converts a tuple of digits into a number
mantissa = sign * reduce(lambda a, b: 10 * a + b, mantissa)

I'll admit I can't figure out a way to replace reduce without writing
some ugly code here, but I doubt these sorts of things appear often.

 

[Ron Adam]

I'll admit I can't figure out a way to replace reduce without writing
some ugly code here, but I doubt these sorts of things appear often.

It's not ugly or difficult to define a named function.

def digits_to_value(seq):
v = 0
for d in seq:
v = v*10+d
return v


Then where you need it.

mantissa = sign * digits_to_value(mantissa)


One of the motivations is the reduce-lambda expressions are a lot harder
to read than a properly named function.

And a function will often work faster than the reduce-lambda version as
well.

Cheers,
Ron

 

[Dan Bishop]

Here's a couple of examples from my own code:

# from a Banzhaf Power Index calculator
# adds things that aren't numbers
return reduce(operator.add,
(VoteDistributionTable({0: 1, v: 1}) for v in electoral_votes))

return sum([VoteDistributionTable({0:1, v:1} for v in
electoral_votes],VoteDistributionTable({}))
Any time you use operator.add, you can probably use
sum(sequence,initialvalue)

Actually, it's

sum([VoteDistributionTable({0:1, v:1}) for v in electoral_votes],
VoteDistributionTable({0: 1}))

but you're right about being able to use sum here.

I'll admit I can't figure out a way to replace reduce without writing
some ugly code here, but I doubt these sorts of things appear often.

Of the quarter-megabyte of my Python code currently on my hard drive, I
can find two reduces that can't be replaced by sum, product, any, or
all. The other one is

return reduce(lambda x, y: x << 1 | y, bits)

 

[Scott David Daniels]

If do_something_with_an_open_file() is not defined. Then you will get:
TypeError: 'NoneType' object is not callable


If "file_handle" (vs "filehandle") is None. Then you will still get an
error as soon as you tried to use the invalid file handle.

Ah, but where you use it can now far from the error in the source. The
None could get packed up in a tuple, stored in a dictionary, all manner
of strange things before discovering it was an unavailable value. I
would love the time back that I spent chasing bugs when Smalltalk told
me (I forget the exact phrasing) "nil does not respond to message abc."
My first reaction was always, "of course it doesn't, this message is
useless." You really want the error to happen as close to the problem
as possible.

--Scott David Daniels
(e-mail address removed)

 

[Dan Sommers]

Since this is a Python 3k item... What would be the consequence of
making None the default value of an undefined name? And then assigning
a name to None as a way to delete it?

[ ... ]

Any drawbacks?

Lots more hard-to-find errors from code like this:
filehandle = open( 'somefile' )
do_something_with_an_open_file( file_handle )
filehandle.close( )
Regards,
Dan

[ ... ]

If "file_handle" (vs "filehandle") is None. Then you will still get an
error as soon as you tried to use the invalid file handle.

AttributeError: 'NoneType' object has no attribute 'read'

This is the one of which I was thinking. So you see this error at the
end of a (long) traceback, and try to figure out where along the (long)
line of function calls I typed the wrong name. Currently, the very end
of the traceback points you right at the bad code and says "NameError:
name 'filehandle' is not defined," which tells me (very nearly) exactly
what I did wrong.

If the error was filehundle.close() you will get:
s/u/a/

;-)

I don't think any of those would be hard to find.

I guess it depends on how long your traceback is and how big those
functions are. Also, from the Zen:

Explicit is better than implicit.

although from previous threads, we know that every pythonista has his or
her own definitions of "explicit" and "implicit."

Regards,
Dan

 

[Stian =?iso-8859-1?Q?S=F8iland?=]

So, who would object the full-word versions for python 3K ?
def -> define
del -> delete
exec -> execute
elif -> else if

I'm all for it. I would even be tempted of changing def to function, but
it would look stupid in:

class A:
function make_my_day(self):
return "Your day"
a = A()


since a.make_my_day() is really a method, not a standalone function.
We could use "function" instead of "lambda" though =)

Objections for the "else if" might be that it sounds like you can
replace "else if" with "else x=94" if you want. Thumbs up for "else if"
because it explains what it is much better than "elif". "elseif" ?

 

[Stian =?iso-8859-1?Q?S=F8iland?=]

I had NEVER even heard the word "tuple" before learning Python. I spent
weeks mispelling it as "turple", and I finally had to look it up in a
dictionary to see if it was a real English word. Out of the four English
dictionaries in my house, none of them have the word.

Agree, I have the problem of writing "tupple" in all comments and
documentations. It's a weird word indeed =)

t = immutable_list(L)
map(anonymous_function x: x+1, L)

Hey, I know!

t = python.util.ImmutableArrayList.fromCollection(L.getAbstractCollection())

python.util.functional.applyFunctionOnCollection(
(class implements python.util.functional.AnonymousFunction:
def anonymousFunction(x):
return x+1
), L)

 

[Stian =?iso-8859-1?Q?S=F8iland?=]

*No more NamesError exceptions!
print value

So you could do lot's of funny things like:

def my_fun(extra_args=None):
if not extraargs:
print "Behave normally"
extra_args = 1337

if extraargs:
asdkjaskdj
..
if extra_args:
kajsdkasjd

and get no errors at all, but switching back and forth between the
different behavours because you actually did expect None, but from an
EXISTING variable.

*No initialization needed for a while loop!

while not something:
if <condition>:
something = True

This is the only "good" case I could find, but opening for a lots of
errors when you get used to that kind of coding:

while not finished:
foo()
finished = calculate_something()

(..)
(..) # Added another loop
while not finished:
bar()
finished = other_calculation()

Guess the amount of fun trying to find out the different errors that
could occur when bar() does not run as it should because the previous
"finished" variable changes the logic.

If you want to experiment with such designs, all you need to do is to
start your code with <?php

*Test if name exists without using a try-except!
if something == None:
something = value

Now this is a question from newcomers on #python each day.. "How do I
check if a variable is set?".

Why do you want to check if a variable is set at all? If you have so
many places the variable could or could not be set, your program design
is basically flawed and must be refactored.

 

[Stian =?iso-8859-1?Q?S=F8iland?=]

map(lambda x: if x == 0: 1; else: math.sin(x)/x,
myList)

And now for the "readable" list comprehension version:

[x==0 and 1 or math.sin(x)/x for x in myList]

Now even though I'm into the short-circuiting of and-or and even
occasionally have used such masturbation techniques as this, I don't
think it qualifies as pythonic.

If it was me, I would probably even have written:

[x and math.sin(x)/x or 1 for x in myList]

 

[Ron Adam]

This is the one of which I was thinking. So you see this error at the
end of a (long) traceback, and try to figure out where along the (long)
line of function calls I typed the wrong name. Currently, the very end
of the traceback points you right at the bad code and says "NameError:
name 'filehandle' is not defined," which tells me (very nearly) exactly
what I did wrong.

The actual error could be improved a bit to refer the the name of the
and line the error is in. Which would help some.

AttributeError: 'NoneType' object "file_handle" has no attribute
'read'

I guess it depends on how long your traceback is and how big those
functions are. Also, from the Zen:

Explicit is better than implicit.

although from previous threads, we know that every pythonista has his or
her own definitions of "explicit" and "implicit."

True, but this isn't any different than any other 'Type' error, or value
error. And considerably easier to find than one off errors.

There would be more restrictions on None than there are now so it
wouldn't be as bad as it seems. For example passing a None in a
function would give an error as the name would be deleted before the
function actually gets it.

So doing this would give an error for functions that require an argument.

def foo(x):
return x

a = None
b = foo(a) # error because a dissapears before foo gets it.


So they wouldn't propagate like you would expect.

Hmmm.... interesting that would mean... lets see.


1. var = None # removes ref var, this is ok

2. None = var # give an error of course

3. var = undefined_var # same as "var = None", that's a problem!

Ok... this must give an error because it would delete var silently!
Definitely not good. So going on, on that basis.

4. undefined == None # Special case, evaluates to True.

5. def foo():return None # same as return without args

6. x = foo() # Would give an error if foo returns None

This might be an improvement over current behavior. Breaks a lot of
current code though.

7. if undefined: # error

8. if undefined==None # Special case -> True

Good for checking if vars exist.

9. if undefined==False # Error, None!=False (or True)

9. while undefined: # error

10 while undefined==None:

Possible loop till defined behavior.


Ok... and undefined var returning None is a bad idea, but using None to
del names could still work. And (undefined==None) could be a special
case for checking if a variable is defined. Otherwise using an
undefined name should give an error as it currently does.

Cheers,
Ron

 

[Ron Adam]

Ah, but where you use it can now far from the error in the source. The
None could get packed up in a tuple, stored in a dictionary, all manner
of strange things before discovering it was an unavailable value. I
would love the time back that I spent chasing bugs when Smalltalk told
me (I forget the exact phrasing) "nil does not respond to message abc."
My first reaction was always, "of course it doesn't, this message is
useless." You really want the error to happen as close to the problem
as possible.

--Scott David Daniels
(e-mail address removed)

Yep, I concede. The problem is if undefined names return None, and None
deletes a name, then assigning a undefined name to an existing name,
will delete it silently.

Definitely Not good!

So it should give an error as it currently does if an undefined name is
used on the right side of an =.

It could still work otherwise to unbind names, and (undefined_name ==
None) could still be a valid way to check if a name is defined without
using a try-except.

But it would definitely be a Python 3000 change as all the functions
that return None would then cause errors.

Cheers,
Ron

 

[Terry Reedy]

What precedence ? I don't know of another language that uses def or del

DOS used DEL, BASIC used DEF.

tjr

 

[Stian =?iso-8859-1?Q?S=F8iland?=]

This is probably the more correct way to do it.

def flatten(seq):
i = 0
while i!=len(seq):
while isinstance(seq,list):
seq[i:i+1]=seq
i+=1
return seq

Or what about a recursive generator?

a = [1,2,[[3,4],5,6],7,8,[9],[],]

def flatten(item):
try:
iterable = iter(item)
except TypeError:
yield item # inner/final clause
else:
for elem in iterable:
# yield_return flatten(elem)
for x in flatten(elem):
yield x

print list(flatten(a))


Of course, one of the problems here is that there is no way to
yield_return except to create yet another stupid-looking for-loop. This
is one of the flaws in the current generator functionallity of Python.

Using yield_return could also make it more obvious that the result is in
fact a generator in functions that wrap generators.