Default method arguments

[Mike Meyer]

I'm not very familiar with Python, so please explain me why should
containers be used?
For example in one of Paul Graham's essays there's an example of
'generator of accumulators' in Python:

def foo(n):
s = [n]
def bar(i):
s[0] += i
return s[0]
return bar

1) So, why just using 's = n' is not suitable? (It doesn't work, Python
'doesn't see' s, but why?)

The Python assignment statements bind a name to a value. By default,
they bind it in the current namespace. Doing "s = n" (or s <op>= n) in
the function bar binds the name s in the function bar, and leaves the
value in foo as it was. "s[0] = i" (or s[0 += i) binds the name s[0],
not the name s, and hence mutates the object bound to s instead of
binding s in the function bar's namespace. In reality, this is
implemented by a mutator method of s, but it *looks* like you're
binding s[0].

2) Is 'foo.s = n' a correct solution? It seems to be a little more
elegant. (I tested it, and it worked well)

It's basically the same solution. You're replacing binding a variable
with mutating an object bound to a name in an outer scope. In one case
the container is named s and is a list that you're setting an element
of. In the other case, the container is named foo and is an object
that you're setting an attribute on.

<miker

 

[Gregory Petrosyan]

Thanks a lot, I understood the rule. Let's don't discuss this
(containers etc.) anymore, or it'll be offtopic.

 

[Steven D'Aprano]

Another solution to this is the use of a 'marker' object and identity test:

_marker = []
class A(object):
def __init__(self, n):
self.data =n
def f(self, x = _marker):
if x is _marker:
x = self.data
print x

I would like to see _marker put inside the class' scope. That prevents
somebody from the outside scope easily passing _marker as an argument to
instance.f. It also neatly encapsulates everything A needs within A.

class A(object):
_marker = []
def __init__(self, n):
self.data =n
def f(self, x = _marker):
if x is self.__class__._marker:
# must use "is" and not "=="
x = self.data
print x

Note the gotcha though: in the method definition, you refer to a plain
_marker, but in the method code block, you need to qualify it.

 

[Fredrik Lundh]

Another solution to this is the use of a 'marker' object and identity test:

_marker = []
class A(object):
def __init__(self, n):
self.data =n
def f(self, x = _marker):
if x is _marker:
x = self.data
print x

I would like to see _marker put inside the class' scope. That prevents
somebody from the outside scope easily passing _marker as an argument to
instance.f.

if you don't want people to be able to easily pass _marker as an argument
to the f method, you probably shouldn't use it as the default value.

</F>

 

[Bengt Richter]

Another solution to this is the use of a 'marker' object and identity test:

_marker = []
class A(object):
def __init__(self, n):
self.data =n
def f(self, x = _marker):
if x is _marker:
x = self.data
print x

I would like to see _marker put inside the class' scope. That prevents
somebody from the outside scope easily passing _marker as an argument to
instance.f.

if you don't want people to be able to easily pass _marker as an argument
to the f method, you probably shouldn't use it as the default value.

LOL ;-)

Regards,
Bengt Richter

 

[Bengt Richter]

Alex Martelli wrote, in part:

FWIT and ignoring the small typo on the inner def statement (the
missing ':'), the example didn't work as I (and possibily others) might
expect. Namely it doesn't make function f() a bound method of
instances of class A, so calls to it don't receive an automatic 'self''
argument when called on instances of class A.

This is fairly easy to remedy use the standard new module thusly:

import new
class A(object):
def __init__(self, n):
self.data = n
def f(self, x = self.data):
print x
self.f = new.instancemethod(f, self, A)

This change underscores the fact that each instance of class A gets a
different independent f() method. Despite this nit, I believe I
understand the points Alex makes about the subject (and would agree).

Or as Alex mentioned, a custom descriptor etc is possible, and can also
protect against replacing f by simple instance attribute assignment
like inst.f = something, or do some filtering to exclude non-function
assignments etc., e.g., (not sure what self.data is really
needed for, but we'll keep it):

BTW, note that self.data initially duplicates the default value,
but self.data per se is not used by the function (until the instance
method is replace by one that does, see further on)
... def __init__(self, inst_fname):
... self.inst_fname= inst_fname
... def __get__(self, inst, cls=None):
... if inst is None: return self
... return inst.__dict__[self.inst_fname].__get__(inst, cls) # return bound instance method
... def __set__(self, inst, val):
... if not callable(val) or not hasattr(val, '__get__'): # screen out some impossible methods
... raise AttributeError, '%s may not be replaced by %r' % (self.inst_fname, val)
... inst.__dict__[self.inst_fname] = val
...

The above class defines a custom descriptor that can be instatiated as a class
variable of a given name. When that name is thereafter accessed as an attribute
of an instance of the latter class (e.g. A below), the decriptor __get__ or __set__
methods will be called (the __set__ makes it a "data" descriptor, which intercepts
instance attribute assignment.
... def __init__(self, n):
... self.data = n
... def f(self, x = self.data):
... print x
... self.__dict__['f'] = f # set instance attr w/o triggering descriptor
... f = BindInstMethod('f')
...
<bound method A.f of <__main__.A object at 0x02EF3B0C>>

Note that a.f is dynamically bound at the time of a.f access, not
retrieved as a prebound instance method.

Since the default is an independent duplicate of a.data
a call with no arg produces the original default: this arg overrides the default

Try to change a.f Traceback (most recent call last):
File "<stdin>", line 1, in ?
File "<stdin>", line 10, in __set__
AttributeError: f may not be replaced by 'sabotage f'

Now use a function, which should be accepted (note: a function, not an instance method) 'not original data 5not original data 5'
That was self.data*2 per the lambda we just assigned to a.f
BTW, the assignment is not directly to the instance attribute.
It goes via the descriptor __set__ method.

>>> a.data = 12
>>> a.f() 24
>>> b = A('bee')
>>> b.f

>>> b.f() bee
>>> b.f('not bee') not bee
>>> b.data 'bee'
>>> b.data = 'no longer bee'
>>> b.f() bee
>>> b.data 'no longer bee'
>>> b.f = lambda self: ' -- '.join([self.data]*3)
>>> b.data 'no longer bee'
>>> b.data = 'ha'
>>> b.f() 'ha -- ha -- ha'
>>> b.f = lambda self, n='default of n':n
>>> b.data 'ha'
>>> b.f(123) 123
>>> b.f() 'default of n'
>>> a.f()

24

Now let's add another name that can be used on instances like f Traceback (most recent call last):
File "<stdin>", line 1, in ?
Traceback (most recent call last):
File "<stdin>", line 1, in ?
File "<stdin>", line 7, in __get__
KeyError: 'g'
No instance method b.g defined yet (A.__init__ only defines f)

Make one with a default
If we bypass method assignment via the descriptor, we can sapotage it:
Traceback (most recent call last):
File "<stdin>", line 1, in ?
File "<stdin>", line 10, in __set__
AttributeError: g may not be replaced by 'sabotage'
That was rejected

but,

>>> a.__dict__['g'] = 'sabotage' # this will bypass the descriptor
>>> a.g

Traceback (most recent call last):
File "<stdin>", line 1, in ?
File "<stdin>", line 7, in __get__
AttributeError: 'str' object has no attribute '__get__'
The descriptor couldn't form a bound method since 'sabotage' was not
a function or otherwise suitable.

But we can look at the instance attribute directly:

>>> a.__dict__['g']

'sabotage'

We could define __delete__ in the descriptor too, but didn't so
Traceback (most recent call last):
File "<stdin>", line 1, in ?
AttributeError: __delete__

We could have made the descriptor return a.g if unable to form a bound method,
but then you'd probably want to permit arbitrary assignment to the descriptor-controlled
attributes too ;-)

Regards,
Bengt Richter

 

[bonono]

What you want is essentially :

if parm_x is not supplied, use self.val_x

So why not just express it clearly at the very beginning of the
function :

def f(self, parm_x=NotSupplied, parm_y=NotSupplied ,,,)
if parm_x is NotSupplied: parm_x = self.val_x
if parm_y is NotSupplied: parm_y = self.val_y

Much easier to understand than the "twisting your arm 720 degree in the
back" factory method, IMO.

 

[Duncan Booth]

I would like to see _marker put inside the class' scope. That prevents
somebody from the outside scope easily passing _marker as an argument
to instance.f. It also neatly encapsulates everything A needs within
A.

Surely that makes it easier for someone outside the scope to pass in
marker:

class A(object):
_marker = []
def __init__(self, n):
self.data =n
def f(self, x = _marker):
if x is self.__class__._marker:
# must use "is" and not "=="
x = self.data
print x
5

What you really want is for the marker to exist only in its own little
universe, but the code for that is even messier:

class A(object):
def __init__(self, n):
self.data =n
def make_f():
marker = object()
def f(self, x = _marker):
if x is _marker:
x = self.data
print x
return f
f = make_f()

6

 

[Steven D'Aprano]

LOL ;-)

Ha ha *wink*

What I meant was to discourage people from treating _marker as just
another ordinary sort of object, then making pointless bugs
reports "when you pass _marker as the argument to the f method, it doesn't
print _marker but instead prints something else."

My philosophy is, any time you have an object that has a magic meaning
(e.g. as a sentinel), don't tempt your users to try to use it as if it
were an ordinary object.

 

[Steven D'Aprano]

Surely that makes it easier for someone outside the scope to pass in
marker:
5

Sure, but they have to explicitly qualify marker with the instance. If
they want to do that, I'm not going to stop them. But I'm trying to avoid
tempting them from doing this:

instance.f(_marker)

and then complain that it doesn't print _marker.

In other words, I don't want to take away their ability to shoot
themselves in the foot, but I want them to have to *think about it* before
doing so.

 

[Fredrik Lundh]

What you really want is for the marker to exist only in its own little
universe, but the code for that is even messier:

class A(object):
def __init__(self, n):
self.data =n
def make_f():
marker = object()
def f(self, x = _marker):

NameError: global name '_marker' is not defined

if x is _marker:
x = self.data
print x
return f
f = make_f()

6

in another universe, perhaps, but not very far away:

>>> instance.f.im_func.func_defaults[0]

>>> inspect.getargspec(A.f)

(['self', 'x'], None, None, (<object object at 0x009EC438>,))

</F>

 

[Duncan Booth]

My philosophy is, any time you have an object that has a magic meaning
(e.g. as a sentinel), don't tempt your users to try to use it as if it
were an ordinary object.

In that case the simplest thing is to give _marker a more appropriate name
such as '_use_late_bound_default_for_argument' or '_foot_gun_aim_fire'.

 

[Martin Miller]

Mike Meyer wrote, in part::

It's basically the same solution. You're replacing binding a variable
with mutating an object bound to a name in an outer scope. In one case
the container is named s and is a list that you're setting an element
of. In the other case, the container is named foo and is an object
that you're setting an attribute on.

Well, perhaps the same in the sense of name binding, but there's a
subtle difference in replacing the 's = [n]' with 'foo.s = n'. Namely
that in the former case (with the essay's original code) a separate
container is created when foo() is first called and is what is used in
subsequent calls to the function returned. Whereas in the latter case
where the foo object itself is used as the container, there's only a
single container used by all returned objects -- which would cause
problems if you try accumulating two or more different totals
simultaneously.

Here's a very contrived test case which illustrates the point I'm
trying to make:

def foo(n):
foo.s = n
def bar(i):
foo.s += i
return foo.s
return bar

a1 = foo(0)
a2 = foo(0)
print "before a1(0):", a1(0)
print "before a2(0):", a2(0)
a1(1)
a2(1)
print "after a1(0):", a1(0)
print "after a2(0):", a2(0)
before a1(0): 0
before a2(0): 0
after a1(0): 2
after a2(0): 2

Notice that it even though each was only incremented by 1 once, they
interacted, and show the effects of two calls. This doesn't happen in
in Paul Graham's version, where the two 'after' calls would correctly
retrun a value of 1.

-Martin

 

[Gregory Petrosyan]

Thanks Martin, you are right.

 

[Peter Otten]

Well, perhaps the same in the sense of name binding, but there's a
subtle difference in replacing the 's = [n]'  with 'foo.s = n'.  Namely
that in the former case (with the essay's original code) a separate
container is created when foo() is first called and is what is used in
subsequent calls to the function returned.  Whereas in the latter case
where the foo object itself is used as the container, there's only a
single container used by all returned objects -- which would cause
problems if you try accumulating two or more different totals
simultaneously.

[snip example using the outer foo() as a container]

You can easily get a unique container using the function attribute style, to
-- just use the inner function bar():
.... def bar(i):
.... bar.i += 1
.... re
........ def bar(i):
.... bar.s += i
.... return bar.s
.... bar.s = n
.... return bar
....(1, 1)

Peter