determining the number of output arguments

[Carlos Ribeiro]

Bengt Richter wrote:



we were discussing a feature that doesn't yet exist. i believe it's possible to
get around the recursion issue by only allowing the __names__ attribute to be
accessed in the immediate explicitly called tuple. i really do like the idea of
named tuple elements.

There are a few requirements that can be imposed to avoid problems.
First, __names__ is clearly a property, acessed via get & set (which
allows to trap some errors). It should accept only tuples as an
argument (not lists) to avoid potential problems with external
references and mutability of the names. As for the validation, I'm not
sure if it's a good idea to check for strings. maybe just check if the
'names' stored in the tuple are immutable (or perhaps 'hashable') is
enough.

--
Carlos Ribeiro
Consultoria em Projetos
blog: http://rascunhosrotos.blogspot.com
blog: http://pythonnotes.blogspot.com
mail: (e-mail address removed)
mail: (e-mail address removed)

 

[Carlos Ribeiro]

? No they're not -- a caller knows them, in order to be able to call
with named argument syntax.

On my original post I pointed out that named arguments are an
exception to this rule. For the most of the times, the caller does not
*need* to know the internal argument names. Named arguments are
entirely optional, and in fact, if you provide the arguments in the
correct order, you can avoid using them in almost every situation
(**kw being the exception).

The point was: fixing names on the return code introduces coupling.
But after thinking a little bit more, the issue does not seem as clear
to me as before. After all, *some* coupling is always needed, and
positional return values also are "coupled" to some extent. I think
that the named tuples idea is nice because it works entirely
symmetrical with the named arguments; it allows to use a shortcut
positional notation, or to use a named access, at the will of the
programmer, for the same function call.

Agreed, by analogy with '%(foo)s %(bar)s' % somedict.


No, enriching dicts to support an optional default value (or default
value factory) looks like a separate fight to me. As does syntax sugar,
IMHO. If we can get the semantics in 2.5 with a PEP, then maybe sugar
can follow once the usefulness is established to GvR's contents.

I'm also trying to focus on tuples, so I'm leaving dicts out of this
for now. But clearly named tuples & immutable dicts are closely
related; but of the two, named tuples have an advantage because they
are naturally ordered.

--
Carlos Ribeiro
Consultoria em Projetos
blog: http://rascunhosrotos.blogspot.com
blog: http://pythonnotes.blogspot.com
mail: (e-mail address removed)
mail: (e-mail address removed)

 

[Duncan Booth]

There are a few requirements that can be imposed to avoid problems.
First, __names__ is clearly a property, acessed via get & set (which
allows to trap some errors). It should accept only tuples as an
argument (not lists) to avoid potential problems with external
references and mutability of the names. As for the validation, I'm not
sure if it's a good idea to check for strings. maybe just check if the
'names' stored in the tuple are immutable (or perhaps 'hashable') is
enough.

Your idea of a __names__ attribute suffers from a problem that the common
use case would be to return a tuple with appropriate names. Right now you
can do that easily in one statement but if you have to assign to an
attribute it becomes messy.

An alternative would be so add a named argument to the tuple constructor so
we can do:

return tuple(('1', '2'), names=('ONE', 'TWO'))

and that would set the __names__ property. If you allow this then you can
make the __names__ property readonly and the problem of introducing a cycle
in the __names__ attributes goes away.

 

[Carlos Ribeiro]

Carlos Ribeiro wrote:




Your idea of a __names__ attribute suffers from a problem that the common
use case would be to return a tuple with appropriate names. Right now you
can do that easily in one statement but if you have to assign to an
attribute it becomes messy.

An alternative would be so add a named argument to the tuple constructor so
we can do:

return tuple(('1', '2'), names=('ONE', 'TWO'))

and that would set the __names__ property. If you allow this then you can
make the __names__ property readonly and the problem of introducing a cycle
in the __names__ attributes goes away.

I think that a better way to solve the problem is to create a names
method on the tuple itself:

return ('1', '2').names('ONE', 'TWO')

It's shorter and clean, and avoids a potential argument against named
parameters for the tuple constructor -- none of the standard
contructors take named parameters to set extended behavior as far as I
know.

--
Carlos Ribeiro
Consultoria em Projetos
blog: http://rascunhosrotos.blogspot.com
blog: http://pythonnotes.blogspot.com
mail: (e-mail address removed)
mail: (e-mail address removed)

 

[Bryan]

I think that a better way to solve the problem is to create a names
method on the tuple itself:

return ('1', '2').names('ONE', 'TWO')

It's shorter and clean, and avoids a potential argument against named
parameters for the tuple constructor -- none of the standard
contructors take named parameters to set extended behavior as far as I
know.

but doesn't this feel more pythonic and more consistant?

return ('ONE':'1', 'TWO':'2')

 

[Carlos Ribeiro]

but doesn't this feel more pythonic and more consistant?

return ('ONE':'1', 'TWO':'2')

The problem is that it involves changing the language, and at this
point, the idea is to devise a solution that *doesn't* need to change
the language. But it's a possibility for the future, after a simpler
version of the same basic feature is approved and implemented.


--
Carlos Ribeiro
Consultoria em Projetos
blog: http://rascunhosrotos.blogspot.com
blog: http://pythonnotes.blogspot.com
mail: (e-mail address removed)
mail: (e-mail address removed)

 

[Dave Brueck]

You take every example mentioned to date, which is largely:

* OS code trying to be as thin as possible (stat, socket)
* math-type code where the tuple really is the class, as you suggest
* more-or-less contrived one- or two-liner code

and you've still got way, way, way less than 1% of the code of the vast
majority of programs.

Like any other code smell, there are times to use it. But I say it's a
*smell*, and those of you implicitly reading that to mean "Returning
tuples is *never* a good idea" are doing me a disservice; please go look
up "code smell" and the word "generally".

Well, the first page I found that listed the term "code smell" also mentioned
"code wants to be simple", and having scanned through a good sized chunk of
Python code looking to see where multiple return values were used, I am now of
the opinion that, not only are multiple return values _not_ automatically code
smell, but they are one of the key ingredients to avoiding monolithic functions
(and thus they help keep code simple).

The code I looked through had about 4500 return statements. Of those, roughly
350 were returns with multiple values. Of those, 26 had 4 or more return values
- if code smell is a hint that something _might_ be wrong, I'd say those 26 have
code smell, but the remaining 320 or so do not automatically raise concern (I
inspected those 26 in detail and saw only 4 or 5 cases that'd be removed next
time the code got refactored).

If you use it every once in a while where it is the right solution, great.
I do too. If you're using it every other function in a module, and it
isn't a thin wrapper around some other library, you've got a code smell.

I found that multiple return values were not often used in "public" functions
(those meant to be accessible from outside the module). Instead the most common
use case was between functions that had been separated to divide the problem
into smaller, testable code chunks (i.e. - rather than having a huge function to
parse a log line, perform computations, and send the results to output, there
was a function that'd take the raw data, call a helper function to parse it,
call another helper to compute the result, and call another to send the results
off for output)

Without the ability to return multiple values, it would be much more cumbersome
to split the code into small, easily understood chunks because for any
non-trivial task, each logical "step" in that task often has multiple inputs as
well as multiple outputs.

After reviewing the code, I think that the use of multiple return values is, in
and of itself, _almost never_ a good hint of a problem - way too many false
positives (a better rule of thumb would be that code that returns BIG tuples has
code small).

(It's probably a class or two trying to get out.)

No, at least not in the code I looked at - in the vast majority of the uses of
multiple return values, the coupling between the functions was already very
tight (specialized) and the callee-caller relationship was one-to-one or
one-to-few (the function returning multiple values was called by one or very few
functions), and the data didn't move around between function as a unit or group,
such that making a class would have been pure overhead like many C structures -
never reused, and used only to cross the "bridge" between the two functions
(immediately unpacked and discarded upon reception).

You've got a *big* code
smell if you are unpacking those return values many times, because now
you've hard coded the size of the tuple into the code in many places.

Yes, this was mentioned previously.

-Dave

 

[Hung Jung Lu]

Dicts are not the best way to make it, because they are not ordered
But perhaps we could have something like "named tuples";

@returns('year','month','day')
def today():
...
return year, month, day

As someone has mentioned, usage of tuples is not very scalable and
tend to lend itself to carcass entries in the long run, or simply to
the breaking of compatibility. But I guess in your case, you want to
do something like printing out the results, so, as long as all items
are involved, an order would be beneficial.

------------------

For whatever it's worth, here are two more approaches for unordered
keyword argument and return.

(1) Usage of structures certainly is painful in C++, but not in
Python. What I often use is some generic object:

class Generic: pass

def today(p):
print p.message
r = Generic()
r.year, r.month, r.day = 2004, 11, 18
return r

p=Generic()
p.message = 'Hello!'
result = today(p)

I suspect a large number of people use this approach. Generic objects
are also good for pickling/serialization. (By the way, why is the new
style class "object()" made so that no dynamic attributes can be
assigned to it? Compared to dictionary, generic object is very helpful
as a syntax gadget and is used by many people. I am a bit surprised
that it's not part of the language. Why make so many people having to
resort to the "class Generic: pass" trick?)

(2) Use the function object itself:


def f():
if not hasattr(f,'x'):
f.x = 1
else:
f.x += 1
f.y, f.z = 2*f.x, 3*f.x

f()
print f.x, f.y, f.z
f()
print f.x, f.y, f.z

Of course, this approach has more limited applicability. (Not good for
multithreaded case, not good for renaming the function object or
passing it around.)

Hung Jung

 

[Duncan Booth]

I think that a better way to solve the problem is to create a names
method on the tuple itself:

return ('1', '2').names('ONE', 'TWO')

It's shorter and clean, and avoids a potential argument against named
parameters for the tuple constructor -- none of the standard
contructors take named parameters to set extended behavior as far as I
know.

It doesn't have to be a named parameter, it could just be a second
positional parameter but I think a named parameter reads better.

The problem with that is that you are still breaking the rule that tuples
should be immutable. Currently there is no easy way to copy a tuple:
True

Once I have a tuple I know it isn't going to change no matter what happens.
You could say that the names method can only be called once, but even then
the tuple is still changing after its creation and you are still going to
have to introduce some way for me to take a tuple with one set of names and
create a new tuple with a different set of names such as changing the
behaviour of the tuple constructor and the copy.copy function.

Another way to avoid the named argument would be to provide tuple with a
factory method. How about:

return tuple.named((1, 2), ('one', 'two))

 

[Elbert Lev]

There are a few requirements that can be imposed to avoid problems.
First, __names__ is clearly a property, acessed via get & set (which
allows to trap some errors). It should accept only tuples as an
argument (not lists) to avoid potential problems with external
references and mutability of the names. As for the validation, I'm not
sure if it's a good idea to check for strings. maybe just check if the
'names' stored in the tuple are immutable (or perhaps 'hashable') is
enough.

With on changes to the language the same can be achived by using db_row module

 

[Lonnie Princehouse]

Not quite the syntax you want, but better imho since it doesn't
involve name redundancy:

locals().update( {'a': 1, 'b': 2, 'c': 3} )

 

[Denis S. Otkidach]

On 18 Nov 2004 10:05:23 -0800

Not quite the syntax you want, but better imho since it doesn't
involve name redundancy:

locals().update( {'a': 1, 'b': 2, 'c': 3} )

Are you sure it will work with locals?
.... locals().update(d)
.... print a
.... Traceback (most recent call last):
File "<stdin>", line 1, in ?
File "<stdin>", line 3, in f
NameError: global name 'a' is not defined

Or even:
.... a = 1
.... locals().update(d)
.... print a
.... 1

 

[Graham Fawcett]

2. Named tuples should, for all practical purposes, be an extension of
standard tuples.

Yes -- and explicitly so. A tuple that has names ought not be a tuple
at all, but rather a tuple subtype. The benefit of adding __names__ as
a tuple attribute is lost on me.

Let's add libraries, not language changes:


class namedtuplewrapper(tuple):
"""
Subclasses wrap existing tuples, providing names.
"""

_names_ = []

def __getattr__(self, name):
try:
x = self._names_.index(name)
except ValueError:
raise AttributeError, 'no such field: %s' % name
if x >= 0:
return self[x]

class namedtuple(namedtuplewrapper):
"""
Sugar for subclasses that construct named tuples from
positional arguments.
"""

def __new__(cls, *args):
return tuple.__new__(cls, args)


if __name__ == '__main__':

# namedtuple example

class foo(namedtuple):
_names_ = ['one', 'two', 'three', 'four', 'five']

f = foo(1, 2, 3, 4, 5)
assert f.one + f.four == f.five == 5


# wrapper example

class loctime(namedtuplewrapper):
_names_ = [
'year', 'month', 'day',
'hour', 'min', 'sec',
'wday', 'yday', 'isdst'
]

import time
loc = loctime(time.localtime())
print loc.year, loc.month, loc.day

# arbitrary naming of instances...
loc._names_ = ['a', 'b', 'c']
print loc.a


-- Graham

 

[Jeff Shannon]

Maybe keyword unpacking could spell that with a '**' assignment target,
e.g.,

** = foo() # update local bindings with all legal-name bindings in returned dict

I would *not* like to see this. ISTM that, when reading a function, I
should be able to see where every name in that function came from.
Having global values appear from outside of the function is bad enough;
introducing a new way to magically create variables whose names I can't
see would be (IMO) very, very bad.

Jeff Shannon
Technician/Programmer
Credit International

 

[Steven Bethard]

def today(p):
print p.message
r = Generic()
r.year, r.month, r.day = 2004, 11, 18
return r
[snip]

I suspect a large number of people use this approach. Generic objects
are also good for pickling/serialization. (By the way, why is the new
style class "object()" made so that no dynamic attributes can be
assigned to it?

My understanding is that this is for efficiency reasons... I remember
some older discussions, but they're kinda hard to google since 'object'
isn't a very good query term... Personally, I don't really care about
being able to assign attributes dynamically to an object() instance, but
I would like to be able to do something like:
(18, 11, 2004)

where object's __init__ takes keyword arguments to designate attributes.
This would let you use object basically as a record. If I remember
right though, the problem with this is that it introduces overhead for
all classes that inherit from object. (I kinda thought it had something
to do with defining __slots__, but I'll wait for someone with more
knowledge in this area to fill in the details...)

def f():
if not hasattr(f,'x'):
f.x = 1
else:
f.x += 1
f.y, f.z = 2*f.x, 3*f.x

f()
print f.x, f.y, f.z
f()
print f.x, f.y, f.z

Of course, this approach has more limited applicability. (Not good for
multithreaded case, not good for renaming the function object or
passing it around.)

Just to clarify for anyone who wasn't following closely, this is "not
good for renaming the function object" in cases like:
.... if not hasattr(f,'x'):
.... f.x = 1
.... else:
.... f.x += 1
.... f.y, f.z = 2*f.x, 3*f.x
....Traceback (most recent call last):
File "<interactive input>", line 1, in ?
File "<interactive input>", line 3, in f
AttributeError: 'int' object has no attribute 'x'

This is a little contrived here, but the point is that relying *inside*
a function on the name that the function is def'd with is probably not a
good idea.

Steve

 

[Fernando Perez]

My understanding is that this is for efficiency reasons...  I remember
some older discussions, but they're kinda hard to google since 'object'
isn't a very good query term...  Personally, I don't really care about
being able to assign attributes dynamically to an object() instance, but
I would like to be able to do something like:

(18, 11, 2004)

Given that the necessary class is literally a 3-liner, I'm not sure a language
extension is truly needed:

In [1]: class bunch:
...: def __init__(self,**kw):
...: self.__dict__.update(kw)
...:

In [2]: r=bunch(year=2004,month=11,day=18)

In [3]: r.day,r.month,r.year
Out[3]: (18, 11, 2004)

Cheers,

f

 

[Steven Bethard]

Steven Bethard wrote: [snip]

(18, 11, 2004)

Given that the necessary class is literally a 3-liner, I'm not sure a language
extension is truly needed:

In [1]: class bunch:
...: def __init__(self,**kw):
...: self.__dict__.update(kw)
...:

How do you think I generated the code above?

Even at 3 lines, do you really want to rewrite those every time you need
this functionality? I don't see what would really be wrong with at
least putting this in a stdlib module somewhere (collections perhaps?)

Heck, I can write a set class in only a few more lines:
.... def __init__(self, seq):
.... self._dict = dict.fromkeys(seq)
.... def __iter__(self):
.... return iter(self._dict)
.... def add(self, item):
.... self._dict[item] = None
....

>>> s = set([1, 3, 3, 5, 2, 7, 5])
>>> list(s) [1, 2, 3, 5, 7]
>>> s.add(2)
>>> list(s) [1, 2, 3, 5, 7]
>>> s.add(8)
>>> list(s)

[1, 2, 3, 5, 7, 8]

But I don't think that's a good reason for not having a builtin set class.

The idea of a 'bunch', 'record', 'struct', 'object with attributes',
etc. gets asked for at least a couple times a month. It might be nice
if that functionality was available *somewhere*, whether it be in object
(not likely, I believe) or in a new class, e.g. 'record'.

On the other hand, I usually find that in the few places where I have
used a record like this, I eventually replace the struct with a real
class...

Steve

 

[Fernando Perez]

Steven Bethard wrote: [snip]

r = object(year=2004, month=11, day=18)
r.day, r.month, r.year

(18, 11, 2004)

Given that the necessary class is literally a 3-liner, I'm not sure a
language extension is truly needed:

In [1]: class bunch:
...: def __init__(self,**kw):
...: self.__dict__.update(kw)
...:

How do you think I generated the code above?


On the other hand, I usually find that in the few places where I have
used a record like this, I eventually replace the struct with a real
class...

Yes, that's true. IPython has this fairly fancy Struct module, which is
yet-another-shot at the same thing. It started as the above 3-liner, and
ended up growing into a fairly complex class:

planck[IPython]> wc -l Struct.py
376 Struct.py

Not huge, but certainly more than 3 lines

I guess I was only arguing for the 3-line version being fairly trivial even to
rewrite on the fly as needed. But if someone does add a fully functional
contribution of this kind, with enough bells and whistles to cover the more
advanced cases, I'd be +100 on that

Best,

f

 

[Bengt Richter]

... [...]

Still, as Carlos pointed out, formal parameter names
are private to a function,

(If I misunderstood and misrepresented what Carlos wrote, I apologize)

? No they're not -- a caller knows them, in order to be able to call
with named argument syntax.

I guess you mean keyword arguments, so yes, but for other args I would argue that the caller's
knowledge of formal parameter names really only serves mnemonic purposes.
I.e., once the calling code is written, e.g. an obfuscated-symbols
version of the function may be substituted with no change in program
behavior. So in that sense, there is no coupling: the writer of the
calling code is not forced to use any of the function's internal names
in _his_ source code. This is in contrast with e.g. a returned dict or
name-enhanced tuple: he _is_ forced to use the given actual names in _his_ code,
much like keyword arguments again.

That, incidentally, is why Carlos' decorator suggestion looked good to me. I.e.,
if it could be used to wrap an outside function that returns a plain tuple,
the choice of names would be back in the caller's source code again. Let' see if I can find
it to quote ... not far away...

"""
@returns('year','month','day')
def today():
...
return year, month, day
"""
Applying that (ignoring optimization ;-) to an outside function:

@returns('year','month','day')
def today(*args,**kw):
return outside_function(*args, **kw)

which my some magic would create a today function than returned
a named tuple, with names chosen by the call coder.

Regards,
Bengt Richter

 

[Bengt Richter]

I would *not* like to see this. ISTM that, when reading a function, I
should be able to see where every name in that function came from.
Having global values appear from outside of the function is bad enough;
introducing a new way to magically create variables whose names I can't

^^^^^^
Not create, just rebind existing locals.

see would be (IMO) very, very bad.

All things in moderation. But I agree that you would have to know and trust foo ;-)

Regards,
Bengt Richter