Why do only callable objects get a __name__?

[John Ladasky]

A few days ago, I asked about getting the original declared name of a function or method, and learned about the __name__ attribute.

https://groups.google.com/forum/#!topic/comp.lang.python/bHvcuXgvdfA

Of course, I have used __name__ for years in the common expression "if __name__ == "__main__") to determine whether a particular module is being run or merely imported. But until recently, I never went deeper than that.

I just created an object using collections.namedtuple, and was surprised todiscover that it didn't have a __name__ -- even though something that behaves like __name__ is clearly accessible and printable. Here's some minimalPython 3.3.2 code and output:

=================================================

from collections import namedtuple

MyNamedTupleClass = namedtuple("ANamedTuple", ("foo", "bar"))
nt = MyNamedTupleClass(1,2)
print(nt)
print(type(nt))
# print(nt.__name__) # this would raise an AttributeError
print(type(nt).__name__) # this is the desired output

=================================================

ANamedTuple(foo=1, bar=2)
<class '__main__.ANamedTuple'>
ANamedTuple

=================================================

As you can see, I snooped around in the object's type. I found that the type, rather than the object itself, had the __name__ I was seeking. I then read the Python docs concerning __name__ and found that this attribute is restricted to callable objects.

This leads me to ask two questions:

1. WHY do only callable objects get a __name__? A __name__ would seem to be a useful feature for other types. Clearly, whoever implemented namedtuple thought it was useful to retain and display that information as a part ofthe string representation of the namedtuple (and I agree).

2. If I created a superclass of namedtuple which exposed type(namedtuple).__name__ in the namespace of the namedtuple itself, would I be doing anything harmful?

Thanks as always for your insights.

 

[John Ladasky]

2. If I created a superclass of namedtuple which exposed type(namedtuple).__name__ in the namespace of the namedtuple itself, would I be doing anything harmful?

Sigh. Of course, that should read "subclass", not "superclass." Because I was thinking of a type being, in a sense, "higher up" the object hierarchy than an instance, I accidentally reached for the wrong word.

 

[Ian Kelly]

A few days ago, I asked about getting the original declared name of a function or method, and learned about the __name__ attribute.

https://groups.google.com/forum/#!topic/comp.lang.python/bHvcuXgvdfA

Of course, I have used __name__ for years in the common expression "if __name__ == "__main__") to determine whether a particular module is beingrun or merely imported. But until recently, I never went deeper than that..

I just created an object using collections.namedtuple, and was surprised to discover that it didn't have a __name__ -- even though something that behaves like __name__ is clearly accessible and printable. Here's some minimal Python 3.3.2 code and output:

=================================================

from collections import namedtuple

MyNamedTupleClass = namedtuple("ANamedTuple", ("foo", "bar"))
nt = MyNamedTupleClass(1,2)
print(nt)
print(type(nt))
# print(nt.__name__) # this would raise an AttributeError
print(type(nt).__name__) # this is the desired output

=================================================

ANamedTuple(foo=1, bar=2)
<class '__main__.ANamedTuple'>
ANamedTuple

=================================================

As you can see, I snooped around in the object's type. I found that the type, rather than the object itself, had the __name__ I was seeking. I then read the Python docs concerning __name__ and found that this attribute isrestricted to callable objects.

This leads me to ask two questions:

1. WHY do only callable objects get a __name__? A __name__ would seem tobe a useful feature for other types. Clearly, whoever implemented namedtuple thought it was useful to retain and display that information as a part of the string representation of the namedtuple (and I agree).

Classes and functions are frequently kept in module namespaces, where
they are known by a specific name. The intent is that the __name__
attribute should match that name by which it is commonly referred.

Specific instances are not typically widely referred to by set names
in this way. They are more commonly stored in variables that are used
to hold a wide variety of objects.

In the namedtuple example that you give, it seems that you would want
the names of all instances of the ANamedTuple class to be the same
"ANamedTuple", and I really don't see what the purpose of giving them
all the same name would be.

2. If I created a superclass of namedtuple which exposed type(namedtuple)..__name__ in the namespace of the namedtuple itself, would I be doing anything harmful?

Probably not. But why not just invent your own name attribute rather
than shadow the one that Python designates for classes and functions?

 

[John Ladasky]

Classes and functions are frequently kept in module namespaces, where
they are known by a specific name. The intent is that the __name__
attribute should match that name by which it is commonly referred.



Specific instances are not typically widely referred to by set names

in this way. They are more commonly stored in variables that are used

to hold a wide variety of objects.



In the namedtuple example that you give, it seems that you would want
the names of all instances of the ANamedTuple class to be the same
"ANamedTuple", and I really don't see what the purpose of giving them
all the same name would be.

I am implementing a state machine. The outputs of the various states in the machine have variable contents. I started by making dictionaries for each state output, but I soon tired of the bracket-and-quote-mark syntax for
referring to the contents of these state output dictionaries. That's why I am switching to named tuples. It doesn't affect the __name__ issue, since dictionaries also cannot be called.

I want to capture the names of the executed states in a record of the state machine's history. This information is already encoded in the namedtuple's type.

Probably not.

I just thought I would ask.

But why not just invent your own name attribute rather
than shadow the one that Python designates for classes and functions?

I will never write to this attribute, only read it. And since the information I want is already there (albeit in a strange place), I am inclined to use it.

 

[Terry Reedy]

Of course, I have used __name__ for years in the common expression "if __name__ == "__main__") to determine whether a particular module is being run or merely imported.

This true statement invalidates your subject line ;-). All modules have
a __name__. The main module has the name (__name__) '__main__'. (A file
named '__main__.py' also has special meaning. If one does 'python -m
package' on a command line and 'package' is a directory with
'__init__.py', 'package/__main__.py' is executed as the main module
'__main__'.

1. WHY do only callable objects get a __name__?

Why do you think this? Is there a mistake in the doc?

 

[John Ladasky]

This true statement invalidates your subject line ;-). All modules have
a __name__.

Yes, I thought about this before I posted. I figured that, if I investigated further I would discover that there was a __main__ function that was being called.

Why do you think this? Is there a mistake in the doc?

Quote below from http://docs.python.org/3/reference/datamodel.html:

=======================================================================

Callable types
These are the types to which the function call operation (see section Calls) can be applied:

User-defined functions
A user-defined function object is created by a function definition (see section Function definitions). It should be called with an argument list containing the same number of items as the function’s formal parameter list.

Special attributes:

Attribute Meaning
__name__ The function’s name Writable

=======================================================================

Perhaps I'm inferring too much from slightly-vague documentation, and my recent experience with objects that cannot be called? Or perhaps the information that I need to read is somewhere in the documentation other than whereI have looked?

Still puzzling over this... I can easily hack a solution as Terry suggested, but it's not elegant, and that kind of thing bugs me.

 

[Steven D'Aprano]

I just created an object using collections.namedtuple, and was surprised
to discover that it didn't have a __name__

I'm not sure why you're surprised. Most objects don't have names,
including regular tuples:

py> some_tuple = (23, 42)
py> some_tuple.__name__
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
AttributeError: 'tuple' object has no attribute '__name__'



Remember, Python has two distinct concepts of names: the name that an
object knows itself by, and the variable name that it is bound to.
Objects may be bound to zero, one, or more variable names. Here are some
examples of the later:

print 42 # zero name
x = 42 # one name
x = y = z = 42 # three names

It simply isn't practical or meaningful to go backwards from the object
42 to the variable name(s) it is bound to -- in the first example, there
is no variable name at all; in the third, there are three. Even if you
wanted to do it, it would be a performance killer. So if you have any
thought that "the name of an object" should be the name of the variable,
scrub that from your head, it will never fly.

That leaves us with the name that objects know themselves by. For the
rest of this post, any time I talk about a name, I always mean the name
an object knows itself by, and never the variable name it is bound to (if
there is such a variable name).

As a general rule, names aren't meaningful or useful for objects. To
start with, how would you give it a name, what syntax would you use? What
would you expect these examples to print?

import random
random.random().__name__

data = [23, 17, 99, 42]
print data[1].__name__


In general, objects are *anonymous* -- they have no inherent name.
Instances come into being in all sorts of ways, they live, they die,
they're garbage-collected by the compiler. They have no need for
individual names, and no way to be given one.

But if you insist on giving them one, you can give it a go. But I
guarantee that (1) you'll find it a lot less useful, and (2) a lot more
inconvenient than you expected:

class NamedList(list):
def __new__(cls, name, *args):
obj = super(NamedList, cls).__new__(cls, *args)
obj.__name__ = name
return obj
def __init__(self, name, *args):
super(NamedList, self).__init__(*args)

py> print NamedList("fred", [1, 2, 3, 4, 5]).__name__
fred


So it works, but what a drag, and you don't get much for the effort.

The three obvious exceptions are:

- modules
- classes/types
- functions/methods

In the first case, modules, there is an obvious interpretation of what
the name ought to be: the filename minus the extension. Since the module
knows where it came from, it can know it's own name:

py> import re
py> re.__name__
're'


Even if you bind the module object to a different variable name, it knows
its own inherent name:

py> import math as fibble
py> print fibble.__name__
math


For functions and classes, such names are especially useful, for
debugging and error messages:

py> def one_over(x):
.... return 1.0/x
....
py> one_over(0)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "<stdin>", line 2, in one_over
ZeroDivisionError: float division by zero


Notice the second last line, where it gives the function's name? That
would be impossible if callables (functions, classes) didn't know their
own name. You'd get something like:

File "<stdin>", line 2, in <function object at 0xb7ea7a04>

which would be useless for debugging. So how fortunately that there is
obvious and simple syntax for setting the name of functions and classes:

class This_Is_The_Class_Name:
def this_is_the_function_name(self):
...

-- even though something that
behaves like __name__ is clearly accessible and printable. Here's some
minimal Python 3.3.2 code and output:

=================================================

from collections import namedtuple

MyNamedTupleClass = namedtuple("ANamedTuple", ("foo", "bar"))

Here you define a class, called "ANamedTuple". Unfortunately, it doesn't
use the standard class syntax, a minor limitation and annoyance of
namedtuples, and so you're forced to give the class name "ANamedTuple"
explicitly as an argument to the function call. But the important thing
here is that it is a class.

nt = MyNamedTupleClass(1,2)

nt, on the other hand, is a specific instance of that class. You might
have hundreds, thousands, millions of such instances. Why would you want
to name them all? What point of doing so is there? They have no need, and
no benefit, to be given individual names. And consequent, when you ask
the instance "What's your name?", they respond "I don't have one":

# print(nt.__name__) # this would raise an AttributeError

However, when you ask the class ANamedTuple what its name it, it has a
name, and can tell you:

print(type(nt).__name__) # this is the desired output

If the instance nt claimed to be ANamedTuple, it would be *lying*. It
isn't the class, and it shouldn't claim to have the same name as the
class.


[...]

As you can see, I snooped around in the object's type. I found that the
type, rather than the object itself, had the __name__ I was seeking.

Yes. And why do you consider this to be a problem?

1. WHY do only callable objects get a __name__? A __name__ would seem
to be a useful feature for other types. Clearly, whoever implemented
namedtuple thought it was useful to retain and display that information
as a part of the string representation of the namedtuple (and I agree).

Because the namedtuple that you create *is* a class. (Type and class are,
to a first approximation, synonyms.) Sadly, when you create a class using
function-call syntax rather than the class keyword, you have to manually
specify the name twice:

SomeTuple = namedtuple("SomeTuple", ...)


but that makes sense when you consider that Python has two concepts of
names. The first SomeTuple, on the left of the equals sign, is the
variable name. The second, inside the parentheses, is the class name.
They need not be the same.

2. If I created a superclass of namedtuple which exposed
type(namedtuple).__name__ in the namespace of the namedtuple itself,
would I be doing anything harmful?

Harmful? No, except perhaps helping muddy the waters between classes
(which have names) and instances (which generally don't). But why would
you bother? The right way to handle this is, when you want the name of
the type, ask for the name of the type:

type(instance).__name__

 

[Steven D'Aprano]

I am implementing a state machine. The outputs of the various states in
the machine have variable contents. I started by making dictionaries
for each state output, but I soon tired of the bracket-and-quote-mark
syntax for referring to the contents of these state output dictionaries.
That's why I am switching to named tuples. It doesn't affect the
__name__ issue, since dictionaries also cannot be called.

I want to capture the names of the executed states in a record of the
state machine's history. This information is already encoded in the
namedtuple's type.

I find this rather confusing. Does every state have it's own unique
namedtuple class?


state1 = namedtuple("State1", ("x", "y"))(1, 2)
state2 = namedtuple("State2", ("x", "y"))(5, 7)
state3 = namedtuple("State3", ("x", "y"))(0, 2)
[...]

Seems excessive -- you're defining many classes, each one of which has
only a single instance. I'd be more inclined to just use a class object
directly, with a little helper function to handle the tedious bits:

def state(name, x, y):
class Inner(object):
pass
Inner.x = x
Inner.y = y
Inner.__name__ = Inner.name = name
return Inner


And in use:

py> state1 = state('state1', 100, 101)
py> state1
<class '__main__.state1'>
py> state1.name
'state1'
py> state1.x
100


But I don't really understand how you are using these state objects, so
I'm not sure if this is appropriate or not.

 

[John Ladasky]

I find this rather confusing. Does every state have it's own unique
namedtuple class?

Yes. The state machine trains neural networks. There are various states which are executed in various orders -- generating a new network variant, calculating a local error gradient, taking a (variable size and curvature sensitive) step down the gradient, and then, possibly repeating.

I want to watch this neural network training process run, in part to debug it. At the end of each state execution, I use a SINGLE, all-purpose function hook in the main loop to print the result. Every state result namedtuple is also saved in a list called "history". I sometimes need information from the first state executed, or the most recent state, or even two states back, to choose the next state transition.

At some point in the future, I can override the print function in the main loop, and have a GUI intercept the output as it is generated.

Many of the parameters returned are common to all states, but quite a few are not. By stepping through the namedtuple's "_fields" and picking out a few special cases, I can get output like this:

=================================================

== START ==

action : START
time : 0.203123
net :
0. 0. 0.
0. 0. 0. 0.
full_err : 0.766030
train_err : 0.780680
validate_err : 0.750036
next_state : VARIANT


== VARIANT ==

action : VARIANT
time : 0.000058
net :
1.0356 -0.1986 -1.7067
-1.0545 1.7081 0.4573 -1.0968
next_state : GRADIENT


== GRADIENT ==

action : GRADIENT
time : 0.094384
net :
1.0356 -0.1986 -1.7067
-1.0545 1.7081 0.4573 -1.0968
grad :
[[-0.0047 -0.0034 -0.0001 0. ]
[ 0.01 0.0069 -0.003 0.0085]]
train_err : 1.226932
incrmag : 0.001000
normag : 0.010000
next_state : LEAP


== LEAP ==

action : LEAP
time : 0.048523
net :
0.9882 -0.2328 -1.7079
-0.9545 1.7774 0.4271 -1.0115
curvature : concave
leap : 10.000000
train_err : 1.118779
incrmag : 0.001000
normag : 0.010000
next_state : GRADIENT


== GRADIENT ==

action : GRADIENT
time : 0.092888
net :
0.9882 -0.2328 -1.7079
-0.9545 1.7774 0.4271 -1.0115
grad :
[[-0.0037 -0.0027 0.0004 0. ]
[ 0.01 0.0068 -0.0043 0.0092]]
train_err : 1.118779
incrmag : 0.001000
normag : 0.010000
next_state : LEAP

[etc.]

=================================================

I'm probably making this sound more complicated than it is. A good object and data structure will make my system run cleanly and flexibly. I feel like I'm already half-way there.

 

[John Ladasky]

Thanks for your replies, Steven. Between this post and your other post, you wrote a lot.

So if you have any
thought that "the name of an object" should be the name of the variable,
scrub that from your head, it will never fly.

I certainly don't, as you would see from the title of my thread from two days ago, which lead to this thread: "Obtaining 'the' name of a function/method".

https://groups.google.com/forum/#!topic/comp.lang.python/bHvcuXgvdfA

It took me a few months to understand the Pythonic concept of binding namesto anonymous objects when I first started with Python... but that was quite a few years ago.

For the
rest of this post, any time I talk about a name, I always mean the name
an object knows itself by, and never the variable name it is bound to (if
there is such a variable name).
[snip]

For functions and classes, such names are especially useful, for
debugging and error messages:

If you read my response to your other post, you will see that debugging is definitely one of the reasons I'm undertaking this approach. But even after the debugging is complete, I will still find it helpful for logging and monitoring purposes.

Here you define a class, called "ANamedTuple". Unfortunately, it doesn't
use the standard class syntax, a minor limitation and annoyance of
namedtuples, and so you're forced to give the class name "ANamedTuple"
explicitly as an argument to the function call. But the important thing
here is that it is a class.

OK, that helps. I just had a look at the namedtuple source code. Part of my conceptual problem stems from the fact that namedtuple() is what I thinkpeople call a "class factory" function, rather than a proper class constructor. I'll read through this until I understand it.

 

[Steven D'Aprano]

I just had a look at the namedtuple source code. Part of my conceptual
problem stems from the fact that namedtuple() is what I think people
call a "class factory" function, rather than a proper class constructor.
I'll read through this until I understand it.

That's right. Since it's a factory, you don't have advantage of the class
syntax -- although that syntax is only syntactic sugar for something
which actually ends up as a function call!

When you write:

class MyClass(ParentClass, OtherClass):
a = 23
def method(self, arg):
code goes here

the interpreter converts that to a function call:


MyClass = type("MyClass", (ParentClass, OtherClass), ns)

where ns is a dict containing:

{'a': 23, 'method': function-object, ...}

and one or two other things automatically inserted for you. A factory
function, being just a function, can't take advantage of any magic
syntax. While you can put anything you like inside the function, the
function can't see what's on the left hand side of the assignment to
retrieve the name, so you have to manually provide it.