Question about 'None'

[flamesrock]

Well, after playing with python for a bit I came across something
weird:

The statement (1 > None) is false (or any other value above 0). Why is
this?


(The reason I ask is sortof unrelated. I wanted to use None as a
variable for which any integer, including negative ones have a greater
value so that I wouldn't need to implement any tests or initializations
for a loop that finds the maximum of a polynomial between certain x
values. Anything is greater than nothing, no?)

 

[Fredrik Lundh]

The statement (1 > None) is false (or any other value above 0). Why is
this?

http://docs.python.org/ref/comparisons.html

"The operators <, >, ==, >=, <=, and != compare the values of
two objects. The objects need not have the same type. If both
are numbers, they are converted to a common type. Otherwise,
objects of different types always compare unequal, and are
ordered consistently but arbitrarily.

(This unusual definition of comparison was used to simplify the
definition of operations like sorting and the in and not in
operators. In the future, the comparison rules for objects of
different types are likely to change.)"

</F>

 

[Steven Bethard]

The statement (1 > None) is false (or any other value above 0). Why is
this?

What code are you executing? I don't get this behavior at all:

py> 100 > None
True
py> 1 > None
True
py> 0 > None
True
py> -1 > None
True
py> -100 > None
True

(The reason I ask is sortof unrelated. I wanted to use None as a
variable for which any integer, including negative ones have a greater
value so that I wouldn't need to implement any tests or initializations
for a loop that finds the maximum of a polynomial between certain x
values. Anything is greater than nothing, no?)

Yup, that's the behavior I get with None.

Steve

 

[Francis Girard]

Le jeudi 27 Janvier 2005 20:16, Steven Bethard a écrit :

What code are you executing? I don't get this behavior at all:

py> 100 > None
True
py> 1 > None
True
py> 0 > None
True
py> -1 > None
True
py> -100 > None
True

Wow ! What is it that are compared ? I think it's the references (i.e. the
adresses) that are compared. The "None" reference may map to the physical 0x0
adress whereas 100 is internally interpreted as an object for which the
reference (i.e. address) exists and therefore greater than 0x0.

Am I interpreting correctly ?

 

[Steven Bethard]

Le jeudi 27 Janvier 2005 20:16, Steven Bethard a écrit :



Wow ! What is it that are compared ? I think it's the references (i.e. the
adresses) that are compared. The "None" reference may map to the physical 0x0
adress whereas 100 is internally interpreted as an object for which the
reference (i.e. address) exists and therefore greater than 0x0.

Am I interpreting correctly ?

Actually, I believe None is special-cased to work like this. From object.c:

static int
default_3way_compare(PyObject *v, PyObject *w)
{
...
if (v->ob_type == w->ob_type) {
...
Py_uintptr_t vv = (Py_uintptr_t)v;
Py_uintptr_t ww = (Py_uintptr_t)w;
return (vv < ww) ? -1 : (vv > ww) ? 1 : 0;
}
...
/* None is smaller than anything */
if (v == Py_None)
return -1;
if (w == Py_None)
return 1;
...
}

So None being smaller than anything (except itself) is hard-coded into
Python's compare routine. My suspicion is that even if/when objects of
different types are no longer comparable by default (as has been
suggested for Python 3.0), None will still compare as smaller than
anything...

Steve

 

[Fredrik Lundh]

So None being smaller than anything (except itself) is hard-coded into Python's compare routine.
My suspicion is that even if/when objects of different types are no longer comparable by default
(as has been suggested for Python 3.0), None will still compare as smaller than anything...

from the change log for Python 2.1:

"An implementation detail changed in 2.1a1 such that None now compares
less than any other object. Code relying on this new behavior (like code
that relied on the previous behavior) does so at its own risk."

the documentation (which I quoted earlier) still applies.

</F>

 

[Francis Girard]

Le jeudi 27 Janvier 2005 21:29, Steven Bethard a écrit :

Actually, I believe None is special-cased to work like this. From
object.c:

static int
default_3way_compare(PyObject *v, PyObject *w)
{
...
if (v->ob_type == w->ob_type) {
...
Py_uintptr_t vv = (Py_uintptr_t)v;
Py_uintptr_t ww = (Py_uintptr_t)w;
return (vv < ww) ? -1 : (vv > ww) ? 1 : 0;
}
...
/* None is smaller than anything */
if (v == Py_None)
return -1;
if (w == Py_None)
return 1;
...
}

So None being smaller than anything (except itself) is hard-coded into
Python's compare routine. My suspicion is that even if/when objects of
different types are no longer comparable by default (as has been
suggested for Python 3.0), None will still compare as smaller than
anything...

Well, here's python doesn't seem to confirm what you're saying :
134536516

It really looks like the addresses are compared when objects are of different
types if there is no __cmp__ or __lt__ user made specification to compare
objects of different types.

If this is case then it is dreadfully dangerous. You end up never really
knowing for sure if you're comparing the references or the values.

What would be the use to compare references anyway in language like Python. I
think the code should raise a big bad exception for such cases.

Francis Girard

 

[Fredrik Lundh]

Wow ! What is it that are compared ? I think it's the references (i.e. the
adresses) that are compared. The "None" reference may map to the physical 0x0
adress whereas 100 is internally interpreted as an object for which the
reference (i.e. address) exists and therefore greater than 0x0.

Am I interpreting correctly ?

your hypothesis might match the observations, but it doesn't match the
implementation. nor the documentation:

" ... objects of different types always compare unequal, and are
ordered consistently but arbitrarily."

(see my others posts in this thread for more details)

</F>

 

[Francis Girard]

Oops, I misunderstood what you said. I understood that it was now the case
that objects of different types are not comparable by default whereas you
meant it as a planned feature for version 3.

I really hope that it will indeed be the case for version 3.

Francis Girard

Le jeudi 27 Janvier 2005 21:29, Steven Bethard a écrit :

 

[Francis Girard]

Le vendredi 28 Janvier 2005 07:49, jfj a écrit :

Not really. If objects of different types are compared (like compare a
string with a list), then no matter what, all strings but be "smaller"
than all lists. Or the oposite.

So the fast way to accomplish this is to compare the addresses of the
object's method table. Something which is common for all objects of
the same type.


G.

I see. There is some rule stating that all the strings are greater than ints
and smaller than lists, etc.

What was the goal behind this rule ?

Francis Girard

 

[Steven Bethard]

Le jeudi 27 Janvier 2005 21:29, Steven Bethard a écrit :

Well, here's python doesn't seem to confirm what you're saying :



134536516

Actually, this code doesn't contradict what I said at all. I said that
None (and only None) was special-cased.

It really looks like the addresses are compared when objects are of different
types if there is no __cmp__ or __lt__ user made specification to compare
objects of different types.

Use the source Luke! Download it from CVS and take a look. The end
of default_3way_compare:

static int
default_3way_compare(PyObject *v, PyObject *w)
{
...
/* None is smaller than anything */
if (v == Py_None)
return -1;
if (w == Py_None)
return 1;

/* different type: compare type names; numbers are smaller */
if (PyNumber_Check(v))
vname = "";
else
vname = v->ob_type->tp_name;
if (PyNumber_Check(w))
wname = "";
else
wname = w->ob_type->tp_name;
c = strcmp(vname, wname);
if (c < 0)
return -1;
if (c > 0)
return 1;
/* Same type name, or (more likely) incomparable numeric types */
return ((Py_uintptr_t)(v->ob_type) < (
Py_uintptr_t)(w->ob_type)) ? -1 : 1;
}

So it looks like Python uses the type name. Testing this:

py> A = type('A', (object,), {})
py> Z = type('Z', (object,), {})
py> lst = [str('a'), dict(b=2), tuple(), Z(), A()]
py> [type(o).__name__ for o in lst]
['str', 'dict', 'tuple', 'Z', 'A']
py> sorted(type(o).__name__ for o in lst)
['A', 'Z', 'dict', 'str', 'tuple']
py> sorted(lst)
[<__main__.A object at 0x011E29D0>, <__main__.Z object at 0x011E29F0>,
{'b': 2}, 'a', ()]

Yup. Looks about right. (Note that the source code also special-cases
numbers...)

So, the order is consistent, but arbitrary, just as Fredrik Lundh
pointed out in the documentation.

Steve

 

[Steven Bethard]

I see. There is some rule stating that all the strings are greater than ints
and smaller than lists, etc.

Yes, that rule being to compare objects of different types by their type
names (falling back to the address of the type object if the type names
are the same, I believe). Of course, this is arbitrary, and Python does
not guarantee you this ordering -- it would not raise backwards
compatibility concerns to, say, change the ordering in Python 2.5.

What was the goal behind this rule ?

I believe at the time, people thought that comparison should be defined
for all Python objects. Guido has since said that he wishes the
decision hadn't been made this way, and has suggested that in Python
3.0, objects of unequal types will not have a default comparison.

Probably this means ripping the end off of default_3way_compare and
raising an exception. As Fredrik Lundh pointed out, they could, if they
wanted to, also rip out the code that special-cases None too.

Steve

 

[Francis Girard]

Le jeudi 27 Janvier 2005 22:07, Steven Bethard a écrit :

Actually, this code doesn't contradict what I said at all. I said that
None (and only None) was special-cased.

You're right. I just posted too soon.
Sorry.

Use the source Luke! Download it from CVS and take a look. The end
of default_3way_compare:

Just want to know the specifications. No matter how it had been implemented.

static int
default_3way_compare(PyObject *v, PyObject *w)
{
...
/* None is smaller than anything */
if (v == Py_None)
return -1;
if (w == Py_None)
return 1;

/* different type: compare type names; numbers are smaller */
if (PyNumber_Check(v))
vname = "";
else
vname = v->ob_type->tp_name;
if (PyNumber_Check(w))
wname = "";
else
wname = w->ob_type->tp_name;
c = strcmp(vname, wname);
if (c < 0)
return -1;
if (c > 0)
return 1;
/* Same type name, or (more likely) incomparable numeric types */
return ((Py_uintptr_t)(v->ob_type) < (
Py_uintptr_t)(w->ob_type)) ? -1 : 1;
}

So it looks like Python uses the type name. Testing this:

py> A = type('A', (object,), {})
py> Z = type('Z', (object,), {})
py> lst = [str('a'), dict(b=2), tuple(), Z(), A()]
py> [type(o).__name__ for o in lst]
['str', 'dict', 'tuple', 'Z', 'A']
py> sorted(type(o).__name__ for o in lst)
['A', 'Z', 'dict', 'str', 'tuple']
py> sorted(lst)
[<__main__.A object at 0x011E29D0>, <__main__.Z object at 0x011E29F0>,
{'b': 2}, 'a', ()]

Yup. Looks about right. (Note that the source code also special-cases
numbers...)

So, the order is consistent, but arbitrary, just as Fredrik Lundh
pointed out in the documentation.

Ok. Thank you,

Francis Girard

 

[Francis Girard]

Very complete explanation.
Thank you

Francis Girard

Le jeudi 27 Janvier 2005 22:15, Steven Bethard a écrit :

 

[Steven Bethard]

134536516

Just to thoroughly explain this example, the current CPython
implementation says that numbers are smaller than everything but None.
The reason you get such a small id for 'b' is that there is only one 10
object (for efficiency reasons):

py> 10 is 5 + 5
True
py> 99 is 50 + 49
True
py> 100 is 50 + 50
False

Note that there may be many different instances of integers 100 and
above (again, in the current CPython implementation). So to get an
integer id above a string id, your integer must be at least 100:

py> a = "100"
py> b = 100
py> id(a), id(b)
(18755392, 18925912)
py> a > b
True
py> b > a
False

So, even though b's id is higher than a's, b still compares as smaller
because the current CPython implementation special-cases the comparison
to guarantee that numbers are always smaller than all other non-None
objects.

Again, these are *all* implementation details of the current CPython,
and depending on these details might run you into troubles if they
change in a future version of CPython, or if you use a different
implementation (e.g. Jython or IronPython).

Steve

 

[flamesrock]

Wow! Thanks for all the replies. I'll have to reread everything to
absorb it.

The test code is pretty simple:
if 2 > None:
print 'true'
else:
print 'false'

It prints false every time. I should also mention that I'm using
version 2.0.1 (schools retro solaris machines )
At home (version 2.3.4) it prints out 'True' for the above code block.

 

[Jeff Shannon]

I should also mention that I'm using
version 2.0.1 (schools retro solaris machines )
At home (version 2.3.4) it prints out 'True' for the above code block.

That would explain it -- as /F mentioned previously, the special case
for None was added in 2.1.

Jeff Shannon
Technician/Programmer
Credit International

 

[jfj]

Wow ! What is it that are compared ? I think it's the references (i.e. the
adresses) that are compared. The "None" reference may map to the physical 0x0
adress whereas 100 is internally interpreted as an object for which the
reference (i.e. address) exists and therefore greater than 0x0.

Am I interpreting correctly ?

Not really. If objects of different types are compared (like compare a
string with a list), then no matter what, all strings but be "smaller"
than all lists. Or the oposite.

So the fast way to accomplish this is to compare the addresses of the
object's method table. Something which is common for all objects of
the same type.


G.

 

[Alex Martelli]

(The reason I ask is sortof unrelated. I wanted to use None as a
variable for which any integer, including negative ones have a greater
value so that I wouldn't need to implement any tests or initializations
for a loop that finds the maximum of a polynomial between certain x
values. Anything is greater than nothing, no?)

To have this work reliably across versions of Python (and you mentioned
you need it to work on 2.0 as well as 2.3...) you have to make your own
sentinel class, such as:

class MinusInfinity:
def __cmp__(self, other):
if isinstance(other, MinusInfinity): return 0
else: return -1
minusInfinity = MinusInfinity()

Now, you should be able to use this 'minusInfinity" as ``a variable for
which any integer ... has a greater value'', as you wished to use None.

(Untested code, as I don't have any 2.0 on which to test it anyway).


Alex

 

[Francis Girard]

Le vendredi 28 Janvier 2005 22:54, jfj a écrit :

If you have a list which contains integers, strings, tuples, lists and
dicts and you sort it and print it, it will be easier to detect what
you're looking for


G.

Mmm. Certainly not one of my top requirements.

Anyway, it would be better to separately provide a compare function that
orders elements according to their types instead of making the comparison
operators default semantics somewhat obscure (and dangerous).

Thank you
Francis Girard