numpy: handling float('NaN') different in XP vs. Linux

[John [H2O]]

I have a script:

from numpy import float
OutD=[]
v=['3','43','23.4','NaN','43']
OutD.append([float(i) for i in v[1]])


On linux:
Python 2.5.1 (r251:54863, Mar 7 2008, 04:10:12)
[GCC 4.1.3 20070929 (prerelease) (Ubuntu 4.1.2-16ubuntu2)] on linux2
[john@andLinux analysis]$ python jnk.py
[[3.0, 43.0, 23.399999999999999, nan, 43.0]]

On XP:
Python 2.5 (r25:51908, Mar 9 2007, 17:40:28) [MSC v.1310 32 bit (Intel)]
Microsoft Windows XP [Version 5.1.2600]
(C) Copyright 1985-2001 Microsoft Corp.

C:\analysis>C:\Python25\python.exe jnk.py
Traceback (most recent call last):
File "jnk.py", line 4, in <module>
OutD.append([float(i) for i in v])
ValueError: invalid literal for float(): NaN


WTF?

 

[Dan Bishop]

I have a script:

from numpy import float
OutD=[]
v=['3','43','23.4','NaN','43']
OutD.append([float(i) for i in v[1]])

On linux:
Python 2.5.1 (r251:54863, Mar  7 2008, 04:10:12)
[GCC 4.1.3 20070929 (prerelease) (Ubuntu 4.1.2-16ubuntu2)] on linux2
[john@andLinux analysis]$ python jnk.py
[[3.0, 43.0, 23.399999999999999, nan, 43.0]]

On XP:
Python 2.5 (r25:51908, Mar  9 2007, 17:40:28) [MSC v.1310 32 bit (Intel)]
Microsoft Windows XP [Version 5.1.2600]
(C) Copyright 1985-2001 Microsoft Corp.

C:\analysis>C:\Python25\python.exe jnk.py
Traceback (most recent call last):
  File "jnk.py", line 4, in <module>
    OutD.append([float(i) for i in v])
ValueError: invalid literal for float(): NaN

Python just uses the atof() function from the underlying C library.
Some of them handle NaN's, and some of them don't.

If you want to get NaN on a platform where float('NaN') doesn't work,
try 1e1000 / 1e1000. Or failing that, struct.unpack('d',
struct.pack('Q', 0xfff8000000000000))[0]

 

[John Machin]

I have a script:

from numpy import float
OutD=[]
v=['3','43','23.4','NaN','43']
OutD.append([float(i) for i in v[1]])

On linux:
Python 2.5.1 (r251:54863, Mar 7 2008, 04:10:12)
[GCC 4.1.3 20070929 (prerelease) (Ubuntu 4.1.2-16ubuntu2)] on linux2
[john@andLinux analysis]$ python jnk.py
[[3.0, 43.0, 23.399999999999999, nan, 43.0]]

On XP:
Python 2.5 (r25:51908, Mar 9 2007, 17:40:28) [MSC v.1310 32 bit (Intel)]
Microsoft Windows XP [Version 5.1.2600]
(C) Copyright 1985-2001 Microsoft Corp.

C:\analysis>C:\Python25\python.exe jnk.py
Traceback (most recent call last):
File "jnk.py", line 4, in <module>
OutD.append([float(i) for i in v])
ValueError: invalid literal for float(): NaN

WTF?

Avoid impolite astonishment; RTFloatingM instead:
"""
float( [x])

Convert a string or a number to floating point. If the argument is a
string, it must contain a possibly signed decimal or floating point
number, possibly embedded in whitespace. Otherwise, the argument may
be a plain or long integer or a floating point number, and a floating
point number with the same value (within Python's floating point
precision) is returned. If no argument is given, returns 0.0.

Note: When passing in a string, values for NaN and Infinity may be
returned, depending on the underlying C library. The specific set of
strings accepted which cause these values to be returned depends
entirely on the C library and is known to vary.
"""

You may like to suggest a minor extension to the docs: after "is known
to vary" add "and may even be empty".

HTH,
John

 

[John [H2O]]

Avoid impolite astonishment; RTFloatingM instead:
"""

HTH,
John

I guess the key here is that it is not an issue with Python, but C... can I
change 'the underlying C code?' if so, WFM should I read for that!?

 

[John Machin]

I guess the key here is that it is not an issue with Python, but C... can I
change 'the underlying C code?'

The underlying C code for the Windows C RTL is probably on a server in
a bunker in Redmond WA ... good luck

Perhaps you could start lashing up something along the lines that Dan
mentioned, e.g.

floated = {
'NaN': 1e1000 / 1e1000,
'Inf': whatever,
}.get

def myfloat(s):
try:
return float(s)
except:
value = floated(s)
if value is not None:
raise
return value

Then when/if your mapping has enough entries to make it worthwhile,
you could maybe suggest that this be done in Numpy or in the Python
core.

Cheers,
John

 

[John [H2O]]

Python just uses the atof() function from the underlying C library.
Some of them handle NaN's, and some of them don't.

As a work around, how would I write this in list comprehension form:

newlist=[]
for i in range(len(v[1])):
try:
newlist.append(float(v[1]))
except:
newlist.append(-999.99) # or just nan possibly?

 

[Robert Kern]

Python just uses the atof() function from the underlying C library.
Some of them handle NaN's, and some of them don't.

As a work around, how would I write this in list comprehension form:

newlist=[]
for i in range(len(v[1])):
try:
newlist.append(float(v[1]))
except:
newlist.append(-999.99) # or just nan possibly?

from numpy import nan

def nanfloat(x):
if x.lower() == 'nan':
return nan
else:
return float(x)

newlist = [myfloat(x) for x in v[1]]

--
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless enigma
that is made terrible by our own mad attempt to interpret it as though it had
an underlying truth."
-- Umberto Eco