Newbie question

[Tommy Grav]

Hi list,

this is somewhat of a newbie question that has irritated me for a
while.
I have a file test.txt:

0.3434 0.5322 0.3345
1.3435 2.3345 5.3433

and this script
lines = open("test.txt","r").readlines()
for line in lines:
(xin,yin,zin) = line.split()
x = float(xin)
y = float(yin)
z = float(zin)

Is there a way to go from line.split() to x,y,z as floats without
converting
each variable individually?

Cheers
Tommy

 

[Stargaming]

Hi list,

this is somewhat of a newbie question that has irritated me for a
while.
I have a file test.txt:

0.3434 0.5322 0.3345
1.3435 2.3345 5.3433

and this script
lines = open("test.txt","r").readlines()
for line in lines:
(xin,yin,zin) = line.split()
x = float(xin)
y = float(yin)
z = float(zin)

Is there a way to go from line.split() to x,y,z as floats without
converting
each variable individually?

Cheers
Tommy

For this case, there are list comprehensions (or map, but you shouldn't
use it any longer):

>>> a = "0.3434 0.5322 0.3345"
>>> b = a.split()
>>> map(float, b) [0.34339999999999998, 0.53220000000000001, 0.33450000000000002]
>>> [float(x) for x in b]

[0.34339999999999998, 0.53220000000000001, 0.33450000000000002]

I think it should be easy to apply this to your example above.

Stargaming

 

[Larry Bates]

Hi list,

this is somewhat of a newbie question that has irritated me for a while.
I have a file test.txt:

0.3434 0.5322 0.3345
1.3435 2.3345 5.3433

and this script
lines = open("test.txt","r").readlines()
for line in lines:
(xin,yin,zin) = line.split()
x = float(xin)
y = float(yin)
z = float(zin)

Is there a way to go from line.split() to x,y,z as floats without
converting
each variable individually?

Cheers
Tommy

Using a list comprehension you would write this as:

for line in lines:
xin, yin, zin=[float(x) for x in line.split()]

This if course expects your data to be perfect. If
you want error handling (e.g. less or more than 3 values,
values that cause exception when passed to float, etc.)
you will have to handle that differently.

-Larry

 

[Larry Bates]

Hi list,

this is somewhat of a newbie question that has irritated me for a while.
I have a file test.txt:

0.3434 0.5322 0.3345
1.3435 2.3345 5.3433

and this script
lines = open("test.txt","r").readlines()
for line in lines:
(xin,yin,zin) = line.split()
x = float(xin)
y = float(yin)
z = float(zin)

Is there a way to go from line.split() to x,y,z as floats without
converting
each variable individually?

Cheers
Tommy

Using a list comprehension you would write this as:

for line in lines:
xin, yin, zin=[float(x) for x in line.split()]

This if course expects your data to be perfect. If
you want error handling (e.g. less or more than 3 values,
values that cause exception when passed to float, etc.)
you will have to handle that differently.

-Larry

 

[Joshua.R.English]

Tommy Grav schrieb:

For this case, there are list comprehensions (or map, but you shouldn't
use it any longer):

I didn't see anything in the docs about this. Is map going away or is
it considered un-Pythonic now?

Josh

 

[Bruno Desthuilliers]

Tommy Grav a écrit :

Hi list,

this is somewhat of a newbie question that has irritated me for a
while.
I have a file test.txt:

0.3434 0.5322 0.3345
1.3435 2.3345 5.3433

and this script
lines = open("test.txt","r").readlines()
for line in lines:
(xin,yin,zin) = line.split()
x = float(xin)
y = float(yin)
z = float(zin)

Is there a way to go from line.split() to x,y,z as floats without
converting
each variable individually?

either map() or a list comprehension

import sys
fname = "test.txt"
lines = open(fname,"r")
for numline, line in enumerate(lines):
try:
x, y, z = map(float, line.split())
# or:
# x, y, z = [float(item) for item in line.split()]
except (ValueError, IndexError), e:
err = "Invalid data format in %s line %s : "%s" (%s)" \
% (fname, numline, line, e)
print >> sys.stderr, err
sys.exit(1)
else:
# do whatever appropriate here - anyway

HTH

 

[Bruno Desthuilliers]

(e-mail address removed) a écrit :

I didn't see anything in the docs about this. Is map going away or is
it considered un-Pythonic now?

It's less used since we have list comps.

 

[Ayaz Ahmed Khan]

"(e-mail address removed)" typed:

I didn't see anything in the docs about this. Is map going away or is
it considered un-Pythonic now?

Most everywhere I've read about map() and filter() seemed to
discourage their use stating that they're becoming depreciated (with the
exception of Dive Into Python which advocates use of these two functions
in preference to even list comprehensions, if I've read it properly).

--
Ayaz Ahmed Khan

Falling in love makes smoking pot all day look like the ultimate in
restraint.
-- Dave Sim, author of "Cerebus".

 

[Gabriel Genellina]

"(e-mail address removed)" typed:

Most everywhere I've read about map() and filter() seemed to
discourage their use stating that they're becoming depreciated (with the
exception of Dive Into Python which advocates use of these two functions
in preference to even list comprehensions, if I've read it properly).

reduce() will disappear on Python 3.0; map and filter may stay.
See http://www.python.org/dev/peps/pep-3100/ and
http://www.artima.com/weblogs/viewpost.jsp?thread=98196

 

[Tommy Grav]

Hi list,

this is somewhat of a newbie question that has irritated me for
a while.
I have a file test.txt:

0.3434 0.5322 0.3345
1.3435 2.3345 5.3433

and this script
lines = open("test.txt","r").readlines()
for line in lines:
(xin,yin,zin) = line.split()
x = float(xin)
y = float(yin)
z = float(zin)

Is there a way to go from line.split() to x,y,z as floats without
converting
each variable individually?

Cheers
Tommy

Using a list comprehension you would write this as:

for line in lines:
xin, yin, zin=[float(x) for x in line.split()]

This if course expects your data to be perfect. If
you want error handling (e.g. less or more than 3 values,
values that cause exception when passed to float, etc.)
you will have to handle that differently.

Thanks for the great response.

So how would you handle this type of error handling?
My main problem is that occasionally there is an entry
in the list that is a string:

0.9834 134.4933 78.009 run11 27

Again I would like to avoid having to individually parse the 3 floats,
while still easily handling the string, but a list comprehension will
not work as far as I can tell. Is there a module that handles this
type of flat ascii tables? Something like:

(x,y,z,id,n) = ParseFile("float","float","float","string","int")

would be great, and I guess

(x,y,z,id,n) = PaseFile2("%f %f %f %s %d")

would be even better.

Cheers
Tommy

 

[Steven D'Aprano]

So how would you handle this type of error handling?
My main problem is that occasionally there is an entry
in the list that is a string:

0.9834 134.4933 78.009 run11 27

How do you want to deal with such an occasional string? What do you expect
the code to do?

The way I see it, if you're expecting five floats, and you get four floats
and a string, that's an error. Do you ignore the entire line, or replace
the string with zero, or what?

Again I would like to avoid having to individually parse the 3 floats,
while still easily handling the string, but a list comprehension will
not work as far as I can tell. Is there a module that handles this
type of flat ascii tables? Something like:

(x,y,z,id,n) = ParseFile("float","float","float","string","int")

would be great, and I guess

(x,y,z,id,n) = PaseFile2("%f %f %f %s %d")

would be even better.

How about this?

def parse_line(line, types):
items = line.split()
return [t(s) for (t, s) in zip(types, items)]

[4.5, 6.7000000000000002, 23, -1, 'fred', 7.9000000000000004]

 

[Jussi Salmela]

Tommy Grav kirjoitti:

<snip>
> Thanks for the great response.

So how would you handle this type of error handling?
My main problem is that occasionally there is an entry
in the list that is a string:

0.9834 134.4933 78.009 run11 27

Again I would like to avoid having to individually parse the 3 floats,
while still easily handling the string, but a list comprehension will
not work as far as I can tell. Is there a module that handles this
type of flat ascii tables? Something like:

(x,y,z,id,n) = ParseFile("float","float","float","string","int")

would be great, and I guess

(x,y,z,id,n) = PaseFile2("%f %f %f %s %d")

would be even better.

Cheers
Tommy

Being just on the first step of the ladder of total clairvoyance, I
can't disambiguate the word "occasionally". I'm assuming that you have
two types of rows: the ones with 3 floats and the ones with 3 floats, a
string and an integer.

Here's a possible solution:

#=======================================
conv = {'f':float, 'i':int, 's':str}
lines = ['0.3434 0.5322 0.3345\n',
'0.9834 134.4933 78.009 run11 27\n',
'1.3435 2.3345 5.3433\n']
for line in lines:
line = line.split()
if len(line) == 3:
pat = 'fff'
(x, y, z) = [conv[p](x) for p,x in zip(pat, line)]
print x, y, z
print type(x), type(y), type(z)
else:
pat = 'fffsi'
(x, y, z, s, i) = [conv[p](x) for p,x in zip(pat, line)]
print x, y, z, s, i
print type(x), type(y), type(z), type(s), type(i)
#=======================================


HTH,
Jussi

 

[Tommy Grav]

Tommy Grav kirjoitti:

<snip>
Thanks for the great response.

So how would you handle this type of error handling?
My main problem is that occasionally there is an entry
in the list that is a string:

0.9834 134.4933 78.009 run11 27

Again I would like to avoid having to individually parse the 3
floats,
while still easily handling the string, but a list comprehension will
not work as far as I can tell. Is there a module that handles this
type of flat ascii tables? Something like:

(x,y,z,id,n) = ParseFile("float","float","float","string","int")

would be great, and I guess

(x,y,z,id,n) = PaseFile2("%f %f %f %s %d")

would be even better.

Cheers
Tommy

Being just on the first step of the ladder of total clairvoyance, I
can't disambiguate the word "occasionally". I'm assuming that you have
two types of rows: the ones with 3 floats and the ones with 3
floats, a
string and an integer.

Here's a possible solution:

#=======================================
conv = {'f':float, 'i':int, 's':str}
lines = ['0.3434 0.5322 0.3345\n',
'0.9834 134.4933 78.009 run11 27\n',
'1.3435 2.3345 5.3433\n']
for line in lines:
line = line.split()
if len(line) == 3:
pat = 'fff'
(x, y, z) = [conv[p](x) for p,x in zip(pat, line)]
print x, y, z
print type(x), type(y), type(z)
else:
pat = 'fffsi'
(x, y, z, s, i) = [conv[p](x) for p,x in zip(pat, line)]
print x, y, z, s, i
print type(x), type(y), type(z), type(s), type(i)
#=======================================

I should have been more specific by saying that all the lines in the
file have the same format. But occasionally I will have a file that
is just not all floats, hence my question for handling a multi-type
line. Since I am the one generating the files I always know which
types the columns in the file have which makes it easier

Your answer, together with Steven's has given my much better
understanding of the issue of parsing in tables like this. Thanks
to the both of you!

Cheers
Tommy

 

[Bruno Desthuilliers]

Steven D'Aprano a écrit :

So how would you handle this type of error handling?
My main problem is that occasionally there is an entry
in the list that is a string:

0.9834 134.4933 78.009 run11 27

How do you want to deal with such an occasional string? What do you expect
the code to do?

The way I see it, if you're expecting five floats, and you get four floats
and a string, that's an error. Do you ignore the entire line, or replace
the string with zero, or what?

Again I would like to avoid having to individually parse the 3 floats,
while still easily handling the string, but a list comprehension will
not work as far as I can tell. Is there a module that handles this
type of flat ascii tables? Something like:

(x,y,z,id,n) = ParseFile("float","float","float","string","int")

would be great, and I guess

(x,y,z,id,n) = PaseFile2("%f %f %f %s %d")

would be even better.

How about this?

def parse_line(line, types):
items = line.split()
return [t(s) for (t, s) in zip(types, items)]

Since the OP mention of last items being here "occasionnaly", I'd go for
this instead (nb: ad-hoc implementation - yours is of course better as a
general solution):

def parse_line(line, *types):
items = line.split()
return [t(s) for (s, t) in zip(items, types)]


My 2 cents