problems with CSV module

[Carlos Grohmann]

Hi all, I'm using csv to read text files, and its working fine, except
in two cases:

- when there is only one line of text (data) in the file
- when there is a blank line after the last data line

this is the kind of data:

45 67 89
23 45 06
12 34 67
....


and this is the function:

def getData(paths):
"""get data from file and create lists with values and column
names."""

filehandle = paths# # filehandle = os.path.join(dirname,
filename)
csvfile = open(filehandle,'r') # Open the file and read the
contents

sample = csvfile.read( 1024 )# Grab a sample
csvfile.seek( 0 )

dialect = csv.Sniffer().sniff(sample) # Check for file format with
sniffer.
csvfile = csv.reader( csvfile, dialect )
if csv.Sniffer().has_header( sample ): #if there is a header
colnames = csvfile.next() # label columns from first line
datalist = list( csvfile ) # append data to a list
else: #if there is NO header
datalist = list( csvfile ) # append data to a list
colnames = ['col_%i' % i for i in range(len(datalist[0]))] #
label columns as col_1, col_2, etc

return datalist, colnames

TIA for any help.

 

[Neil Cerutti]

Hi all, I'm using csv to read text files, and its working fine, except
in two cases:

- when there is only one line of text (data) in the file
- when there is a blank line after the last data line

this is the kind of data:

45 67 89
23 45 06
12 34 67
...

That data doesn't appear to be csv worthy. Why not use str.split
or str.partition?

and this is the function:

def getData(paths):
"""get data from file and create lists with values and column
names."""

filehandle = paths# # filehandle = os.path.join(dirname,
filename)
csvfile = open(filehandle,'r') # Open the file and read the

In Python 2.6 and earlier, you need to open the file in binary
mode.

In Python 3.0 and later, you need to open the file with a mystic
incantation:

csvfile = open(filehandle, newline='')

sample = csvfile.read( 1024 )# Grab a sample
csvfile.seek( 0 )
dialect = csv.Sniffer().sniff(sample) # Check for file format with
sniffer.
csvfile = csv.reader( csvfile, dialect )

Use:

csvfile = csv.reader(csvfile, dialect=dialect)

dialect is a keyword argument.

if csv.Sniffer().has_header( sample ): #if there is a header
colnames = csvfile.next() # label columns from first line
datalist = list( csvfile ) # append data to a list

Do you really need to use the Sniffer? You'll probably be better
off

 

[Neil Cerutti]

Do you really need to use the Sniffer? You'll probably be better
off...

....defining your own dialect based on what you know to be the
file format.

 

[Carlos Grohmann]

Thanks for your prompt response, Neil.

That data doesn't appear to be csv worthy. Why not use str.split
or str.partition?

Well, I should have said that this is one kind of data. The function
is part of a larger app, and there is the possibility that someone
uses headers in the data files, or some other field separator, so I
tried to make it more universal.

In Python 2.6 and earlier, you need to open the file in binary
mode.

I tried that, no changes

Use:
   csvfile = csv.reader(csvfile, dialect=dialect)
dialect is a keyword argument.

thanks for pointing that out.it stopped the errors when there s only
one
data line, but it still can't get the values for that line

Carlos

 

[Neil Cerutti]

thanks for pointing that out.it stopped the errors when there s
only one data line, but it still can't get the values for that
line

Is it possible your data is ill-formed in that case? If it's
lacking a line-end, I don't know what should happen.