csv and iterator protocol

[Philippe C. Martin]

Hi,

I have the following working program:

1) I import data in csv format into internal data structures (dict + list)
2) I can export back to csv
3) I can store my internal data using pickle+bz2
4) I can reload it.


Hovever I notice a factor 10 size loss using pickle.

So I would like to bzip/store in csv format but am not certain how to read
the data back as the csv module as described uses a file name in its
constructor. I would like to avoid reading the data and writing it into a
file prior to re-importing it.

Can I initialize csv with input data stored in RAM (ex: a string) ? - so far
I cannot get that to work. Or to rephrase the question, what Python "RAM"
structure supports the "iterator protocol" ?

Thanks,

Philippe

 

[Kent Johnson]

Can I initialize csv with input data stored in RAM (ex: a string) ? - so far
I cannot get that to work. Or to rephrase the question, what Python "RAM"
structure supports the "iterator protocol" ?

Many, including strings, lists and dicts. For your needs, a list of strings will work, or a cStringIO initialized from your data.

Kent

 

[Philippe C. Martin]

Thanks Kent,

I had a bug in my test program: it works fine with strings

Philippe