extract to dictionaries

[Marius Retegan]

Hello
I have simple text file that I have to parse. It looks something like
this:

parameters1
key1 value1
key2 value2
end

parameters2
key1 value1
key2 value2
end

So I want to create two dictionaries parameters1={key1:value1,
key2:value2} and the same for parameters2.

I woud appreciate any help that could help me solve this.
Thank you

 

[bearophileHUGS]

Marius Retegan:

parameters1
     key1 value1
     key2 value2
end

parameters2
     key1 value1
     key2 value2
end

So I want to create two dictionaries parameters1={key1:value1,
key2:value2} and the same for parameters2.

I have wasted some time trying to create a regex for that. But it's
better to use normal code. Iterate on the lines, keep a dict, and when
you find a string that doesn't start with whitespace, use it to create
a new key-value into the dict, where the key is the stripped line and
the value is an empty dict. Then you can enter a sub loop or set a
"inside" boolean variable, to denote you are in a different part of
the state machine. When you find lines that start with a space, you
can put add them as key-value into the latest dict (so you have to
keep the key name in the superdict, or more efficiently you can keep
your subdict on a side, and you can add it only at the end when you
see a line "end" with no leading spaces. When you find such "end" you
can exit the sub loop or rest the "inside" boolean.
Overall if's just few lines of code, much shorter than my description
of the algorithm.

Bye,
bearophile

 

[Gary Herron]

Hello
I have simple text file that I have to parse. It looks something like
this:

parameters1
key1 value1
key2 value2
end

parameters2
key1 value1
key2 value2
end

So I want to create two dictionaries parameters1={key1:value1,
key2:value2} and the same for parameters2.

I woud appreciate any help that could help me solve this.
Thank you

This looks like a homework problem. But even if it's not, you are not
likely to find someone who is willing to put more work into this problem
than you have.

So why don't you show us what you've tried, and see if someone is
willing to make suggestions or answer specific question about your
attempt at a solution?

Gary Herron

 

[Rhodri James]

Hi,




It's not. I'm passed homework age.



I don't now if posting a code that gets into a while loop and never stops
would demonstrate to you that I've tried.

It would have. At the very least, it would have told us that you've
missed a common idiom.

Be assured that before posting to
the list I did try to solve it myself, because I knew that I might get an
answer like RTFM or similar.

Not posting code (or code snippets at least) makes it more likely that
you'll
be told to RTFM, you do realise!

Maybe I'm not smart enough, but I can't make python to start reading
after
the "parameter1" line and stop at the "end" line. That's all I want a
small
piece of pseudocode to do just that.

I'd be tempted to do it like this

dict_of_dicts = {}
current_dict = {}
current_name = "dummy"

f = open(filename)
for line in f:
# Do something to skip blank lines
if line == '\n':
continue

# A normal 'key value' pair?
if line.startswith(' '):
# Yup. Split apart the key and value,
# and add them to the current dictionary
current_dict.update([line.split()])
elif line == 'end':
# Wrap up what we've got and save the dictionary
dict_of_dicts[current_name] = current_dict
current_name = dummy
current_dict = {}
else:
# New section. Really ought to whinge if
# we haven't ended the old section.
current_name = line.strip()
current_dict = {}

You can then pull the parameter sets you want out of
dict_of_dicts (you can probably think of a more meaningful
name for it, but I don't know the context you're working in).
In real code I would use regular expressions rather than
`startswith` and the equality because they cope more easily
with tabs, newlines and other 'invisible' whitespace.

 

[Gary Herron]

Hi,

On Fri, May 29, 2009 at 2:09 AM, Gary Herron

Marius Retegan wrote:

Hello
I have simple text file that I have to parse. It looks
something like
this:

parameters1
key1 value1
key2 value2
end

parameters2
key1 value1
key2 value2
end

So I want to create two dictionaries parameters1={key1:value1,
key2:value2} and the same for parameters2.

I would appreciate any help that could help me solve this.
Thank you



This looks like a homework problem.


It's not. I'm passed homework age.


But even if it's not, you are not likely to find someone who is
willing to put more work into this problem than you have.
So why don't you show us what you've tried, and see if someone is
willing to make suggestions or answer specific question about your
attempt at a solution?


I don't now if posting a code that gets into a while loop and never
stops would demonstrate to you that I've tried. Be assured that before
posting to the list I did try to solve it myself, because I knew that
I might get an answer like RTFM or similar.
Maybe I'm not smart enough, but I can't make python to start reading
after the "parameter1" line and stop at the "end" line. That's all I
want a small piece of pseudocode to do just that.

OK. Assuming you are open a file with something like:
f = open('data', 'r')

Then this will read lines up to the first "parameters" line
for line in f:
if line.startswith('parameters'):
break

At this point, line contains 'parameters1\n'. Do with it as you will.

Then read and process lines until an end line is reached
for line in f:
if line.beginswith('end'):
break
# Here line contains ' key1 value1\n'.
# Perhaps use line.strip to remove the white space on each end
# and k,v =line.split() to split out the two values on the line.

You'll need more:
A loop to keep the above two going until the end of file
A way to recognize the end of the file.

Gary Herron

 

[Emile van Sebille]

On 5/28/2009 4:03 PM Marius Retegan said...

Hello
I have simple text file that I have to parse. It looks something like
this:

parameters1
key1 value1
key2 value2
end

parameters2
key1 value1
key2 value2
end

So I want to create two dictionaries parameters1={key1:value1,
key2:value2} and the same for parameters2.

I woud appreciate any help that could help me solve this.
Thank you

Assuming you've read the above into paramFile...

for dictvals in [xx.split() for xx in paramFile.split("end") if xx]:
locals()[dictvals[0]]=dict(zip(dictvals[1::2],dictvals[2::2]))

You-can't-really-call-this-helping-ly yrs,

Emile

 

[Mike Kazantsev]

Hello
I have simple text file that I have to parse. It looks something like
this:

parameters1
key1 value1
key2 value2
end

parameters2
key1 value1
key2 value2
end

So I want to create two dictionaries parameters1={key1:value1,
key2:value2} and the same for parameters2.

You can use iterators to efficiently parse no-matter-how-large file.
Following code depends on line breaks and 'end' statement rather than
indentation.


import itertools as it, operator as op, functools as ft
from string import whitespace as spaces

with open('test.src') as src:
lines = it.ifilter(bool, it.imap(lambda x: x.strip(spaces), src))
sections = ( (lines.next(), dict(it.imap(str.split, lines))) for sep,lines in
it.groupby(lines, key=lambda x: x == 'end') if not sep )
data = dict(sections)

print data
# { 'parameters2': {'key2': 'value2', 'key1': 'value1'},
# 'parameters1': {'key2': 'value2', 'key1': 'value1'} }



To save namespace and make it a bit more unreadable you can write it
as a one-liner:

with open('test.src') as src:
data = dict( (lines.next(), dict(it.imap(str.split, lines))) for sep,lines in
it.groupby(it.ifilter(bool, it.imap(lambda x: x.strip(spaces), src)),
key=lambda x: x == 'end') if not sep )


--
Mike Kazantsev // fraggod.net

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.11 (GNU/Linux)

iEYEARECAAYFAkogabEACgkQASbOZpzyXnFOcwCgkxnm6pBNywpwWSeUs2md8dQA
F4oAoPBNqAjf2gcgnCE48rscD0BbeFDV
=rsy6
-----END PGP SIGNATURE-----