using YAML as config

[Larry Edelstein]

Hi all -

I want to write a config file in YAML. In this file I will store a
small graph of objects in an array:

- !mydomain,mydate/Company
name: TuesdayCo
subthings:
- !mydomain,mydate/Person
name: Ruby Tuesday
- !mydomain,mydate/Person
name: Tuesday Weld
- !mydomain,myDate
name: FridayCo
subthings:
etc.

Then I want to read in this file and get an array of Company objects,
each of which has an array of Person objects. I'm defining these
classes myself. I've tried using add_domain_type and hoped to get
instances of my classes, but instead I got instances of
YAML:omainType.

How should I do this?

-larry

 

[Joel VanderWerf]

Hi all -

I want to write a config file in YAML. In this file I will store a
small graph of objects in an array:

- !mydomain,mydate/Company
name: TuesdayCo
subthings:
- !mydomain,mydate/Person
name: Ruby Tuesday
- !mydomain,mydate/Person
name: Tuesday Weld
- !mydomain,myDate
name: FridayCo
subthings:
etc.

Then I want to read in this file and get an array of Company objects,
each of which has an array of Person objects. I'm defining these
classes myself. I've tried using add_domain_type and hoped to get
instances of my classes, but instead I got instances of
YAML:omainType.

How should I do this?

Here's the only way I know to do this (I'm not sure if add_domain_type
is another way):

require 'yaml'

# This assumes that all your objects will serialize as hashes ("maps")
module Base
def to_yaml( opts = {} )
YAML::quick_emit( object_id, opts ) do |out|
out.map( taguri, to_yaml_style ) do |map|
to_yaml_properties.each do |m|
map.add( m, send( m ) )
end
end
end
end

module BaseModuleMethods
def yaml_new( klass, tag, val )
unless Hash === val
raise YAML::TypeError, "Invalid #{self}: " + val.inspect
## do more error checking here, if desired
end

name, type = YAML.read_type_class( tag, self )
st = type.new
val.each do |k,v|
st.send( "#{k}=", v )
end
st
end
end

def self.included mod
mod.extend BaseModuleMethods
end
end

class Company
include Base

attr_accessor :name, :subthings
def initialize name = nil
@name = name
end

yaml_as "tag:my.domain,2007:Company"

def to_yaml_properties
["name", "subthings"]
end
end

class Person
include Base

attr_accessor :name
def initialize name = nil
@name = name
end

yaml_as "tag:my.domain,2007erson"

def to_yaml_properties
["name"]
end
end

co = Company.new "TuesdayCo"
co.subthings = [
Person.new("Ruby Tuesday"),
Person.new("Tuesday Weld")
]

co_yaml = co.to_yaml
puts co_yaml
# ==>
# --- !my.domain,2007/Company
# name: TuesdayCo
# subthings:
# - !my.domain,2007/Person
# name: Ruby Tuesday
# - !my.domain,2007/Person
# name: Tuesday Weld

co2 = YAML.load(co_yaml)
p co2

# ==>
#<Company:0xb7ce3cfc @subthings=[#<Person:0xb7ce44cc @name="Ruby
Tuesday">, #<Person:0xb7ce3ef0 @name="Tuesday Weld">], @name="TuesdayCo">


# Check that error-checking works by
# giving the parser an array instead of a hash:
s = <<END
--- !my.domain,2007/Company
- 1
- 2
END

begin
p YAML.load(s)
rescue YAML::TypeError => ex
puts ex.message # Invalid Company: [1, 2]
end

 

[_why]

- !mydomain,mydate/Company
name: TuesdayCo
subthings:
- !mydomain,mydate/Person
name: Ruby Tuesday
- !mydomain,mydate/Person
name: Tuesday Weld
- !mydomain,myDate
name: FridayCo
subthings:
etc.
[...]

How should I do this?

Heya, larry! The easy way is like this:

class Company
yaml_as 'tag:mydomain.com,2007:Company'
end

class Person
yaml_as 'tag:mydomain.com,2007erson'
end

_why

 

[Larry Edelstein]

Wow Joel - that's a lot of code and a lot to digest. I'm going through
it. Your response and _why's use the yaml_as attribute - I've never
even seen that before.

 

[Joel VanderWerf]

Wow Joel - that's a lot of code and a lot to digest. I'm going through
it. Your response and _why's use the yaml_as attribute - I've never
even seen that before.

It's nice to know that _why's much simpler code works. The other stuff I
wrote is how I've handled validation and other customization before. I
wonder if _why has a better way to do that too...

 

[Rover Rhubarb]

So I come to the part of my little app where I want to load and save a
config file. I want it to be human read/writeable too though - it's not
just for serialization. In fact, usually a human will write it, my
little app will read it, and may adjust it and write it out anew.

So in xml it would look like this
<channels>
<channel>
<name>foo</name>
<id>3</id>
</channel>
<channel>
…
</channels>

I don't care about channels that's just a random root node, and I might
do it with attbs instead of elements, so:

<channel name="foo" id="3"/>

So I thought, lets use YAML instead - it's even easier to read and
write. Google, google, read blogs, docs, google, google, more docs, more
blogs, google to_yaml, fxri: hack hack, google to_yaml_type, fxri: hack,
hack, hack, google, google, yaml_as, hack.

Argh! Seems like day later, and I'm ready to go back to xml. But I'm
sure there's a simple way to do what I want, so I'll post here first and
hope. See, my class is Channel, like so:

class Channel; attr_accessor :name, :id; end

and want my array of channels to look something like this

- Channel
name: foo
id: 3
- Channel
name: bar
id: 4
…

But I just can't work out how to simply the yaml output to this stage.
And I don't want to override a dozen methods and do my own hash mapping.
The easiest way - thanks to why for posting this in the forum - seems to
be to use yaml_as:

class Channel
yaml_as "tag:foobar,2007:Channel"
end

but then I get each entry as

--- !foolbar,2007/Channel
name: foo
id: 3

This is fairly simple, but remember that I want my entries to be
human-writeable the first time. That line of "---!foobar,2007…" is just
way too machiney for us poor humans to get right every time.

I can simplify the taguri like this
yaml_as "tag::Channel"

to get

--- !Channel
...

and that's about as simple as I can get it without it failing to write
yaml. But loading it back in will fail to build the object - it will
give ma a YAML:ynamicType instead - which is what it always gives if
it doesn't recognize the object.
I guess the load method needs a valid taguri

(Side note - I'd never heard of a taguri before, but when I went to read
about it at http://www.taguri.org/ I was _very_ pleasantly surprised at
how quickly I was able to grok the whole thing by reading Sandro Hawke's
four-paragraph tutorial about his dog Taiko. Way to go Sandro - if I'd
been directed to the RFC first it'd be all over)

So anyway, the fact is that while taguris are great for distinguishing
Taiko from his master's descendents' pooches, and while it's very
simple, it's still overkill for me. I don't need a unique ID. I know
what "--- Channel" means in my little app. I don't want to submit my
Channel types to some great panoply of random little types each with
their own special pet name. I just want to damn well read and write it.

I've even considered parsing the text in to_yaml and YAML.load and
gsubbing "!foobar, 2007/" out and in again. But there's got to be a
better way. So far that better way is looking like xml. but there's got
to be a better rubyful way.

What is it?

 

[Greg Donald]

So far that better way is looking like xml. but there's got
to be a better rubyful way.

What is it?

json?

http://json.rubyforge.org/

 

[Ara.T.Howard]

So I come to the part of my little app where I want to load and save a
config file. I want it to be human read/writeable too though - it's not just
for serialization. In fact, usually a human will write it, my little app
will read it, and may adjust it and write it out anew.

So in xml it would look like this
<channels>
<channel>
<name>foo</name>
<id>3</id>
</channel>
<channel>
…
</channels>

I don't care about channels that's just a random root node, and I might
do it with attbs instead of elements, so:

<channel name="foo" id="3"/>

So I thought, lets use YAML instead - it's even easier to read and
write. Google, google, read blogs, docs, google, google, more docs, more
blogs, google to_yaml, fxri: hack hack, google to_yaml_type, fxri: hack,
hack, hack, google, google, yaml_as, hack.

Argh! Seems like day later, and I'm ready to go back to xml. But I'm
sure there's a simple way to do what I want, so I'll post here first and
hope. See, my class is Channel, like so:

class Channel; attr_accessor :name, :id; end

and want my array of channels to look something like this

- Channel
name: foo
id: 3
- Channel
name: bar
id: 4
…

But I just can't work out how to simply the yaml output to this stage.
And I don't want to override a dozen methods and do my own hash mapping.
The easiest way - thanks to why for posting this in the forum - seems to
be to use yaml_as:

class Channel
yaml_as "tag:foobar,2007:Channel"
end

but then I get each entry as

--- !foolbar,2007/Channel
name: foo
id: 3

This is fairly simple, but remember that I want my entries to be
human-writeable the first time. That line of "---!foobar,2007…" is just
way too machiney for us poor humans to get right every time.

I can simplify the taguri like this
yaml_as "tag::Channel"

to get

--- !Channel
...

and that's about as simple as I can get it without it failing to write
yaml. But loading it back in will fail to build the object - it will
give ma a YAML:ynamicType instead - which is what it always gives if
it doesn't recognize the object.
I guess the load method needs a valid taguri

(Side note - I'd never heard of a taguri before, but when I went to read
about it at http://www.taguri.org/ I was _very_ pleasantly surprised at
how quickly I was able to grok the whole thing by reading Sandro Hawke's
four-paragraph tutorial about his dog Taiko. Way to go Sandro - if I'd
been directed to the RFC first it'd be all over)

So anyway, the fact is that while taguris are great for distinguishing
Taiko from his master's descendents' pooches, and while it's very
simple, it's still overkill for me. I don't need a unique ID. I know
what "--- Channel" means in my little app. I don't want to submit my
Channel types to some great panoply of random little types each with
their own special pet name. I just want to damn well read and write it.

I've even considered parsing the text in to_yaml and YAML.load and
gsubbing "!foobar, 2007/" out and in again. But there's got to be a
better way. So far that better way is looking like xml. but there's got
to be a better rubyful way.

What is it?

there are simpler ways, but here is one that quite simple and also quite
extensible:



cfp:~ > ruby a.rb
- Foo:
a: 4
b: 2
- Foo::Bar:
a: 40
b: 1
c: 1
#<Foo:0xb75aadc4 @b=2, @a=4>
#<Foo::Bar:0xb75aacc0 @b=1, @c=1, @a=40>



cfp:~ > cat a.rb
#
# define some classes which impliment the required '.from_hash' and '#to_hash'
# duckface our ObjectList class will expect
#
class Foo
def self.from_hash hash
new hash['a'], hash['b']
end
def initialize a, b
@a, @b = a, b
end
def to_hash
{ 'a' => @a, 'b' => @b }
end
end
class Foo::Bar
def self.from_hash hash
new hash['a'], hash['b'] , hash['c']
end
def initialize a, b, c
@a, @b, @c = a, b, c
end
def to_hash
{ 'a' => @a, 'b' => @b, 'c' => @c }
end
end
#
# an ObjectList class adapted to store any objects which respond_to 'to_hash'
# and whose class responds_to 'from_hash'
#
require 'yaml'
class ObjectList < ::Array
attr 'path'
def initialize path
load path
end
def load path
@path = path
replace YAML.load(( IO.read @path rescue '[]' ))
map!{|entry| obj_from_entry entry}
end
def save path = @path
yaml = map{|obj| entry_from_obj obj}.to_yaml
open(path, 'w'){|fd| fd.write yaml}
end
def obj_from_entry entry
classname, hash = entry.to_a.first
constant_get(classname).from_hash hash
end
def entry_from_obj obj
classname, hash = obj.class.name, obj.to_hash
{ classname => hash }
end
def constant_get hierachy
ancestors = hierachy.split %r/::/
parent = Object
while((child = ancestors.shift))
klass = parent.const_get(child) and parent = klass
end
klass
end
end
#
# setup a object_list object
#
object_list = ObjectList.new 'object_list'
#
# store a few instances in the object_list
#
object_list << Foo.new(4,2)
object_list << Foo::Bar.new(40, 1, 1)
#
# save the object_list
#
object_list.save
#
# show that save works
#
puts IO.read(object_list.path)
#
# show that loading works
#
object_list = ObjectList.new 'object_list'
p object_list.first
p object_list.last




-a

 

[_why]

I can simplify the taguri like this
yaml_as "tag::Channel"

to get

--- !Channel
...

and that's about as simple as I can get it without it failing to write
yaml. But loading it back in will fail to build the object - it will
give ma a YAML:ynamicType instead - which is what it always gives if
it doesn't recognize the object.

Try:

class Channel
yaml_as "x-private:Channel"
end

And you get:

--- !!Channel
id: 3
name: foo

G'luck!!

_why

 

[Rover Rhubarb]

Let me add:

- I meant DomainType, not dynamic type

- The use of yaml_as is redundant since I can get the same effect
without it - where my objects are dumped as

- !ruby/object:Channel

instead of my own silly name.

Still not good enough: how do I get rid of the !ruby/object
machine-talk. (Is it machine-talk, or was YAML designed by bushmen of
the Kalahari, where the scripting language of choice is "click-ruby")

 

[James Edward Gray II]

and want my array of channels to look something like this

- Channel
name: foo
id: 3
- Channel
name: bar
id: 4

Hope this helps:

class Channel < Struct.newname, :id)
def self.parse(io)
io.inject(Array.new) do |channels, line|
if line =~ /\bChannel\b/
channels << new
elsif not channels.empty? and line =~ /\A\s+(name|id):\s+(.+)/
channels.last.send("#{$1}=", $2)
end
channels
end
end
end

if __FILE__ == $PROGRAM_NAME
p Channel.parse(DATA)
end

__END__
- Channel
name: foo
id: 3
- Channel
name: bar
id: 4

James Edward Gray II

 

[Rover Rhubarb]

Thanks all, for the effort made in replying. I'm pleased to see some
solutions, and don't want to seem ungrateful, but I'm still a little
disillusioned that xml is looking more straightforward even now.

_why: thanks for the x-private tip, but I still have to put
"click-click-Channel". Simple enough to read, but honestly to the
average config hand-coder the xml <Channel> is easier to remember and
understand than the arcane "--- !!Channel". YAML should beat XML hands
down - this is one case where it doesn’t. I know it's not the same
because I'm going to have to parse the XML and build the objects -
whereas YAML is going to deserialize into lovely objects right there on
the spot - but still…

James: thanks for giving me a solution that does what I asked exactly.
But to be honest here I wanted to use an existing format, not invent
one. (I guess it could be argued that by wanting to customize my YAML to
get rid of the ---!! I am trying to invent one - but I really want that
to be an existing feature of YAML)

Greg: funny you mention json because I had a similar experience with it
recently. I was coding some ajax for the first time (building a progress
indicator) and I decided to pass my simple values as properties of a
javascript object using json. Hours of tricky debugging later (very hard
to find those bugs in the json code) I got it working but decided next
time I'd use XML. The extra parsing step might cost more, but the bugs
would be easier to find.

I guess _why has the closest to what I was looking for. x-private is
clearly even intended to be used for this. (BTW sorry to bellyache
about docs, since I know its enough effort to code the libs in the first
place, but this can't be an uncommon request so maybe the x-private
could get more airtime in the docs somewhere). And I guess what I'd
really like is for the x-private tag in yaml_as to allow the --- !! to
be skipped to.

thanks again all

 

[James Edward Gray II]

YAML should beat XML hands down - this is one case where it doesn=92t.

I think mine is an unpopular opinion among Rubyists, but I find XML =20
easier to write, as a human, than YAML. YAML just has too many rules =20=

I have to remember. XML is definitely wordier, but I can keep the =20
rules in my head as I work. My editor also makes managing those tags =20=

fairly easy.

If all I'm doing is a simple edit here and there though, YAML is =20
easier on the eyes, for sure.

I know it's not the same because I'm going to have to parse the XML =20=

and build the objects

I'm pretty sure there's a library or two that handles that:

http://raa.ruby-lang.org/cat.rhtml?=20
category_major=3DLibrary;category_minor=3DXML

James: thanks for giving me a solution that does what I asked exactly.
But to be honest here I wanted to use an existing format, not invent
one. (I guess it could be argued that by wanting to customize my =20
YAML to
get rid of the ---!! I am trying to invent one - but I really want =20
that
to be an existing feature of YAML)

This is where we start to disagree.

You asked for simple and even showed examples of what you had in =20
mind. I wrote code that parses it. Simple pays off at both ends.

You wanted to drop the computerese; my version did. In fact, mine =20
doesn't need those dashes. With a one byte change, you could also =20
make indenting the attributes optional. It gets better and better =20
for the human who needs to write it and it's still not complex on =20
your end.

Finally, I think it's easier to read or write than XML or YAML.

When your needs are big, of course you want to go with a proven =20
format. What you showed is simple though and parsed just fine with a =20=

few lines of code. I don't see any shame in that.

James Edward Gray II

 

[Rover Rhubarb]

I think mine is an unpopular opinion among Rubyists, but I find XML
easier to write, as a human, than YAML. YAML just has too many rules
I have to remember. XML is definitely wordier, but I can keep the
rules in my head as I work. My editor also makes managing those tags
fairly easy.

If all I'm doing is a simple edit here and there though, YAML is
easier on the eyes, for sure.

Amen. I might not have agreed with you had I just read about YAML, but
after going through this process I'm a born again xml convert.

I also agree YAML is easier on the eyes. If were just ser-deser I was
after, I'd probably go with YAML.

I'm pretty sure there's a library or two that handles that:

http://raa.ruby-lang.org/cat.rhtml?
category_major=Library;category_minor=XML

Thanks for this tip.
Of these, http://clabs.org/clxmlserial.htm looks the most promising.

Note, btw, the home page of myxml
(http://raa.ruby-lang.org/project/myxml/), where it states:
(* use YAML instead, it's 100% better for small apps ;-))

This is where we start to disagree.

You asked for simple and even showed examples of what you had in
mind. I wrote code that parses it. Simple pays off at both ends.

You wanted to drop the computerese; my version did. In fact, mine
doesn't need those dashes. With a one byte change, you could also
make indenting the attributes optional. It gets better and better
for the human who needs to write it and it's still not complex on
your end.

Finally, I think it's easier to read or write than XML or YAML.

When your needs are big, of course you want to go with a proven
format. What you showed is simple though and parsed just fine with a
few lines of code. I don't see any shame in that.

James Edward Gray II

Fair point.