Concerning Marshalling

[christophe.poucet]

Hello,

I was trying to marshal some objects, however some of these objects
contain references to singletons which of course can't be marshalled.
That's why I tried something as follows:

class Container
end

class SpecialContainer < Container
include Singleton
end

class Test
attr_accessor :container
def _dump(depth)
old = @container
if old == SpecialContainer.instance
@container = nil
end
result = super
@container = old
return result
end
def self._load(str)
super
if @container == nil
@container = SpecialContainer.instance
end
self
end
end

Sadly, however, it seems there is no default implementation for _dump,
which means that in this case the super call results in a
NoMethodError. Is there any other way to resolve this problem without
custom-coding a full _dump method?

Regards,
Christophe

 

[ts]

c> That's why I tried something as follows:

Perhaps best to use marshal_dump/marshal_load

Guy Decoux

 

[Joel VanderWerf]

c> That's why I tried something as follows:

Perhaps best to use marshal_dump/marshal_load

Guy Decoux

That's a good suggestion. The marshal_dump/marshal_load pair can be very
useful, because it integrates well with the whole Marshal.dump or
Marshal.load process. An inherent problem with _dump and _load is that
you have to use strings, so you are responsible for somehow preserving
and restoring references to other objects that are also being dumped.
This is critical when the original dump call was on an object that
referred to your object, not directly to your object.

With marshal_dump/marshal_load, you produce/consume an arbitrary
Marshallable object, possibly containing references, and all the code in
marshal.c takes over the hard task of preserving and restoring
references. Probably an example will say it better:

class Node
attr_accessor rev_node, :next_node

attr_accessor :temp_data
# don't want this to persist (maybe it's a cache, or simply
# undumpable, like a proc, or a file, or a singleton).

def marshal_dump
dumped_obj = [prev_node, next_node]
dumped_obj
end

def marshal_load(dumped_obj)
@prev_node, @next_node = dumped_obj
end
end

node1 = Node.new
node2 = Node.new
node1.next_node = node2
node2.prev_node = node1
node1.temp_data = STDOUT
node2.temp_data = proc {"foo"}

p node1

p Marshal.load(Marshal.dump(node1))

__END__

output:

#<Node:0xb7dff098 @next_node=#<Node:0xb7dfef08
@prev_node=#<Node:0xb7dff098 ...>, @temp_data=#<Proc:0xb7dff37c@-:23>>,
@temp_data=#<IO:0xb7e4c078>>
#<Node:0xb7dfed64 @prev_node=nil, @next_node=#<Node:0xb7dfed28
@prev_node=#<Node:0xb7dfed64 ...>, @next_node=nil>>

 

[Ara.T.Howard]

Hello,

I was trying to marshal some objects, however some of these objects
contain references to singletons which of course can't be marshalled.
That's why I tried something as follows:

class Container
end

class SpecialContainer < Container
include Singleton
end

class Test
attr_accessor :container
def _dump(depth)
old = @container
if old == SpecialContainer.instance
@container = nil
end
result = super
@container = old
return result
end
def self._load(str)
super
if @container == nil
@container = SpecialContainer.instance
end
self
end
end

Sadly, however, it seems there is no default implementation for _dump,
which means that in this case the super call results in a
NoMethodError. Is there any other way to resolve this problem without
custom-coding a full _dump method?

Regards,
Christophe

for some reason the mailing list seemed to have eaten my post about this so
here it is again - sorry if it's a dup:


harp:~ > cat dumpable_singleton.rb

module DumpableSingleton
module InstanceMethods
attr_accessor 'ikey'
def _dump *a, &b
Marshal::dump ikey, *a, &b
end
def == other
self.ikey == other.ikey
end
alias_method 'eql?', '=='
end
module ClassMethods
def new *a, &b
ikey = []
ikey.push *a
ikey.push b if b
if instances.has_key? ikey
instances[ ikey ]
else
begin
ikey.each{|k| Marshal::dump k}
rescue TypeError, Exception
raise ArgumentError, "all arguments must be dumpable!"
end
instances[ (obj = super(*a, &b)).ikey = ikey ] = obj
end
end
def _load buf, *a, &b
new(*(ikey = Marshal::load(buf, *a, &b)))
end
attr_accessor 'instances'
end
def self::included other
other.module_eval{ include InstanceMethods }
other.extend ClassMethods
other.instances = {}
other
end
end

class C
include DumpableSingleton
def initialize(*a, &b); end
end

x = C::new 42
y = C::new 42
z = C::new 43

puts '-' * 42
C::instances.each {|k,c| p k => c}
p x == y
p x == z
p y == z

xd, yd, zd = [x, y, z].map{|c| Marshal::load(Marshal::dump(c))}

puts '-' * 42
C::instances.each {|k,c| p k => c}
p xd == yd
p xd == zd
p yd == zd

puts '-' * 42
C::instances.each {|k,c| p k => c}
p x == xd
p y == yd
p z == zd


harp:~ > ruby dumpable_singleton.rb

------------------------------------------
{[43]=>#<C:0xb75cc3c4 @ikey=[43]>}
{[42]=>#<C:0xb75cc4b4 @ikey=[42]>}
true
false
false
------------------------------------------
{[43]=>#<C:0xb75cc3c4 @ikey=[43]>}
{[42]=>#<C:0xb75cc4b4 @ikey=[42]>}
true
false
false
------------------------------------------
{[43]=>#<C:0xb75cc3c4 @ikey=[43]>}
{[42]=>#<C:0xb75cc4b4 @ikey=[42]>}
true
true
true

i've used something like this in in a few small projects.

hth.

-a
--
===============================================================================
| email :: ara [dot] t [dot] howard [at] noaa [dot] gov
| phone :: 303.497.6469
| Your life dwells amoung the causes of death
| Like a lamp standing in a strong breeze. --Nagarjuna
===============================================================================

 

[christophe (dot) poucet (at) gmail (dot) com]

Hello,

Guy and Joel: Thank you for your helpful comments, this was exactly
what I was looking for and this was sadly not mentioned in the RDoc of
Marshal :/. For the moment I had been using YAML, but this will not be
a sustainable solution once the number of objects to dump grows in
size. Eventually I want to be able to store the objects into a
database based upon a key that is unique per object (all my classes
derive from Managed that ensures a unique identifier) As for the
references being kept, great!

Ara: Your solution is definitely an interesting one, however I fear
that it is not a sustainable solution for larger projects and the
problem was in more than just singletons, singletons being a symptom of
the problem.

Cheers,
Christophe / vincenz

 

[christophe (dot) poucet (at) gmail (dot) com]

Concerning tracking references.

I want to store each object as a separate value. I can assure that
each object has a unique id (all the objects that need to be stored
separately are instance_of? Managed), however if I marshal the objects
one by one, then the references they might hold internally to each
other will of course no long work. How would I solve this problem?
Obviously I need to replace the references by some sort value that
refers to it, or otherwise anything that is referred will be stored
multiple times. However I'm not quite sure how to put the hooks in
place for this.

Christophe.

 

[Joel VanderWerf]

Concerning tracking references.

I want to store each object as a separate value. I can assure that
each object has a unique id (all the objects that need to be stored
separately are instance_of? Managed), however if I marshal the objects
one by one, then the references they might hold internally to each
other will of course no long work. How would I solve this problem?
Obviously I need to replace the references by some sort value that
refers to it, or otherwise anything that is referred will be stored
multiple times. However I'm not quite sure how to put the hooks in
place for this.

Christophe.

Here's one way:

class Node
@@nodes = Hash.new {|h,k| raise ArgumentError, "Missing node
#{k.inspect}"}

# The id of a node can be anything but +nil+.
attr_reader :id

def initialize id
@id = id
@@nodes[id] = self
end

attr_accessor rev_node, :next_node

attr_accessor :temp_data
# don't want this to persist (maybe it's a cache, or simply
# undumpable, like a proc, or a file, or a singleton).

def marshal_dump
[prev_node, next_node].map {|node| node && node.id}
end

def marshal_load(dumped_obj)
@prev_node, @next_node = dumped_obj.map do |node_id|
node_id.nil? ? nil : @@nodes[node_id]
end
end

def self.delete id
@@nodes.delete id
end
end

node1 = Node.new "One"
node2 = Node.new 2

node1.next_node = node2
node2.prev_node = node1

node1.temp_data = STDOUT
node2.temp_data = proc {"foo"}

p node1
node1_dumped = Marshal.dump(node1)
p Marshal.load(Marshal.dump(node1))

Node.delete 2
p Marshal.load(node1_dumped)
# This should raise an exception, to show that the referred node is not
# stored in node1_dumped, and not in the @@nodes hash.

__END__

#<Node:0xb7dfe260 @temp_data=#<IO:0xb7e4c078>, @id="One",
@next_node=#<Node:0xb7dfe24c
@temp_data=#<Proc:[email protected]:40>, @id=2,
@prev_node=#<Node:0xb7dfe260 ...>>>
#<Node:0xb7dfdeb4 @next_node=#<Node:0xb7dfe24c
@temp_data=#<Proc:[email protected]:40>, @id=2,
@prev_node=#<Node:0xb7dfe260 @temp_data=#<IO:0xb7e4c078>, @id="One",
@next_node=#<Node:0xb7dfe24c ...>>>, @prev_node=nil>
marshal-refs2.rb:2: Missing node 2 (ArgumentError)
from marshal-refs2.rb:2:in `call'
from marshal-refs2.rb:24:in `default'
from marshal-refs2.rb:24:in `[]'
from marshal-refs2.rb:24:in `marshal_load'
from marshal-refs2.rb:23:in `map'
from marshal-refs2.rb:23:in `marshal_load'
from marshal-refs2.rb:47:in `load'
from marshal-refs2.rb:47