GC.disable not working?

[Eric Mahurin]

From what I can tell, GC.disable doesn't work. I'm wanting to
use GC.disable/GC.enable around code using objects that I get
from ObjectSpace._id2ref to make sure that the id doesn't
become invalid because of GC. Here is a piece of code that
demonstrates the problem:


#!/bin/env ruby

def finalizer(id)
print("<")
GC.enable and
printf("GC disabled while finalizing %x!!!",id)
@ids.delete(id)
print(">")
end

$stdout.sync=true
@ids = {}
1000.times { |i|
print("[")
obj = "X"*i
id = obj.object_id
@ids[id] = true
ObjectSpace.define_finalizer(obj,methodfinalizer))
GC.disable
print("{")
@ids.each_key { |id|
# go make sure all objects seem valid (test size)
ObjectSpace._id2ref(id).size
}
print("}")
GC.enable
print("]")
}


I tried this on 1.8.2 and a 1.9 CVS version. In both it calls
the finalizer when GC is disabled (GC.enable==true). I thought
this would be impossible. I also get a recycled object error.
In the 1.9 version I'm using, it hangs while trying to finalize
one of the objects (@ids.delete(id) is where it hangs).

Any clues to what is going on with this GC stuff. If I can't
get this to work, I don't see how anybody reliably use
ObjectSpace._id2ref. BTW, I found this problem in my code
after running tens of thousands of random tests. If you don't
do enough testing, you may not find problems related to garbage
collection.





__________________________________
Yahoo! Mail Mobile
Take Yahoo! Mail with you! Check email on your mobile phone.
http://mobile.yahoo.com/learn/mail

 

[Robert Klemme]

From what I can tell, GC.disable doesn't work. I'm wanting to
use GC.disable/GC.enable around code using objects that I get
from ObjectSpace._id2ref to make sure that the id doesn't
become invalid because of GC. Here is a piece of code that
demonstrates the problem:


#!/bin/env ruby

def finalizer(id)
print("<")
GC.enable and
printf("GC disabled while finalizing %x!!!",id)
@ids.delete(id)
print(">")
end

$stdout.sync=true
@ids = {}
1000.times { |i|
print("[")
obj = "X"*i
id = obj.object_id
@ids[id] = true
ObjectSpace.define_finalizer(obj,methodfinalizer))
GC.disable
print("{")
@ids.each_key { |id|
# go make sure all objects seem valid (test size)
ObjectSpace._id2ref(id).size
}
print("}")
GC.enable
print("]")
}


I tried this on 1.8.2 and a 1.9 CVS version. In both it calls
the finalizer when GC is disabled (GC.enable==true).

Doesn't happen with the 1.8.2 I have.

I thought
this would be impossible.

AFAIK there is just one guarantee: the finalizer is called *after* the
object is GC'ed and before the interpreter exits. I don't know whether
there are any guarantees that the finalizer is called *immediately* after
the object was collected. So in theory the object could be collected while
GC is enabled and the finalizer can be called some time later when GC is
disabled. Although you might expect finalization to happen immediately
after collection it would not be wise to require this of a runtime system as
it might limit implementation choices in a way that negatively affect
performance.

I also get a recycled object error.

One reason for this could be that you iterate and manipulate the same
collection (the Hash of oids) at the same time. Also, as I pointed out
above - there is a time lag between collection and finalization. During
this time the id is still in the hash but the object is gone already.

In the 1.9 version I'm using, it hangs while trying to finalize
one of the objects (@ids.delete(id) is where it hangs).

You can avoid that by catching the exception:


$ids = {}
$stdout.sync=true

fin = lambda {|oid| $ids.delete oid}

1000.times do
obj = Object.new
$ids[obj.object_id] = true
ObjectSpace.define_finalizer(obj, &fin)
obj = nil

puts "disable: #{GC.disable}"

$ids.each_key do |oid|
begin
raise "mismatch" unless ObjectSpace._id2ref( oid ).object_id == oid
rescue Exception => e
p [oid, e]
end
end

puts "enable: #{GC.enable}"
end

Any clues to what is going on with this GC stuff. If I can't
get this to work, I don't see how anybody reliably use
ObjectSpace._id2ref. BTW, I found this problem in my code
after running tens of thousands of random tests. If you don't
do enough testing, you may not find problems related to garbage
collection.

Kind regards

robert

 

[nobu.nokada]

Hi,

At Sun, 19 Jun 2005 06:32:44 +0900,
Eric Mahurin wrote in [ruby-talk:145790]:

$stdout.sync=true
@ids = {}
1000.times { |i|
print("[")
obj = "X"*i
id = obj.object_id
@ids[id] = true
ObjectSpace.define_finalizer(obj,methodfinalizer))
GC.disable
print("{")
@ids.each_key { |id|
# go make sure all objects seem valid (test size)
ObjectSpace._id2ref(id).size
}
print("}")
GC.enable

You enable GC here, so it can run by next GC.disable. Then an
exception occurs after GC is disabled, therefore GC will be no
longer enabled.

print("]")
}


I tried this on 1.8.2 and a 1.9 CVS version. In both it calls
the finalizer when GC is disabled (GC.enable==true). I thought
this would be impossible. I also get a recycled object error.

Finailizers don't run immediately after the corresponding
objects get collected. The `finalizer' runs at the process
termination.

In the 1.9 version I'm using, it hangs while trying to finalize
one of the objects (@ids.delete(id) is where it hangs).

It's a bug, I'll investigate it.

 

[Eric Mahurin]

Finailizers don't run immediately after the corresponding
objects get collected. The `finalizer' runs at the process
termination.

This is what the documentation says about define_finalizer:

---
Adds aProc as a finalizer, to be called when obj is about to be
destroyed.
---

So, it should run the finalizer right before the object gets
collected. Running the finalizer right after would also work
fine for me. Running it an arbitrary time later doesn't seem
very useful if you plan on using _id2ref. It definitely
doesn't just run finalizers at process termination.

I also tried using Thread.critical= to control the finalizer,
but it doesn't seem to make a difference:

---

#!/bin/env ruby

def finalizer(id)
print("<")
critical = Thread.critical
begin
Thread.critical = true
print(critical ? "F" : "f")
GC.enable and
printf("!GC disabled while finalizing %x!",id)
@ids.delete(id)
ensure
Thread.critical = critical
end
print(">")
end

$stdout.sync=true
@ids = {}
1000.times { |i|
print("[")
obj = "X"*i
id = obj.object_id
@ids[id] = true
ObjectSpace.define_finalizer(obj,methodfinalizer))
GC.start
GC.disable
critical = Thread.critical
begin
Thread.critical = true
print("{")
@ids.each_key { |id|
begin
#finalizer will still run with this instead
#obj2 = "xyzabc"*i
ObjectSpace._id2ref(id).size
rescue RangeError
print(Thread.critical ? "E" : "e")
end
}
print("}")
ensure
Thread.critical = critical
end
GC.enable
print("]")
}

---

I catch the RangeError (recycled object) exceptions and
Thread.critical is still true ("E" gets printed), but the
finalizer still happily runs. I guess GC and the finalizer are
still considered to be part of the same thread even though
functionally it seems like a different one.

I realize that catching RangeError's would fix 99% of the
problems. But, I would still be concerned about the case where
the object would be GCed and then the space reclaimed by an
object that looks just like it. Is it guaranteed that
finalizers of the orginal object be run before its space is
reclaimed?

My goal is for a certain set of objects to maintain which of
those objects are alive. If there are no other references to
them, I want them to be GCed. For the objects still alive I
need to be able to operate on them (see if any are "open" and
also be able to "close" them all). Does anybody have a better
implementation than above? I'm not sure if the above really
works because of the "reclaimed space" case discussed above.
Using WeakRef's may be a solution too. I'm hesitant to trust
much of this stuff until I understand:

a. When are object finalizers called? The docs don't reflect
the behavior.

b. What does GC.disable do?

c. Is GC and/or calling object finalizers considered another
thread? It sure seems like they should.





__________________________________
Yahoo! Mail
Stay connected, organized, and protected. Take the tour:
http://tour.mail.yahoo.com/mailtour.html

 

[ts]

E> I tried this on 1.8.2 and a 1.9 CVS version. In both it calls
E> the finalizer when GC is disabled (GC.enable==true). I thought
E> this would be impossible. I also get a recycled object error.

it's normal : the finalizer is called when ruby stop and it don't care at
this moment if the GC is enabled or not.

E> Any clues to what is going on with this GC stuff. If I can't
E> get this to work, I don't see how anybody reliably use
E> ObjectSpace._id2ref.

it's best to don't use _id2ref


Guy Decoux

 

[nobu.nokada]

Hi,

At Mon, 20 Jun 2005 00:30:59 +0900,
Eric Mahurin wrote in [ruby-talk:145818]:

This is what the documentation says about define_finalizer:

The documentation is inaccurate or improper. Finalizers have
never be called before the destruction.

I catch the RangeError (recycled object) exceptions and
Thread.critical is still true ("E" gets printed), but the
finalizer still happily runs. I guess GC and the finalizer are
still considered to be part of the same thread even though
functionally it seems like a different one.

That guess is right. But I didn't see "E" nor "e" from your
new example, because GC.start does run finalizers. Commenting
the line out, "E" was printed.

I realize that catching RangeError's would fix 99% of the
problems. But, I would still be concerned about the case where
the object would be GCed and then the space reclaimed by an
object that looks just like it. Is it guaranteed that
finalizers of the orginal object be run before its space is
reclaimed?

They are called after all destruction has done.

a. When are object finalizers called? The docs don't reflect
the behavior.

Within evaluation loop after a method implemented in C ended.
It is possible to change it more frequently (e.g., for each
instructions), but I've not measured how it affects the
performance.

b. What does GC.disable do?

Prohibits running GC. If free slots are exhausted while GC is
disabled, the interpreter just tries to allocate new slots with
malloc().

c. Is GC and/or calling object finalizers considered another
thread? It sure seems like they should.

Well, though the current implementation doesn't, I guess so.

 

[Eric Mahurin]

At Mon, 20 Jun 2005 00:30:59 +0900,
Eric Mahurin wrote in [ruby-talk:145818]:

This is what the documentation says about define_finalizer:

The documentation is inaccurate or improper. Finalizers have
never be called before the destruction.

I catch the RangeError (recycled object) exceptions and
Thread.critical is still true ("E" gets printed), but the
finalizer still happily runs. I guess GC and the finalizer are
still considered to be part of the same thread even though
functionally it seems like a different one.

That guess is right. But I didn't see "E" nor "e" from your
new example, because GC.start does run finalizers.
Commenting
the line out, "E" was printed.

Sorry. I meant to comment out the GC.start. Putting in the
GC.start (sometimes I need 2 of them back-to-back) gets it to
work, but I don't want the penalty of running GC all the time.

They are called after all destruction has done.

And before any of that space is reclaimed? It seems like
normal Ruby processing (including memory allocation) can occur
during GC and between GC and finalizers.

Within evaluation loop after a method implemented in C ended.
It is possible to change it more frequently (e.g., for each
instructions), but I've not measured how it affects the
performance.

For future Ruby revisions, I think you should consider ensuring
that after an object is GCed its finalizers are called before
normal ruby processing continues. Having normal ruby code have
to deal with the situation where an object has been GCed but
its finalizers haven't been called seems unnecessary.

Prohibits running GC. If free slots are exhausted while GC
is
disabled, the interpreter just tries to allocate new slots
with
malloc().

So, it works immediately, but this GC.disable could have
occured between an object being GCed and its finalizers being
called. The finalizers could still continue while GC is
disabled.




__________________________________
Discover Yahoo!
Stay in touch with email, IM, photo sharing and more. Check it out!
http://discover.yahoo.com/stayintouch.html

 

[Robert Klemme]

E> I tried this on 1.8.2 and a 1.9 CVS version. In both it calls
E> the finalizer when GC is disabled (GC.enable==true). I thought
E> this would be impossible. I also get a recycled object error.

it's normal : the finalizer is called when ruby stop and it don't care at
this moment if the GC is enabled or not.

They can be invoked before termination of Ruby. But it's guaranteed that
all finalizers are called before exit.

E> Any clues to what is going on with this GC stuff. If I can't
E> get this to work, I don't see how anybody reliably use
E> ObjectSpace._id2ref.

it's best to don't use _id2ref

I beg to differ: you can safely use _id2ref in conjunction with finalizers -
it just has to be done properly. It will go wrong - as demonstrated - if
the same collection that stores ids is used for iterating / querying.

This approach works ok - you'll have to imagine that x contains information
needed for proper cleanup of a Foo instance, for example, an open IO
instance (although I'm sure that will do proper cleanup on finalization):

class Foo
FOOS = {}

def initialize(x)
self.x=x

ObjectSpace.define_finalizer(self) do |oid|
puts "Cleanup of #{oid} with #{FOOS[oid]}"
end
end

def x=(y)
@x = y
FOOS[object_id] = y
end

def x() @x end
end

Kind regards

robert

 

[Robert Klemme]

They can be invoked before termination of Ruby. But it's guaranteed that
all finalizers are called before exit.


I beg to differ: you can safely use _id2ref in conjunction with
finalizers - it just has to be done properly. It will go wrong - as
demonstrated - if the same collection that stores ids is used for
iterating / querying.

To avoid any confusion let me add some clarification: _id2ref has its uses,
but some caution has to be applied:

- don't use it in finalizers

- if objects might have been collected, take precautions (i.e. catch
exceptions)

Also, WeakReferences can be used in many places if you want to be able to
refer an instance but don't want to prevent garbage collection (storing an
oid often serves the same purpose).

The example of course does not show the usage of _id2ref. You can add a
method like this to class Foo:

def self.list
FOO.each do |oid, x|
begin
p ObjectSpace._id2ref(oid)
rescue RangeError => e
puts "#{oid} collected but not finalized: x=#{x.inspect}"
end
end
end

This approach works ok - you'll have to imagine that x contains
information needed for proper cleanup of a Foo instance, for example, an
open IO instance (although I'm sure that will do proper cleanup on
finalization):

class Foo
FOOS = {}

def initialize(x)
self.x=x

ObjectSpace.define_finalizer(self) do |oid|
puts "Cleanup of #{oid} with #{FOOS[oid]}"
end
end

def x=(y)
@x = y
FOOS[object_id] = y
end

def x() @x end
end

Kind regards

robert

 

[Eric Mahurin]

def self.list
FOOS.each do |oid, x|
begin
p ObjectSpace._id2ref(oid)
rescue RangeError => e
puts "#{oid} collected but not finalized:
x=#{x.inspect}"
end
end
end

I assume you meant FOOS above. I fixed it.

Just catching a RangeError is not all you need. You better
make sure the object is finalized and the finalizer removes its
oid from FOOS before the space is reclaimed. Otherwise oid
could refer to a completely new object and you wouldn't detect
it.

On top of that, these finalizers can't be called while this
FOOS.each loop is going on. Otherwise you'd get an error about
the hash being modified while your iterating over it. It is
like the GC and finalizers are in another thread, but
unfortunately you can't control it like a thread (i.e.
Thread.critical=). This is my primary dilemma.

These issues may be very difficult to detect problems with and
you may need 1000's of tests to excite the GC differently to
detect the problem. You must be prepared for this using
_id2ref.

This approach works ok - you'll have to imagine that x contains
information needed for proper cleanup of a Foo instance, for example, an
open IO instance (although I'm sure that will do proper cleanup on
finalization):

class Foo
FOOS = {}

def initialize(x)
self.x=x

ObjectSpace.define_finalizer(self) do |oid|
puts "Cleanup of #{oid} with #{FOOS[oid]}"
end
end

Won't work. This is the exact mistake I made when first trying
to use a finalizer. In the above, you gave define_finalizer a
Proc that has a Binding with direct access to self (a Foo).
This creates an unintended reference to the object in
ObjectSpace/GC. It will never be GCed because of this. It
took me a while to figure this out. The Proc/Method needs to
be defined in a context that doesn't have access to the object
you are trying to put a finalizer on. For this reason, I don't
think define_finalizer should even allow the block form. Also,
a Proc#unbind would be nice to have in this situation.

Also, I assume you'd want to delete the oid entry from FOOS in
this finalizer, right?

def x=(y)
@x = y
FOOS[object_id] = y
end

def x() @x end
end

____________________________________________________
Yahoo! Sports
Rekindle the Rivalries. Sign up for Fantasy Football
http://football.fantasysports.yahoo.com

 

[Robert Klemme]

I assume you meant FOOS above. I fixed it.
Correct.

Just catching a RangeError is not all you need. You better
make sure the object is finalized and the finalizer removes its
oid from FOOS before the space is reclaimed. Otherwise oid
could refer to a completely new object and you wouldn't detect
it.

Probably. My assumption was, that oids are not reused. But I may be
wrong here.

On top of that, these finalizers can't be called while this
FOOS.each loop is going on. Otherwise you'd get an error about
the hash being modified while your iterating over it. It is
like the GC and finalizers are in another thread, but
unfortunately you can't control it like a thread (i.e.
Thread.critical=). This is my primary dilemma.

Hm... Did you try Mutex or Monitor?

These issues may be very difficult to detect problems with and
you may need 1000's of tests to excite the GC differently to
detect the problem. You must be prepared for this using
_id2ref.

This approach works ok - you'll have to imagine that x contains
information needed for proper cleanup of a Foo instance, for
example, an open IO instance (although I'm sure that will do proper
cleanup on finalization):

class Foo
FOOS = {}

def initialize(x)
self.x=x

ObjectSpace.define_finalizer(self) do |oid|
puts "Cleanup of #{oid} with #{FOOS[oid]}"
end
end

Won't work. This is the exact mistake I made when first trying
to use a finalizer. In the above, you gave define_finalizer a
Proc that has a Binding with direct access to self (a Foo).
This creates an unintended reference to the object in
ObjectSpace/GC. It will never be GCed because of this.

Darn, yes you're right!

Two alternatives would be

class Foo
class<<self
alias :_new :new
def new(*a,&b)
obj = _new(*a,&b)
ObjectSpace.define_finalizer(obj) do |oid|
puts "Cleanup of #{oid} with #{FOOS[oid]}"
end
obj
end
end
end

class Foo
def initialize(x)
self.x=x

self.class.instance_eval do
ObjectSpace.define_finalizer(obj) do |oid|
puts "Cleanup of #{oid} with #{FOOS[oid]}"
end
end
end
end

It
took me a while to figure this out. The Proc/Method needs to
be defined in a context that doesn't have access to the object
you are trying to put a finalizer on. For this reason, I don't
think define_finalizer should even allow the block form. Also,
a Proc#unbind would be nice to have in this situation.

Also, I assume you'd want to delete the oid entry from FOOS in
this finalizer, right?

Yes. Sorry for the errors and omissions - it was late already...

def x=(y)
@x = y
FOOS[object_id] = y
end

def x() @x end
end

What is the real world problem you are trying to solve?

Kind regards

robert

 

[ts]

n> The documentation is inaccurate or improper. Finalizers have
n> never be called before the destruction.

The documentation is trying to say this

svg% cat b.rb
#!/usr/bin/ruby
at_exit {
puts "\nThat's all, folks ! You must die"
}

def finalizer(id)
print("<")
GC.enable and
printf("GC disabled while finalizing %x!!!",id)
@ids.delete(id)
print(">")
end

$stdout.sync=true
@ids = {}
1000.times { |i|
print("[")
obj = "X"*i
id = obj.object_id
@ids[id] = true
ObjectSpace.define_finalizer(obj,methodfinalizer))
GC.disable
print("{")
@ids.each_key { |id|
# go make sure all objects seem valid (test size)
ObjectSpace._id2ref(id).size
}
print("}")
GC.enable
print("]")
}
svg%


svg% b.rb
[{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][!
{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{}][{
That's all, folks ! You must die
/b.rb:26:in `_id2ref': 0x200402a4 is recycled object (RangeError)
from ./b.rb:26
from ./b.rb:24:in `each_key'
from ./b.rb:24
from ./b.rb:16:in `times'
from ./b.rb:16
<GC disabled while finalizing 2004c9b4!!!><><><><><><><><><><><><><><><><><><><><><><><><><><




Guy Decoux

 

[Eric Mahurin]

Probably. My assumption was, that oids are not reused. But
I may be
wrong here.

Yep. Other than immediates, I believe an object id is just a
memory location. After an object is GCed, it will want to
reuse the space at some point. The new object may start at
that same location (same object id) or that location may
correspond to the middle of an object.

Hm... Did you try Mutex or Monitor?

I have tried Thread.critical= which is what Mutex and probably
other mutual exclusivity stuff are based on. The problem is
that GC/finalizers is not considered to be in separate threads.

These issues may be very difficult to detect problems with and
you may need 1000's of tests to excite the GC differently to
detect the problem. You must be prepared for this using
_id2ref.

This approach works ok - you'll have to imagine that x contains
information needed for proper cleanup of a Foo instance, for
example, an open IO instance (although I'm sure that will do proper
cleanup on finalization):

class Foo
FOOS = {}

def initialize(x)
self.x=x

ObjectSpace.define_finalizer(self) do |oid|
puts "Cleanup of #{oid} with #{FOOS[oid]}"
end
end

Won't work. This is the exact mistake I made when first trying
to use a finalizer. In the above, you gave define_finalizer a
Proc that has a Binding with direct access to self (a Foo).
This creates an unintended reference to the object in
ObjectSpace/GC. It will never be GCed because of this.

Darn, yes you're right!

Two alternatives would be

class Foo
class<<self
alias :_new :new
def new(*a,&b)
obj = _new(*a,&b)
ObjectSpace.define_finalizer(obj) do |oid|
puts "Cleanup of #{oid} with #{FOOS[oid]}"
end
obj
end
end
end

class Foo
def initialize(x)
self.x=x

self.class.instance_eval do
ObjectSpace.define_finalizer(obj) do |oid|
puts "Cleanup of #{oid} with #{FOOS[oid]}"
end
end
end
end

Nope. Still doesn't work. Try this:

100000.times { obj=Foo.new("hi") }

The memory size just keeps growing and the obj's are not GCed.
In your second solution above obj isn't defined. I'm not sure
what you intended.

The problem is that the Proc's you give to define_finalizer
still have access to the object you are putting the finalizer
on. This time through a local variable (obj) instead of self.

You see why I say that the block form of define_finalizer isn't
useful? And should be removed?

If you've been creating finalizers this way you've probably
never had issues with _id2ref because the object are never
GCed!

What is the real world problem you are trying to solve?

In my cursor package, I create children cursors and need to
keep track of them. But, I don't want to keep a normal ref on
them so that they can't be garbage collected. For example:

child = parent.postion # child holds the current position
parent.position? # any positions/children outstanding?
parent.position?(child) # is this a valid position?
parent.position! # kill positions/children
parent.delete1Next # update all children after this point
child.succ # next position - use with Range

I don't want the user of this package to have to worry about
closing every single child. It would be a pain to have to do
this especially when intermediate expressions can yield a
child. I need to keep track of any outstanding, but I want GC
to get rid of any that aren't used anymore.


Here is some more code that I did some testing on:

#!/bin/env ruby

require 'set'
require 'weakref'

class WeakRefList
def finalizer(id)
__old_status = Thread.critical if @critical
Thread.critical = true if @critical
begin
print("f")
@ids.delete?(id) or raise
ensure
Thread.critical = __old_status if @critical
end
end
def initialize(flags)
@ids = Set.new
@useWeakRef = flags[0].nonzero?
@disable = flags[1].nonzero?
@start = flags[2].nonzero?
@critical = flags[3].nonzero?
@dummy = flags[4].nonzero?
end
def << (obj)
if @useWeakRef
@ids << WeakRef.new(obj)
else
@ids << obj.object_id
ObjectSpace.define_finalizer(obj,
methodfinalizer))
end
dummy = WeakRef.new("") if @dummy
self
end
def each(&block)
GC.start if @start
GC.disable if @disable
__old_status = Thread.critical if @critical
Thread.critical = true if @critical
begin
@ids.to_a.each { |id|
begin
block.call(@useWeakRef ?
id.__getobj__ :
ObjectSpace._id2ref(id)
)
rescue RangeError,WeakRef::RefError
print("e")
@ids.delete(id)
end
}
ensure
Thread.critical = __old_status if @critical
end
GC.enable if @disable
end
end


if __FILE__==$0
class MyString < String; end
weakrefs = WeakRefList.new((ARGV[0]||0).to_i)
$stdout.sync=true
at_exit {puts}
1000.times { |i|
print(".")
weakrefs << MyString.new("X"*i)
weakrefs.each { |o|
MyString==o.class or
raise("not a MyString: #{o.inspect}")
}
}
end



I tried out a bunch of ways to make this WeakRefList. You can
pass in a flags number ORing the options I provided for this
class:

1: use WeakRef instead of simply an object id
2: use GC.disable/GC.enable around code using _id2ref
4: use GC.start before trying _id2ref
8: use Thread.critical= to try to stop the finalizer
16: add a dummy allocation of a WeakRef

I found that only using WeakRef's, GC.start, and dummy
WeakRef's worked for me. In other cases it looks like the
obect_id got reclaimed by another object before the finalizer
was run on the original object. I don't understand why using
WeakRef worked. Looking at the code it looks like an object id
could still get reused before the finalizer is called and it
would still look OK. I think it is just luck because
allocating dummy WeakRef's also worked. Currently, I trust
using GC.start the most. But I've still seen cases where I
have to call GC.start multiple times back-to-back. Anybody
have a better solution for this WeakRefList? I'll want more
methods eventually, but << and each seem sufficient for
testing.





____________________________________________________
Yahoo! Sports
Rekindle the Rivalries. Sign up for Fantasy Football
http://football.fantasysports.yahoo.com

 

[Robert Klemme]

Yep. Other than immediates, I believe an object id is just a
memory location. After an object is GCed, it will want to
reuse the space at some point. The new object may start at
that same location (same object id) or that location may
correspond to the middle of an object.

So we have to look at the sources to get a definitive answer...

I have tried Thread.critical= which is what Mutex and probably
other mutual exclusivity stuff are based on. The problem is
that GC/finalizers is not considered to be in separate threads.

But in that case one of the two might help - the one that isn't reentrant,
would it?

Nope. Still doesn't work. Try this:

100000.times { obj=Foo.new("hi") }

The memory size just keeps growing and the obj's are not GCed.
In your second solution above obj isn't defined. I'm not sure
what you intended.

obj = self - but then again, as you said it doesn't matter whether it's self
or obj that keeps the instance alive.

The problem is that the Proc's you give to define_finalizer
still have access to the object you are putting the finalizer
on. This time through a local variable (obj) instead of self.

You see why I say that the block form of define_finalizer isn't
useful? And should be removed?

It dawns on me. But wait:

Robert@Babelfish2 /c/TEMP
$ ruby gc3.rb > xx

Robert@Babelfish2 /c/TEMP
$ wc -l xx
10001 xx

Robert@Babelfish2 /c/TEMP
$ sort xx|wc -l
10001

Robert@Babelfish2 /c/TEMP
$ fgrep -n end xx
10001:end

Robert@Babelfish2 /c/TEMP
$ cat gc3.rb


def testit
obj = Object.new
ObjectSpace.define_finalizer(obj) {|oid| puts "cleanup #{oid}" }
obj = nil
end


10.times do
1000.times { testit }
GC.start
sleep 1
end

puts "end"

If you comment the "obj = nil" the binding is not modified and the instances
are kept. The way it is here, instances are collected, which you can see
from the "end" statement in the output IMHO. The trick is to use the local
var and modify the binding afterwards.

Also, the oids seem not reused (see the sort output).

In my cursor package, I create children cursors and need to
keep track of them. But, I don't want to keep a normal ref on
them so that they can't be garbage collected. For example:

child = parent.postion # child holds the current position
parent.position? # any positions/children outstanding?
parent.position?(child) # is this a valid position?
parent.position! # kill positions/children
parent.delete1Next # update all children after this point
child.succ # next position - use with Range

I don't want the user of this package to have to worry about
closing every single child. It would be a pain to have to do
this especially when intermediate expressions can yield a
child. I need to keep track of any outstanding, but I want GC
to get rid of any that aren't used anymore.

A solution to tackle the oid reuse issue (if it's an issue) would be to
store something that can verify that the oid belongs to the data stored for
that oid. Unfortunately I can't think of a way ATM...

Kind regards

robert

 

[Eric Mahurin]

threads.

But in that case one of the two might help - the one that
isn't reentrant,
would it?

I'll go look at this again. I all of this had to do with
exclusivity between threads. If I can do it within one thread,
that may work.

def testit
obj = Object.new
ObjectSpace.define_finalizer(obj) {|oid| puts "cleanup
#{oid}" }
obj = nil
end


10.times do
1000.times { testit }
GC.start
sleep 1
end

puts "end"
...

Also, the oids seem not reused (see the sort output).

I find them reused. I modified your code a little to track
what's been used:

@used = {}

def testit
obj = Object.new
id = obj.object_id
@used[id]==false and puts("reused #{id} after finalizing")
@used[id] = true
ObjectSpace.define_finalizer(obj) {|oid|
#puts "cleanup #{oid}"
begin
ObectSpace._id2ref(oid)
puts("reused #{id} before finalizing!!!!!!!!!!")
rescue RangeError
end
@used[oid] = false
}
obj = nil
end

10.times do
1000.times { testit }
GC.start
sleep 1
end


Fortunately, I never see it reuse id's before the object for
that id is finalized.

A solution to tackle the oid reuse issue (if it's an issue)
would be to
store something that can verify that the oid belongs to the
data stored for
that oid. Unfortunately I can't think of a way ATM...

That's kind of what WeakRef does. Unfortunately it is about
20X slower than using simple object ids. Using GC.start with
object ids is much faster.





____________________________________________________
Yahoo! Sports
Rekindle the Rivalries. Sign up for Fantasy Football
http://football.fantasysports.yahoo.com

 

[Eric Mahurin]

I think I finally figured out a solution to making a set/list
of "WeakRef"s. The code is below. It mimics much of the Set
interface. This is an order of magnitude faster than (and less
memory) using WeakRef to implement this.

The main thing I was missing was to check to see if the object
was finalized after a successful _id2ref. If it was finalized
that means I got an object that reclaimed the space my old
object had.

I wasn't sure that you could delete elements of a hash while
iterating over it, but it looks to work fine. Should this be
OK? Anybody see any race conditions or other problems with
this?


#!/bin/env ruby

class WeakRefList
include Enumerable
def finalizer(id);@ids.delete(id >> 1);end
private :finalizer
def initialize(enum=[],&block)
replace(enum.collect(&block))
end
def add(o)
@ids[o.__id__ >> 1] = true
ObjectSpace.define_finalizer(o,methodfinalizer))
self
end
alias << add
def each(&block)
@ids.each_key { |id|
begin
o = ObjectSpace._id2ref(id << 1)
# double-check in case it was finalized
block.call(o) if @ids.include?(id)
rescue RangeError
end
}
nil
end
def clear;@ids = {};self;end
def merge(enum);enum.each{|o|add(o)};self;end
def replace(enum);clear;merge(enum);self;end
def delete(o);@ids.delete(o.__id__ >> 1);self;end
def delete?(o);@ids.delete(o.__id__ >> 1)&&self;end
def empty?;each{return(false)};true;end
def include?(o);@ids.include?(o.__id__ >> 1);end
alias member? include?
def inspect;"#<#{self.class}: #{to_a.inspect}>";end
def subtract(enum);enum.each{|o|delete(o)};self;end
def size;n=0;each{n+=1};n;end
alias length size
end


if __FILE__==$0
require 'benchmark'
class MyString < String; end
weakrefs = WeakRefList.new
$stdout.sync=true
times = Benchmark.measure {
10000.times { |i|
print(".")
obj = MyString.new("X"*rand(i+1))
weakrefs << obj
weakrefs.each { |o|
MyString==o.class or
raise("not a MyString: #{o.object_id}
#{o.inspect}")
}
weakrefs.include?(obj) or
raise("#{obj.inspect} disappeared")
if rand(10).zero?
weakrefs.delete(obj)
!weakrefs.include?(obj) or
raise("#{obj.inspect} didn't delete")
end
}
}
p weakrefs
p weakrefs.size
p weakrefs.empty?
p weakrefs.clear
p weakrefs.size
p weakrefs.empty?
puts(times)
end





__________________________________
Do you Yahoo!?
Make Yahoo! your home page
http://www.yahoo.com/r/hs

 

[Franz Hartmann]

Hello all,

first, thank you all for answering my questions. you have holp me a lot and
i think i understand object oriented programming now.

very recently i have had a very interesting talk with a friend who is
studying psychology and sociology but is also very interested in computers.
we were discussing the influence of sexual identity and orientation on the
patterns of logical thinking and vicy versa. i really felt inspired and have
decided to approach users of several programming languages (ruby, java,
intercal, c++, vb, octane, etc) in this. i am personally convinced there
must be some correlations. anyway, its a running joke that c++ is a "macho
language".

so are there any data concerning the male : female ratio among ruby users?
also, how high is the quote of male and female homosexuals among ruby users?
any data?

Franz

_________________________________________________________________
Ungestört surfen. MSN Toolbar mit Pop-up-Blocker. http://toolbar.msn.de/
Jetzt kostenlos downloaden!

 

[Joe Van Dyk]

Hello all,
=20
first, thank you all for answering my questions. you have holp me a lot a= nd
i think i understand object oriented programming now.
=20
very recently i have had a very interesting talk with a friend who is
studying psychology and sociology but is also very interested in computer= s.
we were discussing the influence of sexual identity and orientation on th= e
patterns of logical thinking and vicy versa. i really felt inspired and h= ave
decided to approach users of several programming languages (ruby, java,
intercal, c++, vb, octane, etc) in this. i am personally convinced there
must be some correlations. anyway, its a running joke that c++ is a "mach= o
language".
=20
so are there any data concerning the male : female ratio among ruby users= ?
also, how high is the quote of male and female homosexuals among ruby use= rs?
any data?
=20
Franz

Well, all Java programmers are definitely gay.

 

[Robert Klemme]

Well, all Java programmers are definitely gay.

Ya don't know what ya talkin' about: we're at least bi.

robert

 

[Glenn Parker]

Well, all Java programmers are definitely gay.

Please please please do not bite the flamebait!