memory sizing an object or array?

[jonT]

Hi,

I'm trying to write a queue that is memory limited to foo, i.e. if you
add items that push the queue size > foo, then it drops items off the
end until the memory size drops under foo.

Is there any way in Ruby of obtaining either an Array or Object's
memory use, or failing that, any objective metric that'll do instead?

Cheers,

jonT

 

[Robert Klemme]

Hi,

I'm trying to write a queue that is memory limited to foo, i.e. if you
add items that push the queue size > foo, then it drops items off the
end until the memory size drops under foo.

Is there any way in Ruby of obtaining either an Array or Object's
memory use, or failing that, any objective metric that'll do instead?

Array#size will give you the number of elements in the array.

Btw, usually a bounded queue is implemented such that adding an element
to a filled queue blocks the calling thread.

If you just want to restrict memory then WeakReference may also help
you. It depends...

Kind regards

robert

 

[jonT]

Essentially I have a history of items stored in an Array and i'd like
to evict older items when newer ones are added, after I reach some max
size (maybe 100MB).

The items themselves vary massively in size though, so basing eviction
on Array length wouldn't work.

I had a look at Weak Reference but it's not really appropriate: the max
size would be based on the user's system rather than my chosen limit.

 

[Robert Klemme]

Essentially I have a history of items stored in an Array and i'd like
to evict older items when newer ones are added, after I reach some max
size (maybe 100MB).

John, it's very difficult to determine the size of an object in Ruby.
The question is how far do you travel the object graph started at the
instance you want to determine the size for? It's easy if your instance
just references basic types (e.g. numbers, true, false...) but it gets
complicated even for String because several instances may share memory.
Trying to determine the size is slow, inaccurate and error prone.

Using the collection's size is the easiest and probably most efficient
approach here.

The items themselves vary massively in size though, so basing eviction
on Array length wouldn't work.

If you think about determining the process's memory footprint and then
delete until you go below your limit won't work either because it's
fairly inaccurate, the GC doesn't kick in automatically, you don't know
which objects use the memory (could be in a completely different part of
the program) etc.

I had a look at Weak Reference but it's not really appropriate: the max
size would be based on the user's system rather than my chosen limit.

What's wrong with that?

Cheers

robert

 

[jonT]

John, it's very difficult to determine the size of an object in Ruby.
The question is how far do you travel the object graph started at the
instance you want to determine the size for? It's easy if your instance
just references basic types (e.g. numbers, true, false...) but it gets
complicated even for String because several instances may share memory.
Trying to determine the size is slow, inaccurate and error prone.

Ah! I wasn't aware that Ruby did that (though it is rather
intelligent...). My data structure is purely hierarchical in nature, so

I had been hopeful.

Using the collection's size is the easiest and probably most efficient
approach here.

And it's the approach I think I'll have to take.

What's wrong with that?

I've written a Ruby shell (much better than Irb) that stores a complete

history of executions (inputs and result). That history is stored
across
sessions using mashaling (now you see why I want a queue with limited
size)...

[Of course the downer of my approach is that you can define a class /
method
in one session, the marshaling won't bring it back. Worst still if you
had
an instance of it.Still haven't find an elegant solution to this]

Anyway, using weak references:

(1) doesn't give me too much control, e.g. to ensure that history
entries
are deleted oldest first.

(2) means that I can't choose to set the history to an arbitrary size.

Cheers,
jonT