Blocks, serailizing, and "configure, don't integrate"

[Hugh Sasse Staff Elec Eng]

I seem to have run into a conceptual barrier, and it has caused me
to rethink over ideas I've had about procs and blocks in the
past....

I'm trying to build a gui based system -- the application doesn't
matter for this discussion -- and I'd like to use Tk because it is
more dynamic that the C++ based kits. So I think about the ideas in
The Pragmatic Programmer about Providing Options, and Configure,
Don't Integrate, and it seems a good idea to make some of the
windows pickup their controls from a file.

That's all well and good, and things like Marshal or Yaml handle
those details nicely. But my widgets have callbacks, and in Ruby/Tk
those are proc objects which can't be marshalled. Ok, I can save
the block as a string and load it back in and do eval on it, or
instance eval or some such, but then the block doesn't get checked
for correctness very early because it is no longer code. Also this
opens the whole thing up for abuse: the usual "deliver us from eval"
issues of importing code from files and trusting it. I'm going to
be encouraging people to tweak these configuration files, so I'd be
inviting them to insert code, which is bad. I accept that having the
ruby code on the machine is a weakness anyway, but most users won't
poke around in that.

I could introduce a mini language, so that my procs defined
elsewhere get picked up, but that feels like an abstraction too far,
if I try to make it flexible enough to be useful it will be too big
to manage. I think, anyway.

Is there another way to meet these conflicting goals of flexibility
and some security?


The second part of this is:

I find myself wanting blocks to be real objects before they become
procs. I'm trying to think this out but it feels like grasping wet
soap -- I can't quite get a hold on the idea. I'm thinking
something like this:

We can't marshal procs because they carry external state.
We can't marshal blocks or lambdas for the same reason.

When you marshal an object you give it enough information to be
re-constructed, possibly minus a few features (things that don't get
dumped).

Can we marshal a proto-block, that when loaded in a context
causes a block to come into being in that context, without
opening ourselves up to a full eval? Then we could do something
like Block.new(Marshal.load(...)) to get a block, to pass to
proc or whatever. Possibilities that this would have include
warnings on the code within the block before/after it is
serialized/restored, and formatting as some AST so it is less
prone to casual hacking when serialized [caveats about security
through obscurity apply].

I'm thinking that simplifying this would enable greater utility, as
is true for all abstractions, as well as aid debugging.

This is probably hard for me to think about clearly because there's
a big flaw in it....

Thank you,
Hugh

 

[Robert Klemme]

Hmmm,

as the writing and reading instance is the same, what about writing a
class that acts as a block proxy and that recreates the code after loading
the way you described but with a single difference: the code is encryptet.
Of course, with reasonable effort it's still easy to play foul - but at
least you can make sure that noone changes serialized code accidentally or
on purpose and then simply loads it.

Another thing in the direction of your mini-language could be this: you
write a class that describes the callback relationship. Callbacks
typically just forward an event to somewhere (or you can enforce this
rule). Then the callbacks are generated from the connection information
(source instance, source event, target instance, target method to invoke).

Just my 0.02EUR...

Kind regards

robert

 

[Hugh Sasse Staff Elec Eng]

Hmmm,

as the writing and reading instance is the same, what about writing a
class that acts as a block proxy and that recreates the code after loading
the way you described but with a single difference: the code is encryptet.

Yes, that would solve one of the problems. I could keep the
key in the code on the previously given assumption that people
won't poke about in the code so much as in config files...

Of course, with reasonable effort it's still easy to play foul - but at
least you can make sure that noone changes serialized code accidentally or
on purpose and then simply loads it.

and with suitable checksums I can make it a bit more difficult to
forge.

Another thing in the direction of your mini-language could be this: you
write a class that describes the callback relationship. Callbacks
typically just forward an event to somewhere (or you can enforce this
rule). Then the callbacks are generated from the connection information
(source instance, source event, target instance, target method to invoke).

Yes, that would do it, though I couldn't see how to do that before.
Now I think [having gone AFK for coffee] that there might only be a
small number of callback procs I'm likely to need anyway, so I could
represent them as symbols, and never serialize procs at all.

Just my 0.02EUR...

Thank you.

Kind regards

robert

Hugh

 

[leon breedt]

Hi Hugh,

I don't see much benefit to be gained from serializing the callback
procs themselves. I would rather go for the approach where you
serialize an identifier associated with the callback.

A good example of what I'm getting at is Glade. Its a UI description
format (XML) for Gtk/Gnome. When you load in the Glade file, you get
back a container that holds all the Gtk UI object instances. The Glade
file only stores the callback name (they call it signals); in your
code you then dispatch to the appropriate method based on that name.

The Ruby Glade bindings can be found together with the Ruby Gnome2 bindings:

http://ruby-gnome2.sourceforge.jp/

 

[Phil Tomson]

The second part of this is:

I find myself wanting blocks to be real objects before they become
procs. I'm trying to think this out but it feels like grasping wet
soap -- I can't quite get a hold on the idea. I'm thinking
something like this:

We can't marshal procs because they carry external state.
We can't marshal blocks or lambdas for the same reason.

When you marshal an object you give it enough information to be
re-constructed, possibly minus a few features (things that don't get
dumped).

Can we marshal a proto-block, that when loaded in a context
causes a block to come into being in that context, without
opening ourselves up to a full eval? Then we could do something
like Block.new(Marshal.load(...)) to get a block, to pass to
proc or whatever. Possibilities that this would have include
warnings on the code within the block before/after it is
serialized/restored, and formatting as some AST so it is less
prone to casual hacking when serialized [caveats about security
through obscurity apply].

I'm thinking that simplifying this would enable greater utility, as
is true for all abstractions, as well as aid debugging.

Have you looked at nodewrap?
http://rubystuff.org/nodewrap/

I also sometimes think that it would be useful to have a Block class and
that code blocks would be instances of Block... but I suspect that would
be too big of a change to Ruby at this point.

Anyway, if there were a Block class it might work something like:

block = { puts "my code block" }
puts block.class #=> Block
puts block.to_s #=> "{ puts \"my code block\"}"
p = block.to_proc

def useblock(&b) #could take Block object and convert to proc
b.call
end

#Then useblock could be called in two ways:

useblock block
useblock { puts "use this block" }


But then, maybe there wouldn't be enough difference between a Block and a
Proc to make this kind of change worthwhile. Also, while in this simple
case you could easily marshal 'block', other cases where a block needed to
carry around more context (references to in-scope variables, etc. ) would
still seem to be problematic unless you were to say that Block objects do
not carry around any context, but that they 'inherit' the context in which
they are loaded (the problem then being that if the Block refers to some
variables that it expects to have in scope but aren't in the scope where
the marshalled Block is loaded).

Phil

 

[Hugh Sasse Staff Elec Eng]

Have you looked at nodewrap?
http://rubystuff.org/nodewrap/

No, I'll take a look. Thank you.

I also sometimes think that it would be useful to have a Block class and
that code blocks would be instances of Block... but I suspect that would
be too big of a change to Ruby at this point.

Oh, certainly too big for 1.x but maybe 2 could have it if it is
worthwhile...

Anyway, if there were a Block class it might work something like:

[example elided]
That was how I was thinking.

But then, maybe there wouldn't be enough difference between a Block and a
Proc to make this kind of change worthwhile. Also, while in this simple

well, a Block would be the syntactic structure, the proc would be it
brought to life...

case you could easily marshal 'block', other cases where a block needed to
carry around more context (references to in-scope variables, etc. ) would
still seem to be problematic unless you were to say that Block objects do
not carry around any context, but that they 'inherit' the context in which
they are loaded (the problem then being that if the Block refers to some

Yes, I was thinking the latter.

variables that it expects to have in scope but aren't in the scope where
the marshalled Block is loaded).

That can apply to any loaded object really, it may possibly contain
information irrelavent to its context, but that person doing the
loading should know what context to load it into.

Phil

Thank you,
Hugh

 

[Hugh Sasse Staff Elec Eng]

Hi Hugh,

I don't see much benefit to be gained from serializing the callback
procs themselves. I would rather go for the approach where you
serialize an identifier associated with the callback.

A good example of what I'm getting at is Glade. Its a UI description

Yes, I was beginning to think something like this but it would be
good to have an example to follow. Thank you.
Hugh