How to access to local variables in enclosing scopes?

[Stefan Lang]

Replying to multiple posts:



I recognize that access to nonlocal vars can be simulated with
instance variables.

Instance variables don't simulate anything.

It does seem weird to allow two different
paradigms that interact with each other in a confusing - or at the
least non-trivial - matter (just look at my confusion).

It seems to me you are confused because you are learning by
posting questions on mailing lists instead of following
some coherent book or introduction material.

I guess my big beef with Ruby is that it is too complex and bloated
with features that often are inconsistent or lack symmetry with each
other. It's too "magical", making it hard to learn exactly how the
language works. From a language design perspective, Ruby really seems
like a mess to me, although with moments of brilliance. Most of the
time, when I'm metaprogramming, I'm guessing and looking at the
output, hoping I got so-and-so quirk correct.

Then stop guessing and learn first. Start with Programming Ruby,
here's an older version online:
http://ruby-doc.org/docs/ProgrammingRuby/

And search ruby-doc.org for more material. Seriously, you sound
like you're parroting some preconceived views about Ruby I've
seen before from other Python programmers, without even trying
to learn.

Stefan

 

[Brian Candler]

There is a another reason why class and def create fresh scopes, related
to the way Ruby resolves method / local variable ambiguity.

One of the benefits of Ruby is its lack of verbosity: in particular no
need to declare variables before using them, and no need to put '()'
after a method call if it doesn't have any arguments. The result is
compact and readable code.

However it leaves an ambiguity: is a bareword like "x" to be interpreted
as a method call, i.e. self.x(), or as referring to a local variable?

This is resolved statically at parse time with a simple rule. If an
assignment expression "x = ..." has been seen earlier in the current
scope (regardless of whether it is actually executed), then a bareword
"x" is a local variable, otherwise it's a method call.

So:

def x
"hello"
end

def foo
if false
x = "bye"
end
puts x # prints nil (local variable, no value assigned)
end

This is unusual, but once you've used it, it makes perfect sense.

Now consider what would happen if 'def' didn't start a new scope.

x = 123

.... 1000 lines of code

def x
"hello"
end

def foo
puts x # 123 instead of hello?
end

The code would behave differently, because of the binding to the 'x'
outside.

For me, the important thing is this: within the method body of 'foo', I
can tell for sure whether x is a local variable or a method call, by
inspecting *only* the code within that method. If it weren't for this
flushing of the scope, then I couldn't be sure without reading the
*whole* of the rest of the source file up to that point.

The same argument applies to a class, and class methods:

# silly example
memoize = 99

... 1000 lines ...

class Foo
extend Flurble
memoize
end

Just by inspecting the body between 'class Foo' and 'end', I can be sure
that memoize must be a method call - perhaps something in module
Flurble, or else some method already added to class Class. Without this
scope flushing, I would have to scan the whole of the rest of the source
file.

This behaviour may not sit comfortably with users of certain other
programming languages. I say that's fine. Ruby is a different language,
and has different idioms. If you are more comfortable with Lisp or
Haskell, then use those instead.

There are of course pros and cons to each way of doing things, but if
you open your mind to new ways of working, you may find the pros are
surprisingly substantial. (And I'm sure the same is true when moving in
the opposite direction too

 

[Robert Dober]

Not particularly convinced with that. If Python handles it with ease

and apparently no runtime performance penalty, I don't see why Ruby
can't do the same.

Of course Python pays a (small) penalty for the lookup.

Closures might be more expensive though because of the local copies. I
however do not know
how this is optimized in the different VM.
Cheers
Robert

 

[David A. Black]

Hi --

I recognize that access to nonlocal vars can be simulated with
instance variables. It does seem weird to allow two different
paradigms that interact with each other in a confusing - or at the
least non-trivial - matter (just look at my confusion).

Everyone gets confused by something along the way in learning a
language. It can't really be used as a one-to-one measure of what's
wrong with the language. I think you're making this all much more
difficult for yourself than it needs to be, perhaps by starting from
the premise that it must be badly designed with no workable solutions.
You may well end up concluding that -- not everyone is destined to
love Ruby -- but I do think there's much more that you can let the
language tell you before you give up on it.

One concept to keep in mind is "self". Instance variables are pegged
to self; their visibility rule is that if self is x, then @var belongs
to x. There are no exceptions to that, and it doesn't interfere with
or really have anything to do with local variables.

I guess my big beef with Ruby is that it is too complex and bloated
with features that often are inconsistent or lack symmetry with each
other. It's too "magical", making it hard to learn exactly how the
language works. From a language design perspective, Ruby really seems
like a mess to me, although with moments of brilliance. Most of the
time, when I'm metaprogramming, I'm guessing and looking at the
output, hoping I got so-and-so quirk correct.

I'd be interested in seeing some of these examples. In my experience,
a great deal of what people find confusing or opaque in Ruby turns out
to be confusing because it is so simple, and because there are so few
exceptions to the rules. I've always been impressed with the
strictness with which Ruby applies a small number of underlying
principles. It's possible that a bit of analysis of some examples
could help you nail some of those.


David

--
Rails training from David A. Black and Ruby Power and Light:
Intro to Ruby on Rails January 12-15 Fort Lauderdale, FL
Advancing with Rails January 19-22 Fort Lauderdale, FL *
* Co-taught with Patrick Ewing!
See http://www.rubypal.com for details and updates!

 

[Yuh-Ruey Chen]

Instance variables don't simulate anything.

I mean that whatever you could do with access to nonlocal vars you
could do with a class with instance variables design. Just a different
programming methodology.

It seems to me you are confused because you are learning by
posting questions on mailing lists instead of following
some coherent book or introduction material.

I actually held off from posting here for a while during my
experimenting and reading of online docs.

Then stop guessing and learn first. Start with Programming Ruby,
here's an older version online:http://ruby-doc.org/docs/ProgrammingRuby/

And search ruby-doc.org for more material. Seriously, you sound
like you're parroting some preconceived views about Ruby I've
seen before from other Python programmers, without even trying
to learn.

Aside: I don't have any "preconceived" views on Ruby. I've heard about
how elegant it is, and indeed I found most of the syntax examples
elegant in some way or fashion. I'm also not primarily a Python
programmer, although I greatly admire how it integrates iteration so
well into the language. I work with all sorts of languages: C, Java,
Python, JS, or whatever else I find fascinating to experiment in.

I do feel that there are several aspects of Ruby that could be
simplified. It feels like Ruby has been influenced a lot by Perl, and
I absolutely loath Perl.

But there are several aspects of Ruby that I do admire: everything
being an object, open classes, how the block syntax meshes with a
function call, the various function calling sugar like omitting
parenthesis (although I think I slightly prefer Python's sugar with
regards to * and **).

 

[Yuh-Ruey Chen]

I understand what you are proposing. It cannot be called a mechanism
for lexical scope, because it's fundamentally inconsistent with the
term. A 'def' method definition has a _different_ scope, not a nested
scope. _By definition_, a scope cannot refer to lexical variables in a
different scope. That's the whole point of lexical local variables.

There's a reason why Lisp moved to lexical scoping and never looked
back. Ruby uses lexical scoping for the exact same reason. It is
better.

I'm not following you. Ruby uses lexical scope? AFAIK, it only does
that constants and not local variables.

My point is that if |def| were to use lexical scope with regards to
local variables, then there needs to be some sort of mechanism to
access those local variables. It could be:

scope.x

or

hey_ruby_interpreter__the_following_identifiers_are_not_methods__they_are_local_variables_from_parent_scopes
x

Again, just make your shared data explicit, like the SHARED struct in
the previous example. That's all. Others will be glad you were upfront
about what is shared. When debugging, it's valuable to know at a glance
what variables can possibly affect state.

Think of the alternative. When tracking down a bug, under your proposal
you would have to check -- recursively! -- for scope.x references to
each and every variable which exists.

Nope, you're clearly not understanding my proposal. |def| currently
introduces a new empty scope. I'm saying that if |def| were changed so
that when first read by the interpreter, it looks at all the local
variables that could be accessed lexically if |def| had lexical scope,
and that within the |def|, there would be a mechanism for accessing
those local variables.

This paragraph does not make sense to me. In my example, the SHARED
OpenStruct instance is not a local variable. It starts with a capital
letter.

Yes, I'm aware that it's not a local variable. I'm saying that using
constants like that cannot adequately simulate the semantics and
restrictions of local variables.

 

[Yuh-Ruey Chen]

There is a another reason why class and def create fresh scopes, related
to the way Ruby resolves method / local variable ambiguity.

One of the benefits of Ruby is its lack of verbosity: in particular no
need to declare variables before using them, and no need to put '()'
after a method call if it doesn't have any arguments. The result is
compact and readable code.

However it leaves an ambiguity: is a bareword like "x" to be interpreted
as a method call, i.e. self.x(), or as referring to a local variable?

This is resolved statically at parse time with a simple rule. If an
assignment expression "x = ..." has been seen earlier in the current
scope (regardless of whether it is actually executed), then a bareword
"x" is a local variable, otherwise it's a method call.

So:

def x
"hello"
end

def foo
if false
x = "bye"
end
puts x # prints nil (local variable, no value assigned)
end

This is unusual, but once you've used it, it makes perfect sense.

Now consider what would happen if 'def' didn't start a new scope.

x = 123

.... 1000 lines of code

def x
"hello"
end

def foo
puts x # 123 instead of hello?
end

The code would behave differently, because of the binding to the 'x'
outside.

For me, the important thing is this: within the method body of 'foo', I
can tell for sure whether x is a local variable or a method call, by
inspecting *only* the code within that method. If it weren't for this
flushing of the scope, then I couldn't be sure without reading the
*whole* of the rest of the source file up to that point.

The same argument applies to a class, and class methods:

# silly example
memoize = 99

... 1000 lines ...

class Foo
extend Flurble
memoize
end

Just by inspecting the body between 'class Foo' and 'end', I can be sure
that memoize must be a method call - perhaps something in module
Flurble, or else some method already added to class Class. Without this
scope flushing, I would have to scan the whole of the rest of the source
file.

This behaviour may not sit comfortably with users of certain other
programming languages. I say that's fine. Ruby is a different language,
and has different idioms. If you are more comfortable with Lisp or
Haskell, then use those instead.

There are of course pros and cons to each way of doing things, but if
you open your mind to new ways of working, you may find the pros are
surprisingly substantial. (And I'm sure the same is true when moving in
the opposite direction too

Brian, I am very aware of this issue. This is exactly why I didn't
propose that identifiers by themselves refer to local vars in
enclosing scopes. It would be incredibly fragile. To use my |scope.x|
syntax example with yours:

x = "hi"

def x
"hello"
end

def foo
if false
x = "bye"
end
puts x # prints nil (local variable, no value assigned)
puts self.sendx) # prints hello
puts scope.x # prints hi
end

 

[ara.t.howard]

I'm not following you. Ruby uses lexical scope? AFAIK, it only does
that constants and not local variables.

if you are having a hard time doing something perhaps you could show
it? one thing ruby does well is get things done with a happy medium
of abstraction. unless something stands in the way of getting things
done simply it's sort of a non-issue don't you think?

cheers.

a @ http://codeforpeople.com/

 

[Yuh-Ruey Chen]

I'd be interested in seeing some of these examples. In my experience,
a great deal of what people find confusing or opaque in Ruby turns out
to be confusing because it is so simple, and because there are so few
exceptions to the rules. I've always been impressed with the
strictness with which Ruby applies a small number of underlying
principles. It's possible that a bit of analysis of some examples
could help you nail some of those.

Okay, there are several little quirks of Ruby that annoy me, but I'll
just a list a couple:

1) When I saw the method! notation for mutation methods, I thought
that was particularly brilliant. But then I noticed that methods are
only postfixed with ! if there was already another method that did the
same thing except on a new copy. It would've been nice of all mutation
methods were postfixed with !. One use case for this would be trapping
all mutation methods of a class for some purpose (e.g. making a read-
only class, or a class that notifies an observer upon mutation).

2) Awkwardness of defining singleton methods. Contrast:

class << obj
def singleton_method
'foo'
end
end

with:

obj.singleton_method = function() {
return 'blah'
}

Of course, YMMV with regards to "elegance".

3) There are public/protected/private modifiers for functions, but not
for modules. (Yes, I'm aware that Ruby isn't Java, and that "private"
means "instance-private" not "class-private".)

4) "Scoping" weirdness (or whatever it's called) with modules and
classes.

class A
class << self
attr_accessor :x

def foo
@x = 20
end
end
end

A.foo
p A.x

B = Class.new
B.instance_eval do
attr_accessor :x

def foo
@x = 40
end
end

B.foo
p B.x # error because self within B.instance_eval is still B, not the
singleton thingamajig. Yet by some magic, |def foo| makes a singleton
method.

So there's really another type of "scope" or whatever it's called
here: module vs. instance. And I have to remember not to trip up on
it.

Anyway, I can list other Ruby gripes, but this is all getting off
topic.

 

[Yuh-Ruey Chen]

(Newgroup apparently ate my reply, but if it didn't, sorry for the
double post)

I'd be interested in seeing some of these examples. In my experience,
a great deal of what people find confusing or opaque in Ruby turns out
to be confusing because it is so simple, and because there are so few
exceptions to the rules. I've always been impressed with the
strictness with which Ruby applies a small number of underlying
principles. It's possible that a bit of analysis of some examples
could help you nail some of those.

Okay, there are several little quirks of Ruby that annoy me, but I'll
just a list a couple:

1) When I saw the method! notation for mutation methods, I thought
that was particularly brilliant. But then I noticed that methods are
only postfixed with ! if there was already another method that did the
same thing except on a new copy. It would've been nice of all mutation
methods were postfixed with !. One use case for this would be trapping
all mutation methods of a class for some purpose (e.g. making a read-
only class, or a class that notifies an observer upon mutation).

2) Awkwardness of defining singleton methods. Contrast:

class << obj
def singleton_method
'foo'
end
end

with:

obj.singleton_method = function() {
return 'blah'
}

Of course, YMMV with regards to "elegance".

3) There are public/protected/private modifiers for functions, but not
for modules. (Yes, I'm aware that Ruby isn't Java, and that "private"
means "instance-private" not "class-private".)

4) "Scoping" weirdness (or whatever it's called) with modules and
classes.

class A
class << self
attr_accessor :x

def foo
@x = 20
end
end
end

A.foo
p A.x

B = Class.new
B.instance_eval do
attr_accessor :x

def foo
@x = 40
end
end

B.foo
p B.x # error because self within B.instance_eval is still B, not the
singleton thingamajig. Yet by some magic, |def foo| makes a singleton
method.

So there's really another type of "scope" or whatever it's called
here: module vs. instance. And I have to remember not to trip up on
it.

Anyway, I can list other Ruby gripes, but this is all off topic.

 

[Yuh-Ruey Chen]

if you are having a hard time doing something perhaps you could show
it? one thing ruby does well is get things done with a happy medium
of abstraction. unless something stands in the way of getting things
done simply it's sort of a non-issue don't you think?

cheers.

a @http://codeforpeople.com/

I have already posted an example:

def foo
a = 10
def bar
# should somehow be able to access a
end
bar
end
foo
# here, we should not be able to access foo's a, but if there is
another a in scope, we can access that

Of course, afterwards I found out that Ruby doesn't really have nested
procedures, even if it tricks you into thinking that has them. So it's
kinda a moot point now.

 

[Gary Wright]

Okay, there are several little quirks of Ruby that annoy me, but I'll
just a list a couple:

1) When I saw the method! notation for mutation methods, I thought
that was particularly brilliant.

The method! notation is not a notation for mutation methods. That
particular syntax is strictly a stylistic convention. It has no
inherent semantics from the language perspective. The convention
is that the bang method is more 'dangerous' than the 'regular'
non-bang method. The meaning of 'dangerous' is completely relative
to the the method and class context. Sometimes it means that the
object will be mutated but sometimes it means something else. Since
the convention is based on comparing two methods (m and m!) it breaks
the convention to define m! without also defining m.

2) Awkwardness of defining singleton methods. Contrast:

class << obj
def singleton_method
'foo'
end
end

with:

obj.singleton_method = function() {
return 'blah'
}

Try:

def obj.singleton_method
'foo'
end

 

[ara.t.howard]

I have already posted an example:

def foo
a = 10
def bar
# should somehow be able to access a
end
bar
end
foo
# here, we should not be able to access foo's a, but if there is
another a in scope, we can access that

Of course, afterwards I found out that Ruby doesn't really have nested
procedures, even if it tricks you into thinking that has them. So it's
kinda a moot point now.

it's not moot, it's easy and you *can* have nested procedures:



cfp:~ > cat a.rb
def foo
a = 10
this = self.class.methodfoo)
this.singleton_class{ define_methodbar){ a } }
this.bar()
end


p foo()
p bar()


BEGIN {
def singleton_class &b
sc =
class << self; self end
b ? sc.module_eval(&b) : sc
end
}




cfp:~ > ruby a.rb
10
a.rb:11: undefined method `bar' for main:Object (NoMethodError)



here 'bar' is a static method (attached to the method object *itself*)
of foo. of course it has access to the local variables. this
construct makes even methods which take blocks trivial


cfp:~ > cat a.rb
def foo &block
a = 10
this = self.class.methodfoo)
this.singleton_class{ define_methodbar){ block.call + a } }
this.bar()
end


p foo{ 32 }


BEGIN {
def singleton_class &b
sc =
class << self; self end
b ? sc.module_eval(&b) : sc
end
}




cfp:~ > ruby a.rb
42




i'll accept that you might not like the syntax, but as someone who
coded perl professionally for 5 years i know exactly the pain that
'my' can inflict - here is one of the last perl scripts i wrote

http://codeforpeople.com/lib/perl/ozone/ozone.pl

i save it to remind myself of the insanity you have to jump through to
provide data encapsulation, which 30 years of software design has
proven to be a 'good thing' (recall that 'my' is/was heavily
reccomended by all the perl style guids (at the time). in any case,
we do the same with ml/lisp etc - only insane people don't use 'let'
and then only with good reason. all of us who remember that recall
using a debugger to figure out where that damn variable came from (i
have never, in 7 years, needed a debugger with ruby). (OT - i *did*
learn a ton about these concepts from http://www.manning.com/conway -
highly recommended for people that want to know what it is to *make*
objects from scratch using little else but scoping rules)

i totally see that you are correct, but it really seems like you are
arguing for something as provably arcane and slippery as GOTO - it
might make certain programs shorter, but it makes *all* programs more
difficult to maintain.

in the end it seems vastly simpler to *provide* a scope when it's
needed, rather than to always be wondering what the scope actually
is. my brain hates stack frames....

the key to seeing how to do this is to realize blocks *always* cary
the scope/closure in ruby, you can always do this:

cfp:~ > cat a.rb
def foo &block
b = block.call
a = eval('a', block)
a + b
end

a = 32
p foo{ 10 }


cfp:~ > ruby a.rb
42


and you can always use Kernel.binding, which ERB certainly does.


as matz says, "it's no harm to learn new programming languages". or,
as a british guy i used to now used to say: "suck it and see"

good luck!




a @ http://codeforpeople.com/

 

[David A. Black]

Hi --

Okay, there are several little quirks of Ruby that annoy me, but I'll
just a list a couple:

1) When I saw the method! notation for mutation methods, I thought
that was particularly brilliant. But then I noticed that methods are
only postfixed with ! if there was already another method that did the
same thing except on a new copy. It would've been nice of all mutation
methods were postfixed with !. One use case for this would be trapping
all mutation methods of a class for some purpose (e.g. making a read-
only class, or a class that notifies an observer upon mutation).

The ! methods are not specifically mutation methods. See:
http://dablog.rubypal.com/2007/8/15/bang-methods-or-danger-will-rubyist
for an explanation of what they signify (and why it's cool, and why
putting ! on Array#pop and so forth would not be cool).

2) Awkwardness of defining singleton methods. Contrast:

class << obj
def singleton_method
'foo'
end
end

with:

obj.singleton_method = function() {
return 'blah'
}

Of course, YMMV with regards to "elegance".

When writing instance methods, I like to be inside a code definition
block, without regard to whether the class is a singleton class or
not. If you don't, you can always do:

def obj.some_method

but that gets tiresome if you've got a lot of them. class << obj just
puts you in the class block, and then you proceed as you always do
with instance methods. There's really no need for a whole separate
apparatus; the class/object model is already designed to give you the
interface to singleton behavior that you need.

3) There are public/protected/private modifiers for functions, but not
for modules. (Yes, I'm aware that Ruby isn't Java, and that "private"
means "instance-private" not "class-private".)

4) "Scoping" weirdness (or whatever it's called) with modules and
classes.

class A
class << self
attr_accessor :x

def foo
@x = 20
end
end
end

A.foo
p A.x

B = Class.new
B.instance_eval do
attr_accessor :x

def foo
@x = 40
end
end

B.foo
p B.x # error because self within B.instance_eval is still B, not the
singleton thingamajig. Yet by some magic, |def foo| makes a singleton
method.

It's by decree. It's extremely rare to use def inside a code block, so
the idea, as I understand it, is that this is a way to introduce the
ability to do the almost only imagineable such scenario: namely,
creating a singleton method. The other imagineable similar scenario
is represented by class_eval (aka module_eval).

It's also related to the fact that def is a keyword, not a method. In
your example, attr_accessor is simply a method call without an
explicit receiver, which is always sent to "self" (B, in this case).
There's no complexity there. def, however, doesn't have a receiver;
it's a different kind of construct. So decisions always have to be
made about what its effect will be in a given context. Its treatment
in instance_eval is definitely special-cased, but if it weren't (if
def just created an outer-level method, as it would in a
non-special-cased code block), it would never be used (just as def
never is in normal code blocks).

So it's just a bit of quasi-vacant semantic space with a possible
use-scenario, and it got engineered accordingly.

So there's really another type of "scope" or whatever it's called
here: module vs. instance. And I have to remember not to trip up on
it.

In general you have to remember not to trip up on all the idioms of
any language It sounds like you're disgruntled about Ruby
specifically because you're finding things that you have to learn, and
that you can't just slide into based on what you've done in other
languages. That's correct: Ruby really is its own language, and you do
have to learn its idioms and toolsets.

I would second what Ara said, namely that it would be interesting to
see some actual things you're trying to do, and points at which you
feel that Ruby's scoping rules and so forth are stopping you.


David

--
Rails training from David A. Black and Ruby Power and Light:
Intro to Ruby on Rails January 12-15 Fort Lauderdale, FL
Advancing with Rails January 19-22 Fort Lauderdale, FL *
* Co-taught with Patrick Ewing!
See http://www.rubypal.com for details and updates!