Macros in Ruby

[Yohanes Santoso]

The macro version has made the code much less clear in my mind, which is
precisely the argument against macros in ruby.

Actually, the example Jesse gave was probably bad. A macro, like any
other language construct, can be created to enhance clarity or
obfucate. Thus, one should not make a hasty judgement based on that
one example.

How about this example (which I hope is a better example of
easy-with-macro,-hard-(impossible?)-without):

Suppose we want to do self-documenting methods (not source file),
similar to docstring in emacs:

An implementation with macro would look something like:

def foo(arg)
"This method will return <code>arg</code>"
arg
end

Sure, you can implement this without macro (trivial implementation):

DOCSTRINGS={}
def def_with_docstring(klazz, method_name, docstring, arg_spec, body)
id = "#{klazz.to_s}##{method_name}"
DOCSTRINGS[id] = docstring
klazz.module_eval(%{def #{method_name}(#{arg_spec})
#{body}
end}, __FILE__, 9999) # wish that there isn't
# anything on line 9999.
end

def_with_docstring(Object, "foo", "returns arg", "arg", "arg")

Object.new.foo("hi") # => "hi"

Yes, I am aware you can do more sophisticated implementation so you
can make the above nicer. Yet, as nice as it will be, it won't be as
simple as the example with macro; there will be quirkiness caused by
the language syntax.

For example, rake has this in its documentation: "NOTE: Because of a
quirk in Ruby syntax, parenthesis are required on rule when the first
argument is a regular expression."

Another example: I am not even sure you can turn the above example to
take lambdas as the body. Being able to take lambdas would be nice as
you can take advantage of your editor's syntax highlighting and
auto-indent.


YS.

 

[Gavin Sinclair]

Well, what are the differences between blocks and proc objects? As far
as I can tell the only difference is that proc objects are explicitly
declared in method argument lists and are first class objects. Blocks,
on the other hand, are not listed in the method list (unless you use
&block to create a proc) and called using yield.

How does that make the language more expressive?

Which do you prefer?

(1..10).map { |n| n ** 2 }

(1..10).map( lambda { |n| n ** 2 } )

(If the latter, then I guess nothing further can be said.)

Blocks are something that are used _all the time_ in Ruby. Perhaps
more than you appreciate. They are semantically the same as explicit
blocks, but the syntax is optimised for the common use. And it's not
limiting:

square = lambda { |n| n ** 2 }
(1..10).map(&square)

Please remember Joel's "percent-case" solution, extensively using
blocks. That was nothing if not expressive. Ruby can be bent into
1000 different shapes, primarily thanks to blocks. Those shapes would
be possible without them (just use explicit lambdas) but not nearly as
appealing, and therefore people wouldn't bother, and the world would
be poorer.

Implicitly passing arguments seems actively bad and using "yield lhs
rhs" seems worse than using "compare.call lhs rhs".

Same comments as above apply. "Yield" is the perfect English word to
describe the behaviour.

For some words from the source, I recommend you look up Matz's
interviews with Bill Venners on artima.com.

Ruby is a lot of things, but it's not a LISP/Dylan wanna-be. You do
yourself no favours by insisting on looking at it from that angle.

Cheers,
Gavin

 

[Jim Weirich]

Yes, a beautiful example of the flexibility of Ruby. But I don't see
how having both blocks and proc objects improves Ruby's expressiveness.

It doesn't. You could drop the implicitly passed blocks and always deal
with an explicit &block argument and not miss a bit of expressiveness
There is a performance difference difference however, and I suspect that
is the real reason why the yield syntax is supported (i.e. to avoid the
overhead of reifying blocks into full fledged objects). Perhaps a
future optimization will make the &block syntax as efficient as the
implicit yield syntax. When that happens, the "block" concept can
retreat to the interior of the interpretor (where it belongs) and not
haunt developers with this weird "almost a proc, but not quite"
semantics. At least that's my view.

Actually what surprises me most about Ruby's one-block per method
technique is how flexible it turns out to be. One would think that
having the ability to pass multiple blocks would be big improvement, but
I suspect that the improvement in flexibility is merely incremental.

What surprises me about Ruby is the about of expressiveness you get,
even though you are limited to only one block per method call. You
would think that having the ability to pass multiple closures to a
method would be a big improvement,

Plenty of languages have blocks, but fewer languages could build
declarative structures as nicely as Joel was able to do. You also need
instance_eval and it helps a lot if the syntax is fairly free-form so
you can omit parenthesis in function calls.

All those help. I've used both Lisp and Forth and really admire their
ability to create tailored languages for a domain. I think Ruby comes
close to their level of flexibility, but does it with different tools
that either Forth or Lisp (i.e. macros and run-time code generation).

 

[Robert Klemme]

The macro version has made the code much less clear in my mind, which is
precisely the argument against macros in ruby.

Actually, the example Jesse gave was probably bad. A macro, like any
other language construct, can be created to enhance clarity or
obfucate. Thus, one should not make a hasty judgement based on that
one example.

How about this example (which I hope is a better example of
easy-with-macro,-hard-(impossible?)-without):

Suppose we want to do self-documenting methods (not source file),
similar to docstring in emacs:

An implementation with macro would look something like:

def foo(arg)
"This method will return <code>arg</code>"
arg
end

Sure, you can implement this without macro (trivial implementation):

DOCSTRINGS={}
def def_with_docstring(klazz, method_name, docstring, arg_spec, body)
id = "#{klazz.to_s}##{method_name}"
DOCSTRINGS[id] = docstring
klazz.module_eval(%{def #{method_name}(#{arg_spec})
#{body}
end}, __FILE__, 9999) # wish that there isn't
# anything on line 9999.
end

This will print wrong file names for all methods not defined in that file.
I'd prefer something like this:

module DocMix
def doc(item, text)
(@docs ||= {})[item.to_s] = "#{item} defined in #{caller[0]}: #{text}"
end

def get_doc(item)
@docs ? @docs[item.to_s] : nil
end
end

class Class ; include DocMix end
class Module ; include DocMix end

class Foo
doc :bar, "Method that prints hallo 'name'"
def bar(name)
puts "hallo #{name}"
end
end

puts Foo.get_doc( :bar )

module Test
doc :"Test.bar", "Method that prints hallo 'name'"
def self.bar(name)
puts "hallo #{name}"
end
end

puts Test.get_doc( :"Test.bar" )

Of course one could easily add argument names etc. Personally I never
needed Macros until now and I've yet to see a use case where I would
really need them.

def_with_docstring(Object, "foo", "returns arg", "arg", "arg")

Object.new.foo("hi") # => "hi"

Yes, I am aware you can do more sophisticated implementation so you
can make the above nicer. Yet, as nice as it will be, it won't be as
simple as the example with macro; there will be quirkiness caused by
the language syntax.

For example, rake has this in its documentation: "NOTE: Because of a
quirk in Ruby syntax, parenthesis are required on rule when the first
argument is a regular expression."

Another example: I am not even sure you can turn the above example to
take lambdas as the body. Being able to take lambdas would be nice as
you can take advantage of your editor's syntax highlighting and
auto-indent.

You could do that. But you couldn't define methods that receive a body as
lamba that way. That's a restriction of define_method which you would
have to use in this case.

Regards

robert

 

[Kristof Bastiaensen]

Hi,


shouldn't that be '(1..10).map( &lambda { |n| n ** 2 } )'?

I prefer a light-weight syntax for something as commonly used as anonymous
functions. On the other hand, I still don't know why {|n| n ** 2} can't
simply create a proc object.

It can! Wether or not a proc object gets created depends on the
called method. If that method uses yield, then no object is
created. If however it captures a block using the '&' operator,
then a Proc object gets created.

On the other hand, a Proc created this way is different from one
created using Proc.new or lambda.

The difference which I can think of are:
- Argument passing, a block isn't strict about the arguments
it gets passed, (i.e. it can take any argument). A Proc
object in contrast is strict (like methods). The reason I
can think of is that it is often usefull not to get all arguments
in a block, i.e. 3.times{ puts "Hi, world!" }

- The place where return goes to: in a block 'return' returns
from the method in which it is defined. In a lambda (Proc)
it returns from the lambda expression.

Yes, a beautiful example of the flexibility of Ruby. But I don't see how
having both blocks and proc objects improves Ruby's expressiveness.

Well, it is very expressive to be able to 'return' from a block.
That 's one of those thing I find uncomfortable in lisp and
scheme. Sure, there is a good reason there is no 'return' in
scheme: every statement is an expression, it wouldn't know where
to return from.
I find it much more practical in Ruby, (with some gotha's, like
LocalJumpError).

Also block make it possible to do 3.times { puts "hello"} as
explained above. That's more expressive than having to type
3.times { |i| puts "hello"}

Sure, it's a good word to describe what the interpreter will do when it
hits that line, but it completely fails to describe the more important
semantic behavior. And if the language already has proc objects why
bother supporting yield?

-- Jesse

I believe yield is there historically. In early versions of Ruby
yield used to be the only way to pass arguments to blocks. Then
in was unified with Proc objects. I may be totally mistaken about
this.

The other thing is performance. Creating a proc object is quite
expensive.

Also, but perhaps less important, it is a little easier to
type yield, than to create block, and use block.call every
time.

I hope this explains a little why I find Ruby's way very
expressive. I believe Ruby isn't semantically as clean as
Scheme or SmallTalk. (I don't think lisp is semantically very
clean). However for me it has the right mix of being consistent
and being practical.

Regards,
KB

 

[Kristof Bastiaensen]

What surprises me about Ruby is the about of expressiveness you get, even
though you are limited to only one block per method call. You would think
that having the ability to pass multiple closures to a method would be a
big improvement,

That's possible, but not in a block passing way (if that is
what you mean). You can always create a closure using
lambda and pass it to the method like any other value.

Regards,
KB

 

[gabriele renzi @ google]

def_with_docstring(Object, "foo", "returns arg", "arg", "arg")

Object.new.foo("hi") # => "hi"

Yes, I am aware you can do more sophisticated implementation so you
can make the above nicer. Yet, as nice as it will be, it won't be as
simple as the example with macro; there will be quirkiness caused by
the language syntax.

not that much, did you see the MetaTags system or the various
implementations of docstrings?

Something on the lines of:

doc 'this things does foo'
def foo
...
end

is quite trivial to do.

For example, rake has this in its documentation: "NOTE: Because of a
quirk in Ruby syntax, parenthesis are required on rule when the first
argument is a regular expression."

IIRC this is related to bison not being good enough

 

[Robert Klemme]

Hi,



shouldn't that be '(1..10).map( &lambda { |n| n ** 2 } )'?


It can! Wether or not a proc object gets created depends on the
called method. If that method uses yield, then no object is
created. If however it captures a block using the '&' operator,
then a Proc object gets created.

On the other hand, a Proc created this way is different from one
created using Proc.new or lambda.

The difference which I can think of are:
- Argument passing, a block isn't strict about the arguments
it gets passed, (i.e. it can take any argument). A Proc
object in contrast is strict (like methods). The reason I
can think of is that it is often usefull not to get all arguments
in a block, i.e. 3.times{ puts "Hi, world!" }

I don't know what exactly you mean by this. Argument passing is the same
for blocks and procs IMHO. Can you explain the difference?

- The place where return goes to: in a block 'return' returns
from the method in which it is defined. In a lambda (Proc)
it returns from the lambda expression.

Ah, thanks for reminding me! That's IMHO the single most crucial
difference between blocks and procs created from blocks vs. stand alone
lambdas / procs. And it's a really important difference!

Kind regards

robert

 

[Yohanes Santoso]

not that much, did you see the MetaTags system or the various
implementations of docstrings?

Something on the lines of:

doc 'this things does foo'
def foo
...
end

More like:
class_info <<-DOC
!Class: MyClass
!Desc: This class does something

!attr a: A: The "A" attribute
!attr b: B
DOC
class MyClass
...
end

and this is what the doc has to say about the '!Class' tag:

!Class - A string indicating the fully-qualified class name. This
is a required field.


Robert's example is about similar to what MetaTags does. Thus, they
both make one to repeat one self; you have to say: "this docstring is
for this method of this class", even if the information is available
nearby. I think this counts as quirkiness.

My point was not whether it is possible to do something in Ruby. The
point was to show Rando Christensen that macro can be used to enhance
clarity as well as to obfuscate, and that some things are easier done
with macro then without. I think my example shows that with macro, you
can extend the language naturally (instead of having to repeat
yourself if done within the language's limitation).

IIRC this is related to bison not being good enough

Somewhere, somehow, there will always be deficiencies. With macro, you
can alleviate those somewhat. Not that I think macro would help rake
in the above case since that is a syntax problem, rather than semantic
problem (I have no idea if macro can save you from invalid syntax
since the only macro system I've used is lisp's where the syntax is
not as rich as ruby's and also doesn't work if you have an invalid
syntax anyway).

I am not advocating for macro in Ruby. I am neutral; I do not care if
there is macro or not. If there is macro, then that's great, although
there are only so many legitimate usages. If there isn't, it is also
not such a big of a loss since there are only few legitimiate usages.

Even the above example (repeating yourself), can be somewhat
eliminated with the help of a good editor.

Have a nice day, thanks for the input! I didn't know about MetaTags
before. Thank you.


YS.

 

[David A. Black]

Hi --

Actually, the example Jesse gave was probably bad. A macro, like any
other language construct, can be created to enhance clarity or
obfucate. Thus, one should not make a hasty judgement based on that
one example.

How about this example (which I hope is a better example of
easy-with-macro,-hard-(impossible?)-without):

Suppose we want to do self-documenting methods (not source file),
similar to docstring in emacs:

An implementation with macro would look something like:

def foo(arg)
"This method will return <code>arg</code>"
arg
end

You can already do that, if you don't mind a warning with -w But
assuming you intend a macro to let you do things that are otherwise
illegal syntax, that's where the problem arises of having everyone's
code, potentially, look and act different, and, as others have said,
just not to look like Ruby. (Unless we define "Ruby" as "a macro
language in which any syntax is possible", but I'd rather not


David

 

[Yohanes Santoso]

You can already do that, if you don't mind a warning with -w But

Hi David,

My example is incomplete. There should be this following that:

assuming you intend a macro to let you do things that are otherwise
illegal syntax, that's where the problem arises of having everyone's

In my reply to gabriele, I mentioned that I do not know if macro can
save one from an invalid syntax since my experience with macro is
limited to CL's and elisp's macros which won't work if the syntax is
invalid.

Rather, what the example above shows that with macro you can change
the semantic of method definition, i.e.: if there is a string at the
top of the method body, take that as the docstring for that method,
instead of giving a warning about unused literal.

Sorry, I am unable to come up with a likely macro definition that
would allow the above example to be true. I had difficulty imagining
how you can define such macro in ruby; my imagination is lacking.

I also believe that macro, like any other tool, can be used for good
or evil. More importantly, I do not care if Ruby has macro or not,
although now that I have more time to think since my reply to
gabriele, I am leaning toward leaving things the way it is (without
macro) since macro is susceptible to being abused.

Good day,
YS.

 

[Mauricio Fernández]

I don't know what exactly you mean by this. Argument passing is the same
for blocks and procs IMHO. Can you explain the difference?

ArgumentError: wrong number of arguments (2 for 4)
from (irb):2
from (irb):2:in `call'
from (irb):2=> "2004-07-30"

 

[Robert Klemme]

ArgumentError: wrong number of arguments (2 for 4)
from (irb):2
from (irb):2:in `call'
from (irb):2
=> "2004-07-30"

Thanks! Also:

irb(main):001:0> def foo(&b) b.call 1,2 end
=> nil
irb(main):002:0> foo {|a,b,c,d|}
=> nil

robert

 

[Kristof Bastiaensen]

I don't know what exactly you mean by this. Argument passing is the same
for blocks and procs IMHO. Can you explain the difference?

Oh, it seems I was wrong (after looking it up with ri).
What I said in the previous post only applies to procs created
with lambda (including the return thing), not procs created with
Proc.new.
So I guess that Proc.new creates exactly the same Proc object
as a block does (in the case that a Proc object gets created
from a block). Only lambda expressions are different.
That would also mean that the best way to get rid of the
LocalJumpError problem is to use a lambda expression.

Here is an example about what I meant with argument passing:

b1 = Proc.new { |a,b|
puts "a == #{a.inspect}, b == #{b.inspect}" }
b2 = lambda { |a,b|
puts "a == #{a.inspect}, b == #{b.inspect}" }

b1[1]
b2[1]
#--- output: ----
a == 1, b == nil
/tmp/blocks.rb:5: wrong number of arguments (1 for 2) (ArgumentError)
from /tmp/blocks.rb:5:in `[]'
from /tmp/blocks.rb:9
#----------------

Sorry for the confusion.

Cheers,
KB

 

[Caio Chassot]

What I said in the previous post only applies to procs created

with lambda (including the return thing), not procs created with
Proc.new.

and is there any difference between lambda {...} and proc {...} ?

 

[Kristof Bastiaensen]

and is there any difference between lambda {...} and proc {...} ?

Well, according to ri they are the same method, and a short
investigation seemes to confirm that.

KB

 

[David A. Black]

Hi --

and is there any difference between lambda {...} and proc {...} ?

They're synonyms, but proc is being phased out, because it looks too
similar to Proc.new {} but does something different.

It does all get rather complex, in my opinion. I think it's
reasonable to say that a certain number of us are hoping for some kind
of unification/simplification of the whole lambda/Proc/method/block
landscape.


David

 

[gabriele renzi @ google]

Yohanes Santoso wrote...

and this is what the doc has to say about the '!Class' tag:

!Class - A string indicating the fully-qualified class name. This
is a required field.


Robert's example is about similar to what MetaTags does. Thus, they
both make one to repeat one self; you have to say: "this docstring is
for this method of this class", even if the information is available
nearby. I think this counts as quirkiness.

Actually, you don't:

# utterly broken but shows the concept:

[nickel@UltimaThule nickel]$ irb --simple-prompt

def doc str
@is_test=str
end => nil
class Module
Docs=Hash.new
def method_added(id)
Docs[id]=@is_test if @is_test
end
end => nil
class Foo
doc "this speaks"
def meow
"meoww"
end
end => nil
Module:ocs[:meow] => "this speaks"
module M
def foo
end
doc "yuppy"
def baz
'naia'
end
end => nil
Module:ocs[:baz]

=> "yuppy"

anyway, I agree this is really far from perfect

 

[Austin Ziegler]

Actually what surprises me most about Ruby's one-block per method
technique is how flexible it turns out to be. One would think that
having the ability to pass multiple blocks would be big improvement, but
I suspect that the improvement in flexibility is merely incremental.

What surprises me about Ruby is the about of expressiveness you get,
even though you are limited to only one block per method call. You
would think that having the ability to pass multiple closures to a
method would be a big improvement,

When I ported Diff::LCS, the Perl Algorithm:iff version from which I
started used a hash of anonymous procs. I modified this to use a
callback *object*. I think that by the time you start needing multiple
procs, it's just as easy (or easier) to require an object that
encapsulates that.

-austin

 

[Paul Brannan]

I prefer a light-weight syntax for something as commonly used as
anonymous functions. On the other hand, I still don't know why {|n| n
** 2} can't simply create a proc object.

Remember how the lambda method works: it takes a block parameter and
creates an object out of it. IMO this is very clean; blocks are the
basic building block and procs and bindings are built on top of them.

As to why we cannot remove the call to the lambda method, consider:
1. Ruby does have blocks, and
2. Ruby does not require parens for method calls.

If I write:

(1..10).map { |n| n ** 2 }

Should this be interpreted as:

(1..10).map(lambda { |n| n ** 2 })

or as:

(1..10).map do |n| n ** 2; end

If it were possible to merge blocks and proc objects without
introducting ambiguity and without losing backward compatibility, it
might not be a bad idea. I do not think this is possible, though.

Paul