[Chocolate Ketchup Dressing] A working Ruby source code filter

[Florian Gross]

Moin!

I've implemented a Ruby source code filter for the frequently requested
".=" operator. (Replace by result of method call)

Here's a boring example of how it is used:

#!/usr/bin/ruby -rfilter
obj = "foobar"
obj .= reverse
p obj # => "raboof"

I'm using IRB's Ruby lexer for this so it should in theory not screw you
up as badly as the Switch.pm module for Perl 5.

Details and source code are available at
http://www.codepaste.org/view/paste/300 -- but the source code and the
Binding.of_caller dependency are also attached to this mail.

Maybe this technique can be used for trying out other, more complex
modifications of the Ruby language?

Kind regards,
Florian Gross


require 'irb'
require 'stringio'
require 'binding_of_caller'

module Filter
extend self

def handle(filename)
code = case filename
when "-": STDIN.read
else File.read(filename)
end

eval(code, nil, filename)
end

def filter(code)
lines = code.split("\n")
p lines if $DEBUG
lex = RubyLex.new
io = StringIO.new(code + "\n")
lex.set_input(io)

tokens, state = Array.new, Array.new
while token = lex.token
case state.last
when nil
case token
when RubyToken::TkIDENTIFIER
state << token
end

when RubyToken::TkIDENTIFIER
case token
when RubyToken::TkSPACE
next
when RubyToken::TkDOT
state << token
else
state.clear
end

when RubyToken::TkDOT
case token
when RubyToken::TkASSIGN
state << token
else
state.clear
end

when RubyToken::TkASSIGN
case token
when RubyToken::TkSPACE, RubyToken::TkNL,
RubyToken::TkCOMMENT
next
when RubyToken::TkIDENTIFIER, RubyToken::TkFID
state << token
lvalue, dot, assign, rvalue = *state
first_line, last_line = lvalue.line_no - 1, rvalue.line_no - 1

new_line = ""
new_line << lines[first_line][0 ... lvalue.char_no].to_s
new_line << "#{lvalue.name} = #{lvalue.name}.#{rvalue.name}"
new_line << lines[last_line][(rvalue.char_no + rvalue.name.length) .. -1].to_s

if $DEBUG then
puts "Replacing lines at #{first_line} .. #{last_line}:",
lines[first_line .. last_line],
"with:",
new_line
end

lines[first_line .. last_line] = new_line
end
end
end

return lines.join("\n")
end
end

module Kernel
alias ld_eval :eval
def eval(code, context = nil, *more)
# Ignore calls from Binding.of_caller to avoid endless loops
if caller.first["in `of_caller'"]
return old_eval(code, context, *more)
end

begin
Binding.of_caller do |caller_context|
old_eval(Filter.filter(code), context || caller_context, *more)
end
rescue ArgumentError
old_eval(Filter.filter(code), TOPLEVEL_BINDING, *more)
end
end

def load(filename, wrap = false)
if File.expand_path(filename) != filename then
$LOAD_PATH.each do |path|
fullname = File.join(path, filename)

if File.exist?(fullname) then
filename = fullname
break
end
end
end

unless File.exist?(filename) then
raise(LoadError, "No such file to load -- #{filename}")
end

code = File.read(filename)

context = case wrap
when true then
Module.new.sendbinding)
else TOPLEVEL_BINDING
end

eval(code, context, filename)

return true
end

alias ld_require :require
def require(filename)
case File.extname(filename)
when ".so", ".o", ".dll" then
old_require(filename)
when ""
is_binary = $LOAD_PATH.any? do |path|
%w{.so .o .dll}.any? do |ext|
fullname = File.join(path, filename + ext)
File.exist?(fullname)
end
end

if is_binary then
old_require(filename)
else
filename += ".rb"
end
end

if $LOADED_FEATURES.include?(filename) then
return false
else
load(filename)
$LOADED_FEATURES << filename
return true
end
end

alias ld_instance_eval :instance_eval
def instance_eval(code = nil, *more, &block)
if not code then
old_instance_eval(*more, &block)
elsif block
old_instance_eval(Filter.filter(code), *more, &block)
end
end
end

class Module
alias :module_eval :instance_eval
alias :class_eval :instance_eval
end

begin
unless %w{irb -e}.include?($0)
Filter.handle($0)
exit
end
rescue Exception; end

begin
require 'simplecc'
rescue LoadError
def Continuation.create(*args, &block)
cc = nil; result = callcc {|c| cc = c; block.call(cc) if block and args.empty?}
result ||= args
return *[cc, *result]
end
end

# This method returns the binding of the method that called your
# method. It will raise an Exception when you're not inside a method.
#
# It's used like this:
# def inc_counter(amount = 1)
# Binding.of_caller do |binding|
# # Create a lambda that will increase the variable 'counter'
# # in the caller of this method when called.
# inc = eval("lambda { |arg| counter += arg }", binding)
# # We can refer to amount from inside this block safely.
# inc.call(amount)
# end
# # No other statements can go here. Put them inside the block.
# end
# counter = 0
# 2.times { inc_counter }
# counter # => 2
#
# Binding.of_caller must be the last statement in the method.
# This means that you will have to put everything you want to
# do after the call to Binding.of_caller into the block of it.
# This should be no problem however, because Ruby has closures.
# If you don't do this an Exception will be raised. Because of
# the way that Binding.of_caller is implemented it has to be
# done this way.
def Binding.of_caller(&block)
old_critical = Thread.critical
Thread.critical = true
count = 0
cc, result, error, extra_data = Continuation.create(nil, nil)
error.call if error

tracer = lambda do |*args|
type, filename, context, extra_data = args[0], args[1], args[4], args
if type == "return"
count += 1
# First this method and then calling one will return --
# the trace event of the second event gets the context
# of the method which called the method that called this
# method.
if count == 2
# It would be nice if we could restore the trace_func
# that was set before we swapped in our own one, but
# this is impossible without overloading set_trace_func
# in current Ruby.
set_trace_func(nil)
cc.call(eval("binding", context), nil, extra_data)
end
elsif type != "line"
set_trace_func(nil)
error_msg = "Binding.of_caller used in non-method context or " +
"trailing statements of method using it aren't in the block."
cc.call(nil, lambda { raise(ArgumentError, error_msg) }, nil)
end
end

unless result
set_trace_func(tracer)
return nil
else
Thread.critical = old_critical
case block.arity
when 1 then yield(result)
else yield(result, extra_data)
end
end
end

 

[gabriele renzi]

Florian Gross ha scritto:

Moin!

I've implemented a Ruby source code filter for the frequently requested
".=" operator. (Replace by result of method call)


hypercool!


Maybe this technique can be used for trying out other, more complex
modifications of the Ruby language?

very interesting.. you rock

 

[Edgardo Hames]

Moin!

I've implemented a Ruby source code filter for the frequently requested
".=" operator. (Replace by result of method call)

Here's a boring example of how it is used:

#!/usr/bin/ruby -rfilter
obj = "foobar"
obj .= reverse
p obj # => "raboof"

Hi, Florian. How does .= differ from bang methods?
I mean obj.reverse seems to be the same as obj.reverse!

Thanks and kind regards,
Ed

 

[Robert Klemme]

Hi, Florian. How does .= differ from bang methods?
I mean obj.reverse seems to be the same as obj.reverse!

Not quite: the bang method usually saves an intermediate instance *and* it
might have undesirable side effects (if someone else holds a reference to
the instance and doesn't expect it to be modified). As a consequence,
".=" will work with frozen objects, while bang methods usually don't. As
usual it's a tradeoff / design decision which of both approaches you use.

Kind regards

robert