parse tree?

G

Giles Bowkett

I have a situation where it may make sense, although for some reason I
don't want to, to use something like ParseTree to pull out each
complete Ruby expression and pass it to a block which will then
evaluate it in a particular context and make all kinds of magic
happen.

Is this a reasonable ParseTree use case?

@expressions_parse_tree_found.each |exp|
ArbitraryModule.module_eval(&exp)
end

--
Giles Bowkett

Podcast: http://hollywoodgrit.blogspot.com
Blog: http://gilesbowkett.blogspot.com
Portfolio: http://www.gilesgoatboy.org
Tumblelog: http://giles.tumblr.com
 
J

Judson Lester

pt = ParseTree.new.parse_tree_for_string("1+2")
=> [[:call, [:lit, 1], :+, [:array, [:lit, 2]]]]
Ruby2Ruby.new.process(pt.first)
=> "(1 + 2)"
ryan. even though i've seen that before i thought i'd add: that is
insanely cool!
It is really cool. I've been using R2R to build an auto-rspec thing
that re-creates the context of an expression so that you can test
against it. It's actually quite fantastic.


Judson
 
G

Giles Bowkett

Sorry, that was a dumb question, PT gives you sexps, not Ruby, you'd
need Ruby2Ruby for that, and even then it might not work.

why not?

As far as I can tell you get the whole shebang back, and I just need
particular expressions.
pt = ParseTree.new.parse_tree_for_string("1+2")
=> [[:call, [:lit, 1], :+, [:array, [:lit, 2]]]]
Ruby2Ruby.new.process(pt.first)
=> "(1 + 2)"

does that help?

Sort of. It is obviously full of nifty and then some, and I think you
could use it to unit test metaprogramming, which is something the
metaprogramming-averse complain about the absence of.

But say I do this:
(string):1: warning: useless use of + in void context
=> [[:block, [:call, [:lit, 1], :, [:array, [:lit, 2]]], [:call,
[:lit, 1], :, [:array, [:lit, 2]]]]]=> "(1 + 2)\n(1 * 2)\n"

What if I just want 1 * 2?

--
Giles Bowkett

Podcast: http://hollywoodgrit.blogspot.com
Blog: http://gilesbowkett.blogspot.com
Portfolio: http://www.gilesgoatboy.org
Tumblelog: http://giles.tumblr.com
 
R

Ryan Davis

could use it to unit test metaprogramming, which is something the
metaprogramming-averse complain about the absence of.

But say I do this:
(string):1: warning: useless use of + in void context
=> [[:block, [:call, [:lit, 1], :, [:array, [:lit, 2]]], [:call,
[:lit, 1], :, [:array, [:lit, 2]]]]]=> "(1 + 2)\n(1 * 2)\n"

What if I just want 1 * 2?

It depends on what sort of '1 * 2' you want... Do you want the last
expression in a block/defn/whatever? Like, are you trying to analyze
what the return type is? Or do you want to be aware of that void
context warning and discard the crap? Or do you simply want the second
thingy? Or do you want something entirely different that I am not
anticipating?

There are a couple ways to go about it, depending on your answer
above. The easiest is Sexp/Array manipulation. Let's say you were
interested in the arg list of the last call in the block:
=> s:)block, s:)call, s:)lit, 1), :+, s:)array, s:)lit, 2))),
s:)call, s:)lit, 1), :*, s:)array, s:)lit, 2))))
=> s:)array, s:)lit, 2))

This is a bad example because it has two calls in the block. With
unique sub-sexp types in a sexp, you can just pull them out by name:

But it balks when it is ambiguous.

But... if that isn't what you want, there are more powerful means of
dealing with stuff:

class CallArgAnalyzer < SexpProcessor
def process_call sexp
sexp.shift # :call
recv = sexp.shift
name = sexp.shift
args = sexp.shift

# do something with args

return s:)nil) # or whatever... depends on context
end
end

Granted, this doesn't actually DO anything, but the framework to do
something is there and solid... Do note that SexpProcessor is meant
for transformational processing so it generally expects a sexp back
(you can set what the return type should be for all process_*
methods). The real point is that it is very easy to get at what you
want and mess with it.
 
G

Giles Bowkett

But say I do this:
pt = ParseTree.new.parse_tree_for_string("1 + 2 ; 1 * 2")
(string):1: warning: useless use of + in void context
=> [[:block, [:call, [:lit, 1], :, [:array, [:lit, 2]]], [:call,
[:lit, 1], :, [:array, [:lit, 2]]]]]
Ruby2Ruby.new.process(pt.first)
=> "(1 + 2)\n(1 * 2)\n"

What if I just want 1 * 2?

It depends on what sort of '1 * 2' you want... Do you want the last
expression in a block/defn/whatever? Like, are you trying to analyze
what the return type is? Or do you want to be aware of that void
context warning and discard the crap? Or do you simply want the second
thingy? Or do you want something entirely different that I am not
anticipating?

Well, I'm writing a code generator, and I thought I was very close to
finished, except I was basically doing this:

generated_code = Generators.module_eval(&block)

And that works great for tiny bits of code, it works at the statement
level, but when I want to just collect up *all* the statements in a
block, the whole thing kinds of goes south. So I have something which
generates all the individual statements perfectly but completely falls
apart when I want to create several statements. So it looks as if my
only option might be to rewrite everything, possibly adding a parser,
or using Ruby2C, or something like that. It's kind of a pain in the
ass.

But if I have code like this:

single_line(statement)
statement:)which => spans,
:multiple => lines)

And I can turn it into an array of procs, or blocks, or even strings,
then I can just do

code_chunks.each {|code_chunk| @code << Generators.module_eval(code_chunk)}

Or something similar, and harness everything I have so far without
throwing it all away and starting over from scratch or close to it.
All I really need at that point is a collection of code chunks, in
some format which module_eval will be comfortable with.
There are a couple ways to go about it, depending on your answer
above. The easiest is Sexp/Array manipulation. Let's say you were
interested in the arg list of the last call in the block:


This is a bad example because it has two calls in the block. With
unique sub-sexp types in a sexp, you can just pull them out by name:


But it balks when it is ambiguous.

But... if that isn't what you want, there are more powerful means of
dealing with stuff:

class CallArgAnalyzer < SexpProcessor
def process_call sexp
sexp.shift # :call
recv = sexp.shift
name = sexp.shift
args = sexp.shift

# do something with args

return s:)nil) # or whatever... depends on context
end
end


Granted, this doesn't actually DO anything, but the framework to do
something is there and solid... Do note that SexpProcessor is meant
for transformational processing so it generally expects a sexp back
(you can set what the return type should be for all process_*
methods). The real point is that it is very easy to get at what you
want and mess with it.

Very easy is a relative term. :)

In practice none of the code I'm working with or expecting to work
with has more than one statement on a line, but obviously the whole
problem is that it has many statements in a block.

The thing is, I posted a few days ago about a problem with if, not
being able to override it, and if I'm using a fully-fledged parsing
library then I might as well undo my kludge and enable real support
for conditionals like if, unless, while, etc. But it seems like more
work than I'd intended. But I can't deny that being able to use real
Ruby in the code which generates the other code would be a serious
improvement.

--
Giles Bowkett

Podcast: http://hollywoodgrit.blogspot.com
Blog: http://gilesbowkett.blogspot.com
Portfolio: http://www.gilesgoatboy.org
Tumblelog: http://giles.tumblr.com
 
R

Ryan Davis

Well, I'm writing a code generator, and I thought I was very close to
finished, except I was basically doing this:

generated_code = Generators.module_eval(&block)

And that works great for tiny bits of code, it works at the statement
level, but when I want to just collect up *all* the statements in a
block, the whole thing kinds of goes south. So I have something which
generates all the individual statements perfectly but completely falls
apart when I want to create several statements.

Does this help?
=> "proc {\n single_line(statement)\n statement:)which =>
spans, :multiple => lines)\n}"

There are clean ways to ensure it isn't a proc as well.
 
E

Eivind Eklund

Well, I'm writing a code generator, and I thought I was very close to
finished, except I was basically doing this:

generated_code = Generators.module_eval(&block)

And that works great for tiny bits of code, it works at the statement
level, but when I want to just collect up *all* the statements in a
block, the whole thing kinds of goes south.

From the description you give, it sounds like you could just create a
new block that just calls a list of other blocks, and do ; separation?

Ie, if the line above is in a method called "foo(&block)", call it
with something like

foo do
my_blocks.collect { |block| block.call + ";" }
end

Probably I'm not understanding your requirements, just thought I'd air
it in case it is this simple.

Eivind.
 
G

Giles Bowkett

And that works great for tiny bits of code, it works at the statement
Does this help?


There are clean ways to ensure it isn't a proc as well.

That totally solves my problem. Or at least, ***if*** it's guaranteed
to turn multi-line statements into one-liners every time, that very
probably totally solves my problem. (I haven't tried it yet, I've just
proved it to be true, etc.) I can just pull it out of the proc{} with
a regex and then split it on the newlines. Baddabing baddaboom.

--
Giles Bowkett

Podcast: http://hollywoodgrit.blogspot.com
Blog: http://gilesbowkett.blogspot.com
Portfolio: http://www.gilesgoatboy.org
Tumblelog: http://giles.tumblr.com
 
G

Giles Bowkett

Well, I'm writing a code generator, and I thought I was very close to
From the description you give, it sounds like you could just create a
new block that just calls a list of other blocks, and do ; separation?

Ie, if the line above is in a method called "foo(&block)", call it
with something like

foo do
my_blocks.collect { |block| block.call + ";" }
end

Probably I'm not understanding your requirements, just thought I'd air
it in case it is this simple.

The hard part is splitting an existing block into an array of blocks,
or strings, based on the logical division of statements; in other
words to say "turn this block into an array of statements" (in a way
that Ruby understands) is the tricky part.

--
Giles Bowkett

Podcast: http://hollywoodgrit.blogspot.com
Blog: http://gilesbowkett.blogspot.com
Portfolio: http://www.gilesgoatboy.org
Tumblelog: http://giles.tumblr.com
 
R

Ryan Davis

That totally solves my problem. Or at least, ***if*** it's guaranteed
to turn multi-line statements into one-liners every time, that very
probably totally solves my problem. (I haven't tried it yet, I've just
proved it to be true, etc.) I can just pull it out of the proc{} with
a regex and then split it on the newlines. Baddabing baddaboom.

What is your 1-liner requirement for?
 
G

Giles Bowkett

Ask a Question

Want to reply to this thread or ask your own question?

You'll need to choose a username for the site, which only take a couple of moments. After that, you can post your question and our members will help you out.

Ask a Question

Members online

Forum statistics

Threads
474,275
Messages
2,571,375
Members
48,067
Latest member
MackenzieP

Latest Threads

Top