Ruby as a domain specific language?

[cnmaclean]

Hi

In my company, we're looking at creating a domain-specific language.
Instead of being a totally proprietary language, which we'd have to
support and which would probably be limited in various ways etc., I was
thinking about the possibility of basing it on an existing language,
and Ruby in particular seemed to come to my attention.
I haven't really used Ruby at all, but from what I've read, it seems
like it could be quite suitable.

What I'm wondering is, are there any good resources on using Ruby as a
domain-specific language? Is there any documentation on the different
way in which Ruby can be extended to support domain-specific
constructs?

What we're trying to do in particular is define a data description
langauge, where we can define human-readable "translations" for hex
data. This would be largely in a declarative style (e.g. "DataItem
Name='My Data' Coding='Language' Length=2"); using something like Ruby
might allow us to do the simple things (like the above) simply, but
still keep open the possibility of using the power of a programming
language where required - e.g. where the data structure is particularly
dynamic.
The other characteristic of our language is that it will probably be
fairly hierarchically structured. For example, we have the concept of
translations for files, which may be contained in folders. So we'd
want to specify parent folders (with identifiers), and other
files/folders as children of these folders; the "leaf" files would have
translations etc. associated with them.

Thanks in advance for your help,
Calum

 

[jim]

In my company, we're looking at creating a domain-specific language.
Instead of being a totally proprietary language, which we'd have to
support and which would probably be limited in various ways etc., I was
thinking about the possibility of basing it on an existing language,
and Ruby in particular seemed to come to my attention.
I haven't really used Ruby at all, but from what I've read, it seems
like it could be quite suitable.

What we're trying to do in particular is define a data description
langauge, where we can define human-readable "translations" for hex
data. This would be largely in a declarative style (e.g. "DataItem
Name='My Data' Coding='Language' Length=2"); using something like Ruby
might allow us to do the simple things (like the above) simply, but
still keep open the possibility of using the power of a programming
language where required - e.g. where the data structure is particularly
dynamic.
The other characteristic of our language is that it will probably be
fairly hierarchically structured. For example, we have the concept of
translations for files, which may be contained in folders. So we'd
want to specify parent folders (with identifiers), and other
files/folders as children of these folders; the "leaf" files would have
translations etc. associated with them.

Do you mean someting like:

define_data {
data_item { |t|
t.name "My Data"
t.coding "Language"
t.length 2
}

files "." {
decode_with "My Data"
}
}

 

[cnmaclean]

In our current (proprietary) language, it's something like this:

Define File 1234
Name = "My File"
Parent = 5678
DataItem = "DataItem 1", Decimal, 10 bytes
DataItem = "DataItem 2", Hex, 10 bytes

This format allows you to do simple things fairly easily, but it might
be impossible to do more complicated things.

Currently, the format is almost entirely declarative. You can't have
any logic (e.g. determining the second data item based on the contents
of the first), and there's only a very primitive and limited form of
variables. Doing some new means extending the language definition.

I'd also like (if possible) to allow declaring files hierarchically, so
that I can define File 1234 in the scope of 5678.

As I said, I'm also just looking for general articles/mailings etc. on
using Ruby as a domain-specific language - examples, showing what's
possible/allowed etc.

Thanks,
Calum

 

[jim]

In our current (proprietary) language, it's something like this:

Define File 1234
Name = "My File"
Parent = 5678
DataItem = "DataItem 1", Decimal, 10 bytes
DataItem = "DataItem 2", Hex, 10 bytes

Currently, the format is almost entirely declarative. You can't have
any logic (e.g. determining the second data item based on the contents
of the first), and there's only a very primitive and limited form of
variables. Doing some new means extending the language definition.

I'd also like (if possible) to allow declaring files hierarchically, so
that I can define File 1234 in the scope of 5678.

As I said, I'm also just looking for general articles/mailings etc. on
using Ruby as a domain-specific language - examples, showing what's
possible/allowed etc.

Sorry, but I don't know of any write-ups on this. However, it
was a hot topic at the last Ruby Conf, and I bet we see at least
one presentation on this at the next Ruby Conf.

Maybe I can help with some examples, or better yet, write
your example in a Ruby DSL form.

define_file 1234 do |f|
f.name "My File"
f.parent 5678
f.data_item "DataItem 1", Decimal, 10.bytes
f.data_item "DataItem 2", Hex, 10.bytes
end

Here, we have defined #name, #parent and #data_item as methods.
As these are constructed, references can be constructed to give
you the 'smarts' that you are looking for.
With these smarts, we can to do programmatical things like iteration:

define_file 5678 do |f|
f.name "Parent File"
f.parent 1
f.data_item :inherit_from_parent
f.children { |child| child.do_whatever }
end

 

[Martin DeMello]

In our current (proprietary) language, it's something like this:

Define File 1234
Name = "My File"
Parent = 5678
DataItem = "DataItem 1", Decimal, 10 bytes
DataItem = "DataItem 2", Hex, 10 bytes

Ruby would let you come fairly close:

#-----------------------------------------------------------------------#

class DataFile
def initialize(file_id, &block)
@file_id = file_id
$registry[file_id] = self
@data = []
instance_eval(&block)
end

def name(name)
@name = name
end

def parent(pid)
@parent = pid
end

def data_item(*args)
p args
@data << args
end
end

def define(klass, *args, &block)
klass.new(*args, &block)
end

$registry = {}

define(DataFile, 1234) {
name "My File"
parent 5678
data_item "DataItem 1", ecimal, "10 bytes"
data_item "DataItem 2", :Hex, "10 bytes"
}

p $registry

#-----------------------------------------------------------------------#

If you don't like the string quoting in "10 bytes" you can do

class Fixnum
def bytes
self
end
end

and then write 10.bytes

martin

 

[Alexey Verkhovsky]

Actually, quick and dirty domain language in Ruby is very easy. E.g.,
you can write in less than a day some Ruby that will take this:

class File1234Parser < FileParser::Base
define_file 1234,
:name => 'My File',
arent => 5678 {
decimal "DataItem 1", 10.bytes
hex "DataItem 2", 10.bytes
}
end

File1234Parser.new(open('~/files/foo.1234')).parse
... or whatever...

This would work by invoking a method FileParser::Base.define_file, that
would take whatever arguments you give it, make some Ruby code out of it
and then

self.module_eval <<-EOL
def parse
<code generated from arguments>
end
...
EOL

For things like parsing and/or generating a flat, fixed-width file, this
can be done in a day (I did it). I also used the same approach once to
generate a test oracle for a fairly complicated billing application. The
"price definition files" there looks like:

class GprsPromotion << StandardPricePlan
market_ratesDEFAULT_PREPAID_RATES)
rate :voice => flat_rate(1.00.ntd.per_minute, :rounding_factor =>
15.seconds, :round => :up)
rate :sms => event_rate(2.00.ntd.per_event)
# etc
end

You can show it to someone who doesn't know Ruby and s/he will
understand it.

It is also very easy to add special behaviors, because you can

define_file :Foo
alias :generic_parse arse
def parse
generic_parse
do.some { ruby.wizardry } if necessary?
end

The dark side is that it all has to be valid Ruby syntax, and that is a
bit more commas and braces than in the perfect world.

--
Best regards,

Alexey Verkhovsky

Ruby Forum: http://ruby-forum.org (moderator)
RForum: http://rforum.andreas-s.net (co-author)
Instiki: http://instiki.org (maintainer)

 

[Assaph Mehr]

In my company, we're looking at creating a domain-specific language.
Instead of being a totally proprietary language, which we'd have to
support and which would probably be limited in various ways etc., I was
thinking about the possibility of basing it on an existing language,
and Ruby in particular seemed to come to my attention.
I haven't really used Ruby at all, but from what I've read, it seems
like it could be quite suitable.

What I'm wondering is, are there any good resources on using Ruby as a
domain-specific language? Is there any documentation on the different
way in which Ruby can be extended to support domain-specific
constructs?

There was once a post by Phil Tomson about a Quantum Design Language
which might be close to what you're after:
http://blade.nagaokaut.ac.jp/cgi-bin/scat.rb/ruby/ruby-talk/99734

What we're trying to do in particular is define a data description
langauge, where we can define human-readable "translations" for hex
data. This would be largely in a declarative style (e.g. "DataItem
Name='My Data' Coding='Language' Length=2"); using something like Ruby
might allow us to do the simple things (like the above) simply, but
still keep open the possibility of using the power of a programming
language where required - e.g. where the data structure is particularly
dynamic.
The other characteristic of our language is that it will probably be
fairly hierarchically structured. For example, we have the concept of
translations for files, which may be contained in folders. So we'd
want to specify parent folders (with identifiers), and other
files/folders as children of these folders; the "leaf" files would have
translations etc. associated with them.

The other option is to go with yaml for the data file and process it
with Ruby. It might not be the best thing for human editing (as it's
whitespace significant) but it should be very easy to define
hierarchical constructs to be later processed by Ruby.

HTH,
Assaph

 

[Phil Tomson]

Hi

In my company, we're looking at creating a domain-specific language.
Instead of being a totally proprietary language, which we'd have to
support and which would probably be limited in various ways etc., I was
thinking about the possibility of basing it on an existing language,
and Ruby in particular seemed to come to my attention.
I haven't really used Ruby at all, but from what I've read, it seems
like it could be quite suitable.

What I'm wondering is, are there any good resources on using Ruby as a
domain-specific language? Is there any documentation on the different
way in which Ruby can be extended to support domain-specific
constructs?

What we're trying to do in particular is define a data description
langauge, where we can define human-readable "translations" for hex
data. This would be largely in a declarative style (e.g. "DataItem
Name='My Data' Coding='Language' Length=2"); using something like Ruby
might allow us to do the simple things (like the above) simply, but
still keep open the possibility of using the power of a programming
language where required - e.g. where the data structure is particularly
dynamic.
The other characteristic of our language is that it will probably be
fairly hierarchically structured. For example, we have the concept of
translations for files, which may be contained in folders. So we'd
want to specify parent folders (with identifiers), and other
files/folders as children of these folders; the "leaf" files would have
translations etc. associated with them.

Thanks in advance for your help,
Calum

I would tend to say it more like: Ruby for creating a domain specific
language (DSL).

Ruby excells in creating DSLs primarily because of it's code blocks. I've
created a couple of DSLs with Ruby:

QDL (Quantum Description Language) - a language for describing quantum
logic circuits. See:
http://nexp.cs.pdx.edu/~qc/cgi-bin/view/QC/QuantumDesignLanguage

RHDL (Ruby Hardware Description Langauge) - a language for describing
hardware (and simulating concurrency). See:
http://www.aracnet.com/~ptkwt/ruby_stuff/RHDL/


Code blocks allow you to create a very natural-looking DSL directly
in Ruby without need of a special parser.


There's also Rake which could definately fall into the catagory of a DSL.

Phil

 

[Michael Neumann]

RHDL (Ruby Hardware Description Langauge) - a language for describing
hardware (and simulating concurrency). See:
http://www.aracnet.com/~ptkwt/ruby_stuff/RHDL/

I add mine:

Ruby for logic formulas:

http://www.ntecs.de/viewcvs/viewcvs/ForSys/

I actually split the DSL into two parts. The "clean"-part, where I use
"regular" Ruby constructs and the other part where I make heavy use of
const_missing and method_missing, this generates objects of classes
defined in the "clean"-part.

Clean:

Impl[
ForAll[ Var[:x], Pred[, Var[:x], Var[:y]] ],
Exists[ Var[:y], Pred[:Q, Var[:y]] & ForAll[ Var[:z], Pred[:R] ] ]
]

"Un-Clean":

V::x { Px, :y) } >= E::y { Qy) & V::z { R() } }

Both is actually valid Ruby!

Regards,

Michael