to use AR or not?

[Jason Cameron]

Hmmm I've been scratching my head at this for a little while and maybe
I'm not thinking it through correctly.

I'm converting a portion of this application to Ruby and I have these
data tables that aren't in a database. Here's a sample:

rate_type location algorithm rate
BCD Here simple 1.2
ATD There fractal 4.3
STE Elsewhere binary 3.2
TSO Really Far polyamory 0.1

I need to be able to create a data structure that I can search easily.
Enabling bulk inserts of some sort would be nice as well.

At the moment I'm looking at activerecord-tableless. Unfortunately I'm
not entirely sure how to create a bunch of activerecords (not just one
at a time)

Is there a better alternative?

Thanks in advance,

M

 

[Brian Candler]

Jason Cameron wrote in post #977491:

rate_type location algorithm rate
BCD Here simple 1.2
ATD There fractal 4.3
STE Elsewhere binary 3.2
TSO Really Far polyamory 0.1

I need to be able to create a data structure that I can search easily.
Enabling bulk inserts of some sort would be nice as well.

At the moment I'm looking at activerecord-tableless. Unfortunately I'm
not entirely sure how to create a bunch of activerecords (not just one
at a time)

Is there a better alternative?

activerecord-tableless won't give you *any* searching functionality.
It's basically just model validations. If you don't need AR-style
validations then you might as well just use OpenStruct (ostruct.rb in
the core library)

I'd say the key points to consider are:

* What sort of searching do you want to do?
* Do you want persistence, or will you always reload your data every
time your application is re-run?

If your database is small and/or your queries are complex, you might
just want to search using a linear scan.

objs.find { |o| ... condition ... }

If you need an indexed search but you only want to search on one field
at a time (e.g. find all fields with location "Here") then you can just
build hash indexes for yourself.

If you want to persist this Ruby object structure you've built, then you
could use something like Madeleine.

If you want to keep everything in memory but have some basic querying
like unions and intersections, maybe "redis" is what you're after. By
leaving the redis daemon running your data can persist in memory and be
accessible to multiple scripts (in ruby and/or other languages).

If you want arbitrary querying, indexes and persistence to disk, then
you'd almost certainly be better off with something like sqlite.

Regards,

Brian.

 

[Jason Cameron]

Brian Candler wrote in post #977595:

Jason Cameron wrote in post #977491:

activerecord-tableless won't give you *any* searching functionality.
It's basically just model validations. If you don't need AR-style
validations then you might as well just use OpenStruct (ostruct.rb in
the core library)

I'd say the key points to consider are:

* What sort of searching do you want to do?
* Do you want persistence, or will you always reload your data every
time your application is re-run?

If your database is small and/or your queries are complex, you might
just want to search using a linear scan.

objs.find { |o| ... condition ... }

If you need an indexed search but you only want to search on one field
at a time (e.g. find all fields with location "Here") then you can just
build hash indexes for yourself.

If you want to persist this Ruby object structure you've built, then you
could use something like Madeleine.

If you want to keep everything in memory but have some basic querying
like unions and intersections, maybe "redis" is what you're after. By
leaving the redis daemon running your data can persist in memory and be
accessible to multiple scripts (in ruby and/or other languages).

If you want arbitrary querying, indexes and persistence to disk, then
you'd almost certainly be better off with something like sqlite.

Regards,

Brian.

Wow you've given me a lot to of options to consider!

Well it looks like the largest table has 6-7 columns and around 70-80
rows.

I'll look into OpenStruct and redis but I'm curious about your linear
scan options.

objs.find { |o| ... condition ... }

Would I have to create the find functionality myself? I'm assuming I'm
creating a class to hold the object.

Thanks again in advance,

M

 

[Jason Cameron]

Brian Candler wrote in post #977595:

Jason Cameron wrote in post #977491:

activerecord-tableless won't give you *any* searching functionality.
It's basically just model validations. If you don't need AR-style
validations then you might as well just use OpenStruct (ostruct.rb in
the core library)

I'd say the key points to consider are:

* What sort of searching do you want to do?
* Do you want persistence, or will you always reload your data every
time your application is re-run?

If your database is small and/or your queries are complex, you might
just want to search using a linear scan.

objs.find { |o| ... condition ... }

If you need an indexed search but you only want to search on one field
at a time (e.g. find all fields with location "Here") then you can just
build hash indexes for yourself.

If you want to persist this Ruby object structure you've built, then you
could use something like Madeleine.

If you want to keep everything in memory but have some basic querying
like unions and intersections, maybe "redis" is what you're after. By
leaving the redis daemon running your data can persist in memory and be
accessible to multiple scripts (in ruby and/or other languages).

If you want arbitrary querying, indexes and persistence to disk, then
you'd almost certainly be better off with something like sqlite.

Regards,

Brian.

Hmmm after some digging I found Struct. How different is that from
OpenStruct? The following code is what I came up with:

cs = Struct.newrate_type, :location, :algorithm, :rate)
cs_col = []
cs_col.push(cs.new('BCD', 'Here', 'simple', 1.2))
cs_col.push(cs.new('ATD', 'There', 'fractal', 4.3))
cs_col.push(cs.new('STE', 'Elsewhere', 'fractal', 3.2))
cs_col.push(cs.new('TSO', 'Really Far', 'polyamory', 0.1))

puts cs_col.find {|i| i.location == 'Elsewhere'}.rate

 

[Brian Candler]

Jason Cameron wrote in post #977627:

Well it looks like the largest table has 6-7 columns and around 70-80
rows.

OK, that's a pretty small dataset.

I'll look into OpenStruct and redis but I'm curious about your linear
scan options.


Would I have to create the find functionality myself?

Nope; if objs is an Array containing your rows, then Array already mixes
in Enumerable which includes a find method (also find_all/select, which
gives all matching rows, not just the first one)

I'm assuming I'm
creating a class to hold the object.

You can certainly do that. As you already found, Struct gives you an
easy way to create a class with accessors. OpenStruct is a more dynamic
object, basically a wrapper around a Hash.