[ANN] arrayfields-3.0.0

[Ara.T.Howard]

URLS:

http://raa.ruby-lang.org/project/arrayfields/
http://www.codeforpeople.com/lib/ruby/arrayfields/

SYNOPSIS

allow keyword access to arrays:

require 'arrayfields'

fields = 'name', 'age'
row = [ 'bob', 30 ]

row.fields = fields

row[ 'name' ] #=> 'bob'
row.indices 'name', 'age' #=> [ 'bob', 30 ]

assigning to un-named fields appends:

stack = []
stack.fields = %w(zero one)
stack['zero'] = 'zero'
stack['one'] = 'one'
stack #=> [ 'zero', 'one' ]

*very* useful for database work

relation = pgconn.query sql
relation.size #=> 65536

# yikes! do we really want to re-construct a hash for for each tuple when
# we already have Arrays?

fields = %w(ssn name position)
table.each{|tuple| tuple.fields = fields}

table[34578]['ssn'] #=> 574865032

LIST OF OVERRIDDEN METHODS

Array#[]
Array#[]=
Array#at
Array#delete_at
Array#fill
Array#values_at
Array#indices
Array#indexes
Array#slice
Array#slice!

LIST OF NEW Array METHODS

Array#fields=
Array#each_with_field

DOCS/USAGE/SAMPLE

lib/arrayfields.rb
test/arrayfields.rb

AUTHOR

(e-mail address removed)

-a
--
===============================================================================
| EMAIL :: Ara [dot] T [dot] Howard [at] noaa [dot] gov
| PHONE :: 303.497.6469
| A flower falls, even though we love it;
| and a weed grows, even though we do not love it.
| --Dogen
===============================================================================

 

[Joao Pedrosa]

Hi,

Cool, I'm gonna give it a whirl.

Thanks a lot,
Joao

URLS:

http://raa.ruby-lang.org/project/arrayfields/

http://www.codeforpeople.com/lib/ruby/arrayfields/

SYNOPSIS

allow keyword access to arrays:

require 'arrayfields'

fields = 'name', 'age'
row = [ 'bob', 30 ]

row.fields = fields

row[ 'name' ] #=> 'bob'
row.indices 'name', 'age' #=> [ 'bob', 30 ]

__________________________________
Do you Yahoo!?
Yahoo! Mail Address AutoComplete - You start. We finish.
http://promotions.yahoo.com/new_mail

 

[Joel VanderWerf]

URLS:

http://raa.ruby-lang.org/project/arrayfields/
http://www.codeforpeople.com/lib/ruby/arrayfields/

SYNOPSIS

allow keyword access to arrays:

require 'arrayfields'

fields = 'name', 'age'
row = [ 'bob', 30 ]

row.fields = fields

row[ 'name' ] #=> 'bob'
row.indices 'name', 'age' #=> [ 'bob', 30 ]

Hm, sorta like something I've been using:

---
module AccessibleIndex
def index_accessor(h)
h.each do |sym, idx|
define_method sym do
self[idx]
end
define_method "#{sym}=" do |val|
self[idx] = val
end
end
end
end

a = [0,1,2,3,4,5]
class << a
extend AccessibleIndex
index_accessor :x => 3
end

a.x = "three"
p a # ==> [0, 1, 2, "three", 4, 5]

 

[Ara.T.Howard]

---
module AccessibleIndex
def index_accessor(h)
h.each do |sym, idx|
define_method sym do
self[idx]
end
define_method "#{sym}=" do |val|
self[idx] = val
end
end
end
end

a = [0,1,2,3,4,5]
class << a
extend AccessibleIndex
index_accessor :x => 3
end

a.x = "three"
p a # ==> [0, 1, 2, "three", 4, 5]

cool. your way is perhaps safer. ;-)

the reason i wrote this in the first place was that i was doing alot of
postgresql work and it was returning these huge result sets that i would then
write code that looked like:

if tuple[17] =~ pat or tuple[9].to_i < 42
...
end

and i was reading my own code thinking - huh? so then i started doing:

res = pgconn.exec sql

tuples = res.result
fields = res.fields

tuples =
tuples.collect do |tuple|
h = {}
fields.each do |field|
h[field] = tuple.shift
end
h
end

this is NOT good if tuple.size == 42000!! finally i wrote the arrayfields
module. it still requires work:

res = pgconn.exec sql

tuples = res.result
fields = res.fields

tuples.each{|t| t.fields = fields}


but now all 42000 tuples SHARE a copy of fields for doing their lookups - i
still have to iterate over all of em - but i don't need to create any new
objects and my code new reads like:

if tuple['name'] =~ pat or tuple['age'].to_i < 42
...
end

which is a lot nicer.

in any case it's just one of those things that i ended up using alot. i'd
like to re-write it in C but don't have the time right now...


cheers.


-a
--
===============================================================================
| EMAIL :: Ara [dot] T [dot] Howard [at] noaa [dot] gov
| PHONE :: 303.497.6469
| A flower falls, even though we love it;
| and a weed grows, even though we do not love it.
| --Dogen
===============================================================================

 

[Ben Giddings]

if tuple[17] =~ pat or tuple[9].to_i < 42
...
end ...
if tuple['name'] =~ pat or tuple['age'].to_i < 42
...
end

Is there any reason you didn't just use either constants or variables like:

if tuple[NAME] =~ pat or tuple[AGE].to_i < 42
...
end

or if your indices weren't constant, set them from a DB query and then do:

if tuple[name] =~ pat or tuple[age].to_i < 42
...
end

arrayfields looks great, and makes things much easier, but it just seems
odd to me that you'd approach it the way you did.

Ben

 

[Ara.T.Howard]

Is there any reason you didn't just use either constants or variables like:

yes.

constants:

can't be reused - obviously - so

tuple_a[NAME]

tuple_b[NAME]

won't work of NAME is different for both

this also doesn't work when the relation is modified. in general there is
no guaruntee on the order of fields returned to this is never a good
option.


variables:

i like to do this

name, age, ssn = tuple.values_at 'name', 'age', 'ssn'

so i already used up those var names - not that i couldn't use 'name_idx',
etc. but this starts getting messy fast if you are dealing for four or
five tables with 10-20 fields each.

lastly there is one nice feature of using the arrayfields method: say you are
generating a report from a relation using something like you are suggesting

res = pgconn.exec sql
tuples, fields = res.tuples, res.fields

name, age, ssn = %w(name age ssn).map{|f| fields.index f}

YAML::dump(
tuples.map do |t|
Hash[
'name' => tuple[name],
'age' => tuple[age],
'ssn' => tuple[ssn],
]
end
)

and later the relation is modified to now have a new column you'll need to
modify the code. whereas if you use the fields this code will work even when
the table is modified

res = pgconn.exec sql
tuples, fields = res.tuples, res.fields

YAML::dump(
tuples.map do |t|
t.fields = fields
fields.inject({}){|h,f| h[f] = t[f]; h}
end
)

of course you could write the code to have this feature either way - my point
is just that when you extract the indexes into named vars you limit the way
your code will work as the database add/drops fields... i was originally also
doing alot of this sort of thing:


def where_clause tuple, op = ' or '
clauses =
tuple.fields.map do |field|
"#{ field } = '#{ tuple[field] }'"
end

clauses.join op
end

which just started to get painful when the tuple didn't know it's own field
names... you can imagine doing alot of this stuff if you have to map fields
to var names and pass this mapping into each sql generation method...


cheers.

-a
--
===============================================================================
| EMAIL :: Ara [dot] T [dot] Howard [at] noaa [dot] gov
| PHONE :: 303.497.6469
| A flower falls, even though we love it;
| and a weed grows, even though we do not love it.
| --Dogen
===============================================================================

 

[Kirk Haines]

On Wed, 30 Jun 2004 06:37:56 +0900, Ara.T.Howard wrote

and i was reading my own code thinking - huh? so then i started doing:

res = pgconn.exec sql

tuples = res.result
fields = res.fields

tuples =
tuples.collect do |tuple|
h = {}
fields.each do |field|
h[field] = tuple.shift
end
h
end

this is NOT good if tuple.size == 42000!! finally i wrote the arrayfields
module. it still requires work:

res = pgconn.exec sql

tuples = res.result
fields = res.fields

tuples.each{|t| t.fields = fields}

but now all 42000 tuples SHARE a copy of fields for doing their
lookups - i still have to iterate over all of em - but i don't need
to create any new objects and my code new reads like:

if tuple['name'] =~ pat or tuple['age'].to_i < 42
...
end

which is a lot nicer.

Okay, admittedly, sometimes I'm too dense for my own good, but I'm
confused. It looks to me like, internally, each array is using a hash to
keep track of the fields to indices mapping, and that hash is not shared
amongst arrays with the same set of fields. So you end up using more memory
using arrayfields for something like database result sets than if you just
used hashes.

I put together a few simple little test programs to try it out, using
arrayfields and using hashes to store similar sets of data, and in practice
it looks like arrayfields uses almost 2x the amount of RAM for a given data
set, as well. Am I missing something, here?


Thanks,

Kirk Haines

 

[Ara.T.Howard]

Okay, admittedly, sometimes I'm too dense for my own good, but I'm
confused. It looks to me like, internally, each array is using a hash to
keep track of the fields to indices mapping, and that hash is not shared
amongst arrays with the same set of fields. So you end up using more memory
using arrayfields for something like database result sets than if you just
used hashes.

I put together a few simple little test programs to try it out, using
arrayfields and using hashes to store similar sets of data, and in practice
it looks like arrayfields uses almost 2x the amount of RAM for a given data
set, as well. Am I missing something, here?


Thanks,

Kirk Haines

not as dense as i am ;-) you are correct about the ram. i 'fixed' myself
into this when i was making some updates for testing - let me have a look at
it and post a fix....

-a
--
===============================================================================
| EMAIL :: Ara [dot] T [dot] Howard [at] noaa [dot] gov
| PHONE :: 303.497.6469
| A flower falls, even though we love it;
| and a weed grows, even though we do not love it.
| --Dogen
===============================================================================

 

[nobu.nokada]

Hi,

At Wed, 30 Jun 2004 05:02:58 +0900,
Ara.T.Howard wrote in [ruby-talk:104864]:

http://raa.ruby-lang.org/project/arrayfields/
http://www.codeforpeople.com/lib/ruby/arrayfields/

$ ruby-1.8 -r arrayfields.rb -e '[][:x]'
-e:1:in `[]': Symbol as array index (TypeError)
from -e:1
$ grep VERSION arrayfields.rb
VERSION = '3.1.0'

`and' and `or' operators have same precedence.

BTW, you couldn't use Struct?

tuples =
fields = Struct.new(nil, res.fields)

tuples = res.collect {|tuple| fields.new(*tuple)}

 

[nobu.nokada]

Hi,

At Wed, 30 Jun 2004 10:52:28 +0900,

$ ruby-1.8 -r arrayfields.rb -e '[][:x]'
-e:1:in `[]': Symbol as array index (TypeError)
from -e:1

Sorry, this is not a problem, I was a stupid.

 

[Ara.T.Howard]

Hi,

At Wed, 30 Jun 2004 05:02:58 +0900,
Ara.T.Howard wrote in [ruby-talk:104864]:

http://raa.ruby-lang.org/project/arrayfields/
http://www.codeforpeople.com/lib/ruby/arrayfields/

$ ruby-1.8 -r arrayfields.rb -e '[][:x]'
-e:1:in `[]': Symbol as array index (TypeError)
from -e:1
$ grep VERSION arrayfields.rb
VERSION = '3.1.0'

`and' and `or' operators have same precedence.

the instance only is extended, and only then after instance.fields= has been
called. eg.

~ > ruby -r arrayfields-3.0.0 -e 'a = [42]; a.fields = [:answer]; p a[:answer]'
42

~ > ruby -r arrayfields-3.1.0 -e 'a = [42]; a.fields = [:answer]; p a[:answer]'
42

so, in the normal case only the method Array#fields= is added when you

require 'arrayfields'

i didn't want to polute the namespace any more than needed.

BTW, you couldn't use Struct?

tuples =
fields = Struct.new(nil, res.fields)

tuples = res.collect {|tuple| fields.new(*tuple)}

i suppose i could... but there the would have a lot of object create on large
result sets no?

regards.

-a
--
===============================================================================
| EMAIL :: Ara [dot] T [dot] Howard [at] noaa [dot] gov
| PHONE :: 303.497.6469
| A flower falls, even though we love it;
| and a weed grows, even though we do not love it.
| --Dogen
===============================================================================

 

[nobu.nokada]

Hi,

At Wed, 30 Jun 2004 11:22:54 +0900,
Ara.T.Howard wrote in [ruby-talk:104892]:

i suppose i could... but there the would have a lot of object create on large
result sets no?

I don't like to hold such huge dataset in one time, so I'd use
PGresult#each instead as possible.

 

[Sean O'Dell]

Ara, any chance of getting arrayfields to not modify the Array methods but
instead simply add an accessor to Array which returns an object that
encapsulates this new functionality?

So, instead of saying:

row = ['bob', 30]
row.fields = ['name', 'age']
row['name'] #=> 'bob'

you would say something like:

row = ['bob', 30]
row.arrayfields.fields = ['name', 'age']
row.arrayfields['name'] #=> 'bob'

My thinking is that if arrayfields functionality were contained in its own
object that linked back to a parent array, you could create arrayfield
objects and attach them to other "array-like" objects, such as database rows
and so on.

But also, it might prevent problems that arise later from taking over Array's
methods where people want to use both arrayfields, and regular Array
functionality.

Just an idea.

Sean O'Dell

 

[Ara.T.Howard]

Ara, any chance of getting arrayfields to not modify the Array methods but
instead simply add an accessor to Array which returns an object that
encapsulates this new functionality?

So, instead of saying:

row = ['bob', 30]
row.fields = ['name', 'age']
row['name'] #=> 'bob'

you would say something like:

row = ['bob', 30]
row.arrayfields.fields = ['name', 'age']
row.arrayfields['name'] #=> 'bob'

My thinking is that if arrayfields functionality were contained in its own
object that linked back to a parent array, you could create arrayfield
objects and attach them to other "array-like" objects, such as database rows
and so on.

But also, it might prevent problems that arise later from taking over Array's
methods where people want to use both arrayfields, and regular Array
functionality.

well - theoretically you lose zero Array functionality since ArrayFields
methods are used if, and only if, the idx paramter is a String or Symbol which
would never be done with a normal Array.

that being said:

jib:~/shared > cat a.rb
require 'arrayfields'

class FieldedArray
attr :fieldset
def initialize fields, array
@a = array
self.fields = fields
end
def method_missing(meth, *args, &block)
@a.send(meth, *args, &block)
end
def fields= fields
extend ArrayFields unless defined? @fieldset

@fieldset =
if ArrayFields::FieldSet === fields
fields
else
ArrayFields::FieldSet.new fields
end
end
attr_reader :fieldset
def fields
@fieldset and @fieldset.fields
end
end

fa = FieldedArray.new %w(zero one two), [0, 1, 2]
fa.each_with_field{|e, f| puts "#{ f } : #{ e }"}

jib:~/shared > ruby a.rb
zero : 0
one : 1
two : 2


i can't bloody believe that worked - go matz!

what'ya think of adding the above to 3.3.0 (along with gem spec)?

-a
--
===============================================================================
| EMAIL :: Ara [dot] T [dot] Howard [at] noaa [dot] gov
| PHONE :: 303.497.6469
| A flower falls, even though we love it;
| and a weed grows, even though we do not love it.
| --Dogen
===============================================================================

 

[Sean O'Dell]

Ara, any chance of getting arrayfields to not modify the Array methods
but instead simply add an accessor to Array which returns an object that
encapsulates this new functionality?

So, instead of saying:

row = ['bob', 30]
row.fields = ['name', 'age']
row['name'] #=> 'bob'

you would say something like:

row = ['bob', 30]
row.arrayfields.fields = ['name', 'age']
row.arrayfields['name'] #=> 'bob'

My thinking is that if arrayfields functionality were contained in its
own object that linked back to a parent array, you could create
arrayfield objects and attach them to other "array-like" objects, such as
database rows and so on.

But also, it might prevent problems that arise later from taking over
Array's methods where people want to use both arrayfields, and regular
Array functionality.

well - theoretically you lose zero Array functionality since ArrayFields
methods are used if, and only if, the idx paramter is a String or Symbol
which would never be done with a normal Array.

that being said:

jib:~/shared > cat a.rb
require 'arrayfields'

class FieldedArray
attr :fieldset
def initialize fields, array
@a = array
self.fields = fields
end
def method_missing(meth, *args, &block)
@a.send(meth, *args, &block)
end
def fields= fields
extend ArrayFields unless defined? @fieldset

@fieldset =
if ArrayFields::FieldSet === fields
fields
else
ArrayFields::FieldSet.new fields
end
end
attr_reader :fieldset
def fields
@fieldset and @fieldset.fields
end
end

fa = FieldedArray.new %w(zero one two), [0, 1, 2]
fa.each_with_field{|e, f| puts "#{ f } : #{ e }"}

jib:~/shared > ruby a.rb
zero : 0
one : 1
two : 2


i can't bloody believe that worked - go matz!

what'ya think of adding the above to 3.3.0 (along with gem spec)?

I like your solution above where none of the arrayfields code goes directly
into the Array class. Benefits: no overhead for Arrays that don't use the
arrayfields feature, and the ability to work with other objects that aren't
Array's but which have array-like functionality.

Sean O'Dell