newbie: working iwth list of tuples

[falcon]

Hi All,
I am fairly new to Python (less than a week). My goal is to write a
small prototype of a database. Rather than build it using the typical
method where one provides selection, projection, aggregation, union,
intersection, etc. functions, I would like to do it in a more
'functional' style (executing a database query by using map, reduce,
filter, etc. I am curious what the best way is to work with a list of
tuples. For example, if I have the following list:
[('a',1),('a',1),('a',3),('b',1),('b',2),('c',2),('c',3),('c',4)]
how do I apply a map to each item in each tuple in the list? What if I
want to filter based on both elements (the number AND the letter)?

The most obvious solution is just to provide a function or a lambda
expression whcih does what I want. However, since this is, eventually,
going to be a database, the user will decide at run time what functions
and operations should be run on a 'table' (really a list of tuples) ...
and the structure of the 'table' will obviously not be known until
run-time.

I didn't find any examples of using idioms from functional programming
beyond basic explanation of reduce and map (I have a couple of Haskell
and ML books, but I know those languages even less than Python).

I hope you understand the problem. Thanks in advance.

 

[Paul Rubin]

I didn't find any examples of using idioms from functional programming
beyond basic explanation of reduce and map (I have a couple of Haskell
and ML books, but I know those languages even less than Python).

The book you really want is SICP,

http://mitpress.mit.edu/sicp/

The full text is online, linked from there.

 

[falcon]

I forgot to add that I passing a tuple of functions to the reduce
function but apparently that is not allowed. My guess was that a tuple
made up of individual (simple) functions might be easier to manipulate
programatically than a function which has to know the structure of a
list.

(My test code)
x=[('a',1),('a',1),('a',3),('b',1),('b',2),('c',2),('c',3),('c',4)]
reduce((lambda x,y: x+y,lambda x,y: x+y),x)

 

[Paul Rubin]

I forgot to add that I passing a tuple of functions to the reduce
function but apparently that is not allowed. My guess was that a tuple
made up of individual (simple) functions might be easier to manipulate
programatically than a function which has to know the structure of a
list.

Python really doesn't support this style all that well. Try Haskell.
See also the SICP book, which uses Scheme.

(My test code)
x=[('a',1),('a',1),('a',3),('b',1),('b',2),('c',2),('c',3),('c',4)]
reduce((lambda x,y: x+y,lambda x,y: x+y),x)

I can't even tell what you're trying to do there.

 

[Raymond Hettinger]

[falcon]

I am fairly new to Python (less than a week). My goal is to write a
small prototype of a database. Rather than build it using the typical
method where one provides selection, projection, aggregation, union,
intersection, etc. functions, I would like to do it in a more
'functional' style (executing a database query by using map, reduce,
filter, etc. I am curious what the best way is to work with a list of
tuples. For example, if I have the following list:
[('a',1),('a',1),('a',3),('b',1),('b',2),('c',2),('c',3),('c',4)]
how do I apply a map to each item in each tuple in the list? What if I
want to filter based on both elements (the number AND the letter)?

The most obvious solution is just to provide a function or a lambda
expression whcih does what I want. However, since this is, eventually,
going to be a database, the user will decide at run time what functions
and operations should be run on a 'table' (really a list of tuples) ...
and the structure of the 'table' will obviously not be known until
run-time.

I didn't find any examples of using idioms from functional programming
beyond basic explanation of reduce and map (I have a couple of Haskell
and ML books, but I know those languages even less than Python).

I hope you understand the problem. Thanks in advance.

This is a somewhat advanced problem for your first week in Python.

Essentially, what you need to do is write parameterized helper
functions to pass to map(), filter(), and reduce().

First, look at a non-parameterized example. Given a record, append a
new field that is the sum of fields 1 and 3:

def z(record):
sum = record[1] + record[3]
return record + (sum,)

Apply it to the database with:

result = map(z, database)

Now, make a parameterized version that allows the user specified fields
and operations:

def pfunc(inputfields, operation):
def z(record):
newfield = operation(*[record[f] for f in inputfields])
return record + (newfield,)
return z

z = pfunc((1, 3), operator.add) # parameters specified by the
user at runtime

result = map(z, database)

Parameterized filter, extract, and reduce functions can be handled in a
like manner.


Raymond

 

[Raymond Hettinger]

[Raymond Hettinger]

Parameterized filter, extract, and reduce functions can be handled in a
like manner.

Just for grins, here is a more worked-out example:

def pfunc(inputfields, operation):
"Parameterized computation of a new field"
# For example, append a field that is the sum of fields 1 and 3:
# z = pfunc((1, 3), operator.add)
# print map(z, database)
def z(record):
newfield = operation(*[record[f] for f in inputfields])
return record + (newfield,)
return z

def rfunc(inputfield, operation):
"Parameterized reduce operation"
#
# For example, find the maximum value of field 2:
# r = rfunc(2, max)
# print reduce(r, database)
def z(cumval, record):
x = record[inputfield]
return operation(cumval, x)
return z

def filt_func(inputfields, operation):
"Parameterized filter operation"
#
# For example, get records where field1 < field2:
# f = filt_func((1, 3), operator.lt)
# print filter(f, database)
def z(record):
i, j = inputfields
return operation(i, j)
return z

def xfunc(fields):
"Parameterized extract operation"
#
# For example, extract fields 1, 3, and 4
# x = xfunc((1,3,4))
# print map(x, database)
def z(record):
return tuple([record[f] for f in fields])
return z


# ---- The examples can be run on the following sample database: ----

database = [
(10, 25, 30, 40, 50, 60),
(100, 250, 300, 400, 500, 600),
(1, 2.5, 3, 4, 5, 6),
]

 

[John Bauman]

I forgot to add that I passing a tuple of functions to the reduce
function but apparently that is not allowed. My guess was that a tuple
made up of individual (simple) functions might be easier to manipulate
programatically than a function which has to know the structure of a
list.

Python really doesn't support this style all that well. Try Haskell.
See also the SICP book, which uses Scheme.

(My test code)
x=[('a',1),('a',1),('a',3),('b',1),('b',2),('c',2),('c',3),('c',4)]
reduce((lambda x,y: x+y,lambda x,y: x+y),x)

I can't even tell what you're trying to do there.

I think that this is what he wants.
def zipfunc(fun):
def runfun(*args):
return tuple(x(*y) for x,y in zip(fun,zip(*args)))
return runfun

Use this as:
x=[('a',1),('a',1),('a',3),('b',1),('b',2),('c',2),('c',3),('c',4)]
reduce(zipfunc((lambda x,y: x+y,lambda x,y: x+y)),x)
I think he wants the apply the first function in the tuple to the first
element in every tuple, and likewise the second, third, etc.

 

[Raymond Hettinger]

[Raymond Hettinger]

Just for grins, here is a more worked-out example:

See the ASPN Cookbook recipe for a version with doctests and a couple
bug-fixes:

http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/473803

 

[Kent Johnson]

Hi All,
I am fairly new to Python (less than a week). My goal is to write a
small prototype of a database. Rather than build it using the typical
method where one provides selection, projection, aggregation, union,
intersection, etc. functions, I would like to do it in a more
'functional' style (executing a database query by using map, reduce,
filter, etc.

You might be interested in a similar attempt:
http://www.jtauber.com/relational_python

Kent

 

[falcon]

Wow, great answers here. Thank you all for the links and examples, I'm
going through them all now.