Applying a function to a 2-D numarray

[Matt Feinstein]

Is there an optimal way to apply a function to the elements of a two-d
array?

What I'd like to do is define some function:

def plone(x):
return x+1

and then apply it elementwise to a 2-D numarray. I intend to treat the
function as a variable, so ufuncs are probably not appropriate-- I
realize that what I'm looking for won't be terrifically efficient,
but I'd like to avoid doing it in the -worst- possible way.

Some things I've looked at include things like

def applyfun(m,f):
elist = [f(e) for e in m]
return reshape(elist,m.shape)

however, I can see that this looks neat but probably generates several
copies of the array, which is not so neat.

Is there a better way?

Matt Feinstein

 

[Steven Bethard]

Is there an optimal way to apply a function to the elements of a two-d
array?

What I'd like to do is define some function:

def plone(x):
return x+1

and then apply it elementwise to a 2-D numarray. I intend to treat the
function as a variable, so ufuncs are probably not appropriate-- I
realize that what I'm looking for won't be terrifically efficient,
but I'd like to avoid doing it in the -worst- possible way.

Some things I've looked at include things like

def applyfun(m,f):
elist = [f(e) for e in m]
return reshape(elist,m.shape)

however, I can see that this looks neat but probably generates several
copies of the array, which is not so neat.

Is there a better way?

I must be missing something, because the simplest possible thing seems
to work for me:

py> import numarray as na
py> def plus1(arr):
.... return arr + 1
....
py> def apply_func(arr, f):
.... return f(arr)
....
py> a = na.arange(20, shape=(4, 5))
py> a
array([[ 0, 1, 2, 3, 4],
[ 5, 6, 7, 8, 9],
[10, 11, 12, 13, 14],
[15, 16, 17, 18, 19]])
py> apply_func(a, plus1)
array([[ 1, 2, 3, 4, 5],
[ 6, 7, 8, 9, 10],
[11, 12, 13, 14, 15],
[16, 17, 18, 19, 20]])

Is this not what you wanted?

STeVe

 

[Matt Feinstein]

I must be missing something, because the simplest possible thing seems
to work for me:

py> import numarray as na
py> def plus1(arr):
... return arr + 1
...
py> def apply_func(arr, f):
... return f(arr)
...
py> a = na.arange(20, shape=(4, 5))
py> a
array([[ 0, 1, 2, 3, 4],
[ 5, 6, 7, 8, 9],
[10, 11, 12, 13, 14],
[15, 16, 17, 18, 19]])
py> apply_func(a, plus1)
array([[ 1, 2, 3, 4, 5],
[ 6, 7, 8, 9, 10],
[11, 12, 13, 14, 15],
[16, 17, 18, 19, 20]])

Is this not what you wanted?

The problem is that I chose an example function that's too simple.
Non-trivial functions aren't so polymorphic, unfortunately.

Sorry for the confusion.

Matt Feinstein

 

[Steven Bethard]

I must be missing something, because the simplest possible thing seems
to work for me:

py> import numarray as na
py> def plus1(arr):
... return arr + 1
...
py> def apply_func(arr, f):
... return f(arr)
...
py> a = na.arange(20, shape=(4, 5))
py> a
array([[ 0, 1, 2, 3, 4],
[ 5, 6, 7, 8, 9],
[10, 11, 12, 13, 14],
[15, 16, 17, 18, 19]])
py> apply_func(a, plus1)
array([[ 1, 2, 3, 4, 5],
[ 6, 7, 8, 9, 10],
[11, 12, 13, 14, 15],
[16, 17, 18, 19, 20]])

Is this not what you wanted?

The problem is that I chose an example function that's too simple.
Non-trivial functions aren't so polymorphic, unfortunately.

Can you give an example of what you really want to do? Probably there
are numarray functions that you can use. In general, you'll do better
applying a sequence of numarray functions than operating element-wise on
an array and converting it from a list back to an array...

STeVe

 

[Matt Feinstein]

Can you give an example of what you really want to do? Probably there
are numarray functions that you can use. In general, you'll do better
applying a sequence of numarray functions than operating element-wise on
an array and converting it from a list back to an array...

Well, for example, suppose I want to modify the elements of the matrix
in some fashion. However, I'm not entirely sure how I want to do it.
As a strawman, I generate a function with a Boolean test in it that
multiplies by one factor if the matrix element is in an interval and
by a different factor if it is outside the interval

def genfunc(xmin, xmax, f_in, f_out):
def booltest(x):
in_interval = x > xmin and x < xmax
if in_interval:
return x*f_in
else:
return x*f_out
return booltest

Generating the function in this way gives me great flexibility in
deciding exactly what function I apply to the matrix. It's why I want
to use Python for this analysis. The part of the function I vary and
play around with is localized in one place in the 'genfunc' function--
I can change that and everything else stays the same. However, I
realize that the gain in flexibility means a loss in efficiency. I'm
limited to not-so-efficient ways of. For this work, it's OK-- I just
want to know the best not-so-efficient way of doing the calculation.

Matt Feinstein

 

[Steven Bethard]

Well, for example, suppose I want to modify the elements of the matrix
in some fashion. However, I'm not entirely sure how I want to do it.
As a strawman, I generate a function with a Boolean test in it that
multiplies by one factor if the matrix element is in an interval and
by a different factor if it is outside the interval

def genfunc(xmin, xmax, f_in, f_out):
def booltest(x):
in_interval = x > xmin and x < xmax
if in_interval:
return x*f_in
else:
return x*f_out
return booltest

If efficiency was a concern, I'd probably write this instead as:

py> import numarray as na
py> def genfunc(xmin, xmax, f_in, f_out):
.... def booltest(arr):
.... is_in = na.logical_and(arr > xmin, arr < xmax)
.... is_out = na.logical_not(is_in)
.... return arr*is_in*f_in + arr*is_out*f_out
.... return booltest
....
py> b = genfunc(5, 11, 2, -2)
py> arr = na.arange(20, shape=(4,5))
py> b(arr)
array([[ 0, -2, -4, -6, -8],
[-10, 12, 14, 16, 18],
[ 20, -22, -24, -26, -28],
[-30, -32, -34, -36, -38]])

Sure, it's a little more complex that your version, and you have to
understand that you're manipulating arrays, not elements of arrays, but
if you want the efficiency of numarray or Numeric, something like this
is probably the way to go.

Generating the function in this way gives me great flexibility in
deciding exactly what function I apply to the matrix. It's why I want
to use Python for this analysis. The part of the function I vary and
play around with is localized in one place in the 'genfunc' function--
I can change that and everything else stays the same. However, I
realize that the gain in flexibility means a loss in efficiency. I'm
limited to not-so-efficient ways of. For this work, it's OK-- I just
want to know the best not-so-efficient way of doing the calculation.

If you're not worried about efficiency, your initial suggestion:

def applyfun(m,f):
elist = [f(e) for e in m]
return reshape(elist,m.shape)

seems pretty reasonable. OTOH, if you're using numarray or Numeric in
the first place, you're obviously somewhat concerned about efficiency.

STeVe

 

[Robert Kern]

Well, for example, suppose I want to modify the elements of the matrix
in some fashion. However, I'm not entirely sure how I want to do it.
As a strawman, I generate a function with a Boolean test in it that
multiplies by one factor if the matrix element is in an interval and
by a different factor if it is outside the interval

As Steven Bethard suggests, using the ufuncs provided to express the
function in a vectorial way is always the best option *if* it's possible.

Otherwise, you may want to look at Scipy's vectorize() function. I don't
think the numarray port is quite working yet, so you may have to use
Numeric. You will still have the same function call overhead since you
will be calling the Python function for each element, but the loops will
be in C.

Type: classobj
String Form: scipy_base.function_base.vectorize
Namespace: Interactive
File:
/Library/Frameworks/Python.framework/Versions/2.4/lib/python2.4/site-packages/scipy_base/function_base.py
Docstring:
vectorize(somefunction) Generalized Function class.
Description:
Define a vectorized function which takes nested sequence objects or
numerix arrays as inputs and returns a numerix array as output,
evaluating the function over successive tuples of the input arrays like
the python map function except it uses the broadcasting rules of numerix
Python.

Input:
somefunction -- a Python function or method

Example:

def myfunc(a,b):
if a > b:
return a-b
else:
return a+b
vfunc = vectorize(myfunc)

>>> vfunc([1,2,3,4],2)

array([3,4,1,2])

--
Robert Kern
(e-mail address removed)

"In the fields of hell where the grass grows high
Are the graves of dreams allowed to die."
-- Richard Harter