Linear Algebra in C++

[et al.]

Hi! I am looking for some linear algebra libraries in C++... without
much success!

Basically, I found that BLAS and LAPACK can, somehow, be interacted
with in C++. However, the whole OO approach of C++ is lost (at least to
my sight).

Do you have any suggestions, apart from building my own wrapper?


PS. C++ plus OpenMP algebra libraries would score even higher!

 

[Paul]

Hi! I am looking for some linear algebra libraries in C++... without much
success!

Basically, I found that BLAS and LAPACK can, somehow, be interacted with
in C++. However, the whole OO approach of C++ is lost (at least to my
sight).

Do you have any suggestions, apart from building my own wrapper?


PS. C++ plus OpenMP algebra libraries would score even higher!

What do you mean by the whole OO approach is lost?
Boost lib is OO isn't it?

 

[SG]

Hi! I am looking for some linear algebra libraries in C++... without
much success!

Check out "Eigen":
http://eigen.tuxfamily.org/index.php?title=Main_Page
I havn't yet used it, but it seems promising and I liked the parts of
the documentation I've read so far.

 

[Rui Maciel]

Hi! I am looking for some linear algebra libraries in C++... without
much success!

Basically, I found that BLAS and LAPACK can, somehow, be interacted
with in C++. However, the whole OO approach of C++ is lost (at least to
my sight).

Do you have any suggestions, apart from building my own wrapper?

OO stuff is usually kept out of numerical stuff mainly due to the fact
that it provides ways to abstract data and concepts that, as a side
effect, add undesired performance penalties to the routines. So, as a
consequence, as those who develop numerical analysis routines tend to want
to milk every drop of performance out of their code, there is a tendency
to keep those techniques away from numerical analysis stuff.

Another issue which tends to lead people to adopt routines that implement
the BLAS API to handle numerical stuff is that they are a de facto
standard in the computing world. So, the BLAS api may not be the most OO-
friendly API available but everyone tends to use it, mainly because
everyone uses it. Adding to this, these routines that are made available
through the BLAS API tend to be highly optimized. Substance counts more
than style.

On the other and, C++ does provide support for templates, and they are
indeed useful.

Now, regarding your question, it really depends on what you want to do
with matrices and the characteristics of the particular problem that you
intend to solve. For example, if you need to solve large systems of
linear equations which end up generating sparse matrices then you are
better off chosing routines that explicitly support sparse matrices and
provide solvers for those problems. Instead, if all you want to do is
perform basic matrix algebra on small dense matrices then your
requirements are entirely different, and you may require some other
library.

So, before giving any suggestion, what stuff do you need to do with
matrices?


Rui Maciel

 

[James Kanze]

OO stuff is usually kept out of numerical stuff mainly due to the fact
that it provides ways to abstract data and concepts that, as a side
effect, add undesired performance penalties to the routines.

I don't know if performance is the only reason (although it
certainly plays a role). A more fundamental reason is that for
the numeric processing, OO doesn't buy you anything. (This
doesn't mean that it's not useful for the surrounding layers,
which manage the calculations.)

[...]

On the other and, C++ does provide support for templates, and they are
indeed useful.

It depends on the type of numeric work you're doing. But for
anything using vectors or matrixes, you'll probably want some
form of expression analysis along the lines of that used in
blitz++. It can be done with derivation (with surprisingly no
real additional runtime overhead), but it's a lot easier using
templates. (And the inheritance used in the pre-template
versions isn't really what I would call OO.)

The more or less "standard" library for matrices and such is
blitz++ (or at least it was, but I haven't heard of anything
recent which would replace it).

 

[Jorgen Grahn]

....
The more or less "standard" library for matrices and such is
blitz++ (or at least it was, but I haven't heard of anything
recent which would replace it).

I just stumbled across Armadillo C++ Library:

Armadillo is a C++ linear algebra library (matrix maths) aiming
towards a good balance between speed and ease of use. Integer,
floating point, and complex numbers are supported, as well as a
subset of trigonometric and statistics functions. Various matrix
decompositions are provided through optional integration with LAPACK
and ATLAS libraries. A delayed evaluation approach, based on
template meta-programming, is used (during compile time) to combine
several operations into one and reduce or eliminate the need for
temporaries.

-- http://freshmeat.net/projects/arma

I haven't tried it and I don't do any linalg work myself. The delayed
evaluation using TMP reminds me of blitz++ (which I also haven't used,
but read about.)

/Jorgen

 

[Bogdan]

Hi! I am looking for some linear algebra libraries in C++... without
much success!

Basically, I found that BLAS and LAPACK can, somehow, be interacted
with in C++. However, the whole OO approach of C++ is lost (at least to
my sight).

Do you have any suggestions, apart from building my own wrapper?

PS. C++ plus OpenMP algebra libraries would score even higher!

Have a look at IT++, it has a MATLAB-like interface and does pretty
much the same thing with the advantage of using C++ instead of an
interpreted language:

http://sourceforge.net/apps/wordpress/itpp/

 

[Rui Maciel]

What do you mean by the whole OO approach is lost?
Boost lib is OO isn't it?

It's also slow and it fails to provide decent support for basic stuff such
as matrix solvers and sparse matrices. If one is forced to use a C++
library to handle matrix operations that implements OO features then,
depending on your needs, you either develop your own or go with Eigen.

According to my experience, the main (and maybe only) advantage that ublas
enjoys is being a part of boost. Other than that there are simply plenty
of alternatives out there which are much better or, if you are up for it,
you can easily develop something that better fits your needs.


Rui Maciel

 

[Rui Maciel]

Check out "Eigen":
http://eigen.tuxfamily.org/index.php?title=Main_Page
I havn't yet used it, but it seems promising and I liked the parts of
the documentation I've read so far.

Eigen is a good option, except if you need to solve systems of linear
equations whose matrix is sparse. In that case Eigen tends to suck, as
officially it doesn't offer any support for any sparse matrix solver and
unofficially it only manages to provide solvers that either don't work or
don't work well. In both cases you don't get any documentation and their
API for this stuff is not stable.

Other than that, if you don't use sparse matrices then it tends to be a
good library, probably the best around. The 3.0 version appears to be
even better.


Rui Maciel

 

[Paul]

It's also slow and it fails to provide decent support for basic stuff such
as matrix solvers and sparse matrices. If one is forced to use a C++
library to handle matrix operations that implements OO features then,
depending on your needs, you either develop your own or go with Eigen.

According to my experience, the main (and maybe only) advantage that ublas
enjoys is being a part of boost. Other than that there are simply plenty
of alternatives out there which are much better or, if you are up for it,
you can easily develop something that better fits your needs.


Rui Maciel

Another idea , not sure if its any use but..
If you have Mathcad check out the docs, there is an underlying API , with
lots of different options from creating scripts to custom DLL's.

 

[ld]

Hi! I am looking for some linear algebra libraries in C++... without
much success!

Basically, I found that BLAS and LAPACK can, somehow, be interacted
with in C++. However, the whole OO approach of C++ is lost (at least to
my sight).

Do you have any suggestions, apart from building my own wrapper?

PS. C++ plus OpenMP algebra libraries would score even higher!

Maybe MTL4 is what you are looking for?

www.mtl4.org/

Laurent.

 

[Rui Maciel]

Maybe MTL4 is what you are looking for?

www.mtl4.org/

It appears that in order to use mtl4 you have to pay at least 300 euros
for that priviledge, with the added problem that according to some tests
it's one of the slowest libraries available[1]. So, if there are plenty
of alternatives which are free, why waste money on an expensive license
that does less for more?


Rui Maciel

[1]
http://eigen.tuxfamily.org/index.php?title=Benchmark-August2008

 

[ld]

Maybe MTL4 is what you are looking for?

www.mtl4.org/

It appears that in order to use mtl4 you have to pay at least 300 euros
for that priviledge, with the added problem that according to some tests
it's one of the slowest libraries available[1].  So, if there are plenty
of alternatives which are free, why waste money on an expensive license
that does less for more?

I installed the Debian Linux package of MTL4 and examples for free
from their repository, and the benchmark you mention is pretty old
since they added a lot of optimizations during the 2009 and later. I
think it's worth at least to try both and compare the results for you
own needs.

Rui Maciel

[1]http://eigen.tuxfamily.org/index.php?title=Benchmark-August2008

Regards,

Laurent.

 

[Rui Maciel]

I installed the Debian Linux package of MTL4 and examples for free
from their repository,

The only reference which I've found to a free MTL4 was from their site,
which provided a trial version which was free for 4 weeks[1]. I've
searche debian's stable repository and I failed to find any reference to
MTL4. Is it possible to provide a link to that package?

and the benchmark you mention is pretty old
since they added a lot of optimizations during the 2009 and later.

The only benchmark simunova which I could find[2] only covers inserting
elements on a matrix object, and they only manage to compare it with
uBLAS and PETSc. I don't belive this benchmark is any relevant, as matrix
algebra operations and assorted solvers tend to be more demanding and the
most used routines. As there is no indication that MTL4 improved it's
performance in these applications, and considering that it's a paid
library, I don't see the point of giving it a try.

I
think it's worth at least to try both and compare the results for you
own needs.

Then the folks responsible for MTL4 should try to make a case for it.
Instead, the only objective information I managed to access is that MTL4,
a non-free commercial library, lags behind a set of free alternatives.
This means that giving it a try would constitute a waste of time.



Rui Maciel


[1] http://www.simunova.com/de/node/146
[2] http://www.simunova.com/system/files/mtl4_fenics.pdf

 

[Daniel]

I don't know if performance is the only reason (although it
certainly plays a role).  A more fundamental reason is that for
the numeric processing, OO doesn't buy you anything.

It can buy you something. For example, with nonlinear solvers,
abstracting out the solver and cost function, to support a common
interface to many different library implementations. Also,
abstracting operator(i,j) would be one way of supporting heterogenous
operations over a wide assortment of matrix representations -
rectangular, sparse, submatrices, transposed (without physically doing
the transpose.) For that, however, it's more common to see
overloading on different matrix data representations.

It depends on the type of numeric work you're doing.  But for
anything using vectors or matrixes, you'll probably want some
form of expression analysis along the lines of that used in
blitz++.

Have you benchmarked blitz? My team found it to be disappointing, out
performed in real problems by boost ublas (ublas has its own problems,
though.)
 
I'm not convinced that expression analysis with templates is required,
sure it makes A+B+C faster, but so does an overloaded function
sum(A,B,C).

The problem with a lot of C++ attempts at matrix libraries is that
they want to be the star. They're not the star, the stars are the
lapack and other C and FORTRAN libraries, rock solid and widely used.
The best C++ matrix libraries are the ones that provide the cleanest
interfaces to these libraries, by supporting their data
representations, and requiring minimal conversion to data
representations on input and output when calling their functions.

-- Daniel

 

[et al.]

What do you mean by the whole OO approach is lost?
Boost lib is OO isn't it?

Boost does not provide a full BLAS implementation. That was my first
guess, however, I was mistaken...

 

[et al.]

Eigen is a good option, except if you need to solve systems of linear
equations whose matrix is sparse. In that case Eigen tends to suck, as
officially it doesn't offer any support for any sparse matrix solver and
unofficially it only manages to provide solvers that either don't work or
don't work well. In both cases you don't get any documentation and their
API for this stuff is not stable.

Other than that, if you don't use sparse matrices then it tends to be a
good library, probably the best around. The 3.0 version appears to be
even better.

Thanks to both of you for pointing out Eigen! It really seems to be a
great library.

Thank you!

 

[et al.]

Now, regarding your question, it really depends on what you want to do
with matrices and the characteristics of the particular problem that you
intend to solve. For example, if you need to solve large systems of
linear equations which end up generating sparse matrices then you are
better off chosing routines that explicitly support sparse matrices and
provide solvers for those problems. Instead, if all you want to do is
perform basic matrix algebra on small dense matrices then your
requirements are entirely different, and you may require some other
library.

So, before giving any suggestion, what stuff do you need to do with
matrices?

Of course. Mainly I am implementing a FEM solver, basically requiring
inversion, transposition, and multiplication of small matrices (less
than 20 rows/cols). These will end up in a very large matrix to be
decomposed (more than 10K rows/cols). In the end, the resulting
decomposition will serve to solve one or more system of linear
equations.

So the problem may be that I need both dense and sparse (actually
banded) matrices.

May be this an issue? Would you still recommend Eigen? (as in the
previous answer to SG)

 

[Rui Maciel]

Of course. Mainly I am implementing a FEM solver, basically requiring
inversion, transposition, and multiplication of small matrices (less
than 20 rows/cols). These will end up in a very large matrix to be
decomposed (more than 10K rows/cols). In the end, the resulting
decomposition will serve to solve one or more system of linear
equations.

So the problem may be that I need both dense and sparse (actually
banded) matrices.

May be this an issue? Would you still recommend Eigen? (as in the
previous answer to SG)

Eigen is able to handle well the element matrix assembly process. It also
provides a hand full of sparse matrix data types. Where it fails, though,
is in providing solvers that handle sparse matrices, and by failing to
officially support solvers for their sparse matrix data types the library
tends to be a bit useless for FEM purposes.

I'v been developing a FEM application for a while and when I started I've
tested a hand full of libraries. Eventually I settled with Eigen. Yet,
the absence of an officially supported solver for sparse matrix types
forced me to develop my own matrix library (including support for sparse
matrices) and solvers. I never bothered to benchmark it, and certainly it
is not the sharpest tool in the shed. Yet, it does work and by developing
it I've managed to avoid wasting my time combing through Eigen's source
code in order to learn what solvers are unofficially provided at this
moment and, among them, which solvers actually do work.

I don't know if anything has changed with the incoming release of Eigen v3
but it appears that it is yet to officially support a sparse matrix
solver. Therefore, I believe that the problem I've mentioned above still
persists.


Rui Maciel

 

[ld]

I installed the Debian Linux package of MTL4 and examples for free
from their repository,

The only reference which I've found to a free MTL4 was from their site,
which provided a trial version which was free for 4 weeks[1].  I've
searche debian's stable repository and I failed to find any reference to
MTL4.  Is it possible to provide a link to that package?

Yes, here is where I pick-up the binary packages (mtl4 and examples)
(to add to Debian repositories)

URI: http://www.simunova.com/debian
Distribution: main/

Despite of the "binary" term, the package contain mainly C++ headers
so you can also have a look to the code.

I don't use it personally but I tried it few times to compare the
performances with my own code and to look at some programming technics
(mainly template meta programming).

and the benchmark you mention is pretty old
since they added a lot of optimizations during the 2009 and later.

The only benchmark simunova which I could find[2] only covers inserting
elements on a matrix object, and they only manage to compare it with
uBLAS and PETSc.  I don't belive this benchmark is any relevant, as matrix
algebra operations and assorted solvers tend to be more demanding and the
most used routines.  As there is no indication that MTL4 improved it's
performance in these applications, and considering that it's a paid
library, I don't see the point of giving it a try.

I
think it's worth at least to try both and compare the results for you
own needs.

Then the folks responsible for MTL4 should try to make a case for it.  
Instead, the only objective information I managed to access is that MTL4,
a non-free commercial library, lags behind a set of free alternatives.  
This means that giving it a try would constitute a waste of time.  

Rui Maciel

[1]http://www.simunova.com/de/node/146
[2]http://www.simunova.com/system/files/mtl4_fenics.pdf