Python/Fortran interoperability

[nmm1]

I am interested in surveying people who want to interoperate between
Fortran and Python to find out what they would like to be able to do
more conveniently, especially with regard to types not supported for C
interoperability by the current Fortran standard. Any suggestions as to
other ways that I could survey such people (Usenet is no longer as
ubiquitous as it used to be) would be welcomed.

My Email address is real, so direct messages will be received.

Specifically, I should like to know the answers to the following
questions:

1) Do you want to use character strings of arbitrary length?

2) Do you want to use Python classes with list members, where the
length of the list is not necessarily fixed for all instances of the
class? Or, equivalently, Fortran derived types containing allocatable
or pointer arrays?

2) Do you want to use Fortran derived types or Python classes that
contain type-bound procedures (including finalizers)? Please answer
"yes" whether or nor you would like to call those type-bound procedures
from the other language.

4) Do you want to call functions where the called language allocates
or deallocates arrays/lists/strings for use by the calling language?
Note that this is specifically Fortran->Python and Python->Fortran.


Regards,
Nick Maclaren.

 

[viper-2]

I am interested in surveying people who want to interoperate between
Fortran and Python to find out what they would like to be able to do
more conveniently, especially with regard to types not supported for C
interoperability by the current Fortran standard. <snip>

Python is still on my "to do" list, but I know I'll be interested in
the following:

1) Do you want to use character strings of arbitrary length?

Yes

2) Do you want to use Python classes with list members, where the
length of the list is not necessarily fixed for all instances of the
class? Or, equivalently, Fortran derived types containing allocatable
or pointer arrays?

Yes

2) Do you want to use Fortran derived types or Python classes that
contain type-bound procedures (including finalizers)? Please answer
"yes" whether or nor you would like to call those type-bound procedures
from the other language.

Don't know yet

4) Do you want to call functions where the called language allocates
or deallocates arrays/lists/strings for use by the calling language?
Note that this is specifically Fortran->Python and Python->Fortran.

Yes

agt


--
Freedom - no pane, all gaiGN!

Code Art Now
http://codeartnow.com
Email: (e-mail address removed)

 

[Stefan Behnel]

I am interested in surveying people who want to interoperate between
Fortran and Python to find out what they would like to be able to do
more conveniently, especially with regard to types not supported for C
interoperability by the current Fortran standard. Any suggestions as to
other ways that I could survey such people (Usenet is no longer as
ubiquitous as it used to be) would be welcomed.

You might want to ask also on the Cython, NumPy and SciPy mailing lists.
NumPy and SciPy have a rather large audience of scientific developers, and
Cython has a running sub-project on providing better Fortran integration
(which might be of interest to you anyway).

Stefan

 

[sturlamolden]

I am interested in surveying people who want to interoperate between
Fortran and Python to find out what they would like to be able to do
more conveniently, especially with regard to types not supported for C
interoperability by the current Fortran standard.  Any suggestions as to
other ways that I could survey such people (Usenet is no longer as
ubiquitous as it used to be) would be welcomed.

I think you will find that 99.9% of Python and Fortran programmers are
scientists and engineers that also use NumPy and f2py. Go to scipy.org
and ask your question on the numpy mailing list.

Regards,
Sturla Molden

 

[nmm1]

I think you will find that 99.9% of Python and Fortran programmers are
scientists and engineers that also use NumPy and f2py. Go to scipy.org
and ask your question on the numpy mailing list.

Regards,
Sturla Molden

Thanks. I had forgotten they had a mailing list.

Nick.

 

[JB]

["Followup-To:" header set to comp.lang.fortran.]

I am interested in surveying people who want to interoperate between
Fortran and Python to find out what they would like to be able to do
more conveniently, especially with regard to types not supported for C
interoperability by the current Fortran standard. Any suggestions as to
other ways that I could survey such people (Usenet is no longer as
ubiquitous as it used to be) would be welcomed.

My Email address is real, so direct messages will be received.

Specifically, I should like to know the answers to the following
questions:

1) Do you want to use character strings of arbitrary length?

As in, a signed C int (that most Fortran implementations use to keep
track of string lengths) may not be sufficient? No, I'm not
particularly interested in that.

2) Do you want to use Python classes with list members, where the
length of the list is not necessarily fixed for all instances of the
class? Or, equivalently, Fortran derived types containing allocatable
or pointer arrays?
Yes.

2) Do you want to use Fortran derived types or Python classes that
contain type-bound procedures (including finalizers)? Please answer
"yes" whether or nor you would like to call those type-bound procedures
from the other language.

In python I use it all the time, haven't used any F2003 OOP features
yet.

4) Do you want to call functions where the called language allocates
or deallocates arrays/lists/strings for use by the calling language?
Note that this is specifically Fortran->Python and Python->Fortran.

Yes.


Generally speaking, f2py today is not that bad, though it's getting
long in the tooth. There is a project called fwrap that aims to create
an improved python/Fortran bridge:

http://conference.scipy.org/static/wiki/smith_fwrap.pdf

This project uses ISO_C_BINDING, and I think that this is the correct
approach rather than trying to keep up with whatever ABI's all those
Fortran compilers use. So from the Fortran side of the fence, I
suppose the path forward would be to improve on the C binding
functionality (this would also of course benefit other language
bindings than just python). Whether TR 29113 is the right path forward
or not I have no strong opinion on. Specifically what's needed is some
way to portably access the array descriptor data, and maybe also how
to access the OOP functionality in a standardized way. The experience
with C++ ABI's suggests that this might not be as straightforward as
it sounds.

 

[nmm1]

["Followup-To:" header set to comp.lang.fortran.]

Sorry - set back again, because you don't provide an Email address,
and there's a significant issue. Thanks for the response.

As in, a signed C int (that most Fortran implementations use to keep
track of string lengths) may not be sufficient? No, I'm not
particularly interested in that.

No, I mean things like 'Kilroy was here'. Currently, Fortran's C
interoperability supports only strings of length 1, and you have
to kludge them up as arrays. That doesn't work very well, especially
for things like function results.

Generally speaking, f2py today is not that bad, though it's getting
long in the tooth. There is a project called fwrap that aims to create
an improved python/Fortran bridge:

Thanks. I will look at that.

This project uses ISO_C_BINDING, and I think that this is the correct
approach rather than trying to keep up with whatever ABI's all those
Fortran compilers use. So from the Fortran side of the fence, I
suppose the path forward would be to improve on the C binding
functionality (this would also of course benefit other language
bindings than just python). Whether TR 29113 is the right path forward
or not I have no strong opinion on. Specifically what's needed is some
way to portably access the array descriptor data, and maybe also how
to access the OOP functionality in a standardized way. The experience
with C++ ABI's suggests that this might not be as straightforward as
it sounds.

That is precisely what I am investigating. TR 29113 falls a LONG
way before it gets to any of the OOP data - indeed, you can't even
pass OOP derived types as pure data (without even the functionality)
in its model. Nor most of what else Python would expect.


Regards,
Nick Maclaren.

 

[Dennis Lee Bieber]

No, I mean things like 'Kilroy was here'. Currently, Fortran's C
interoperability supports only strings of length 1, and you have
to kludge them up as arrays. That doesn't work very well, especially
for things like function results.

That's a C language problem -- since a string in C is just an array
of character. The last FORTRAN dialect (and implementation) I used
passed strings using a "character string descriptor" structure
(something containing the lower and upper bounds, and a pointer to the
actual string buffer, along with, I believe, flags that signified if the
calling program can handle reassignment of the pointer and/or bounds
values -- FORTRAN strings were typically static, the called function
could not reallocate to fit in longer data and update the descriptor to
match)

Of course, the languages on VMS had support for declaring arguments
to be passed in modes non-native to the language (%val() would put the
value of the argument, rather than the normal address, on to the
stack... %loc() would retrieve the address -- %val(%loc()) thereby being
the normal FORTRAN reference method.)

 

[sturlamolden]

That is precisely what I am investigating.  TR 29113 falls a LONG
way before it gets to any of the OOP data - indeed, you can't even
pass OOP derived types as pure data (without even the functionality)
in its model.  Nor most of what else Python would expect.

I am note sure what you mean. This has the same ABI:

typedef struct {
int m, n;
float r;
} myctype;


use iso_c_binding
type, bind(c) :: myftype
integer(c_int) :: m, n
real(c_float) :: s
end type

You thus can pass derived types between C and Fortran.

 

[sturlamolden]

        That's a C language problem -- since a string in C is just an array
of character. The last FORTRAN dialect (and implementation) I used
passed strings

values -- FORTRAN strings were typically static, the called function
could not reallocate to fit in longer data and update the descriptor to
match)

It is possible to pass an arbitrary length. You have to convert a
pointer from C to Fortran:

subroutine foobar(cstr) bind(c, name='foobar')
use, intrinsic :: iso_c_binding
type(c_ptr) :: cstr
character(*), pointer :: fstr
call c_f_pointer(cptr, fptr)

If you need the length, call strlen from libc.

I think the OP has misunderstood how the Fortran 2003 C bindings work.
They don't convert C pointers to Fortran strings or Fortran arryas.
They convert C pointers to Fortran pointers.


Sturla

 

[sturlamolden]

subroutine foobar(cstr) bind(c, name='foobar')
    use, intrinsic :: iso_c_binding
    type(c_ptr) :: cstr
    character(*), pointer :: fstr
    call c_f_pointer(cptr, fptr)

Which means that you can write a wrapper in Fortran callable from C,
that calls a Fortran routine you want to expose:

subroutine wrap_foobar(cstr) bind(c, name='foobar')
use, intrinsic :: iso_c_binding
type(c_ptr) :: cstr
character(*), pointer :: fstr
call c_f_pointer(cptr, fptr)
call foobar(fstr)
end subroutine

subroutine foobar(fstr)
character(*) :: fstr
! whatever
end subroutine

This by the way is how wrap exposes Fortran functions to Cython.

 

[Richard Maine]

I am note sure what you mean. ...
You thus can pass derived types between C and Fortran.

You missed the word "OOP", which seemed like the whole point. Not that
the particular word is used in the Fortran standard, but it isn't hard
to guess that he means a derived type that uses some of the OOP
features. Inheritance, polymorphism, and type-bound procedure (aka
methods in some other languages) come to mind. Since you say that you
haven't used any of the F2003 OOP features, it isn't too surprising that
you'd miss the allusion.

 

[sturlamolden]

You missed the word "OOP", which seemed like the whole point. Not that
the particular word is used in the Fortran standard, but it isn't hard
to guess that he means a derived type that uses some of the OOP
features. Inheritance, polymorphism, and type-bound procedure (aka
methods in some other languages) come to mind.

But C is not OOP. The ISO C bindings in Fortran are not ISO C++
bindings. This is for a reason: C++ does not have a standard ABI like
ISO C.

 

[Richard Maine]

But C is not OOP. The ISO C bindings in Fortran are not ISO C++
bindings. This is for a reason: C++ does not have a standard ABI like
ISO C.

That seems irrelevant to the question originally asked, which was

Yes, it is no surprise that the C interop stuff fails to address this,
since it isn't in C. Something different/extra would be needed, which is
exactly what Nick said. I'm going to jump out of the middle of this now.
The only reason I jumped in was to point out the gap in communication,
where Nick said that the TR doesn't handle "OOP derived types" and you
replied that it does so do derived types, omitting the OOP part, which
was clearly (to me) Nick's whole point.

However, I don't seem to be facilitating the communication, as the
replies just seem to be wandering farther afield, or maybe it is around
in a circle. I think you just explained that the ISO C bindings in
Fortran don't handle what Nick is asking about... but that somehow this
makes it puzzling that Nick also says that they don't handle it. Well,
I'll leave it to Nick and you, I guess. I'm lost.

 

[sturlamolden]

subroutine foobar(cstr) bind(c, name='foobar')
    use, intrinsic :: iso_c_binding
    type(c_ptr) :: cstr
    character(*), pointer :: fstr
    call c_f_pointer(cptr, fptr)

Actually, this does not work, as it is illegal to create a pointer to
a character(*). However, we can create a pointer to a huge string (say
2 GB or whatever maximum the system allows), and slice that down to a
substring using strlen to obtain the length. So here is how to pass a
variable-length string from C to Fortran, tested with gcc and gfortran
4.1.1.


In foobar.f03:

subroutine wrap_foobar(cstr) bind(c, name='foobar')

! a wrapper for foobar we expose to C

use, intrinsic :: iso_c_binding

interface
function strlen(cstr) bind(c, name='strlen')
use, intrinsic :: iso_c_binding
integer(c_int) :: strlen
type(c_ptr), value :: cstr
end function strlen
end interface

type(c_ptr), value :: cstr
character(2147483647), pointer :: p_fstr
integer :: n

n = strlen(cstr)
call c_f_pointer(cstr, p_fstr)
call foobar(p_fstr(1:n))

end subroutine


subroutine foobar(fstr)
! this is the Fortran function we want to call from C
! it takes a variable length string as argument and print its
length
character(*) :: fstr
write (*,*) len(fstr)
end subroutine


In main.c:

extern void foobar(char *);

int main(int argc, char *argv[])
{
foobar(argv[1]);
return 0;
}






Sturla@STURLA-PC /d/fortrantest
$ gfortran -c foobar.f03

Sturla@STURLA-PC /d/fortrantest
$ gcc -c main.c

Sturla@STURLA-PC /d/fortrantest
$ gcc -o test.exe main.o foobar.o -lgfortran

Sturla@STURLA-PC /d/fortrantest
$ ./test 1234
4

Sturla@STURLA-PC /d/fortrantest
$ ./test 0123456789
10

So it works...




Regards,
Sturla Molden

 

[sturlamolden]

Yes, it is no surprise that the C interop stuff fails to address this,
since it isn't in C. Something different/extra would be needed, which is
exactly what Nick said. I'm going to jump out of the middle of this now.
The only reason I jumped in was to point out the gap in communication,
where Nick said that the TR doesn't handle "OOP derived types" and you
replied that it does so do derived types, omitting the OOP part, which
was clearly (to me) Nick's whole point.

You are right, I missed th OOP part.

It might be interesting in teh future to say inherit a Python class
from a Fortran 2003 derived type. I have no idea how to do this (I
don't know Fortran 2003 that well). But possibly one could do some
magic with the ISO C bindings, exposing the Fortran 2003 derived type
as a C etxtension class to Python.

Does anyone use OOP in Fortran anyway? I sure do not. And Fortran 2003
compilers are not ubiquitous. Fortran compilers tend to support a
subset for Fortran 2003, usually ISO C bindings but not OOP.


Sturla

 

[Richard Maine]

Does anyone use OOP in Fortran anyway?

Presumably not many people yet because...

And Fortran 2003 compilers are not ubiquitous.

I'd not only agree, I'd say that was quite a bit understated. Last time
I checked, the number of Fortran 2003 compilers available on the most
widely used platforms was zero, which is quite a bit short of
ubiquitous. No I don't have an IBM workstation handy; or a Cray. Nor do
most people.

I did try playing with the Fortran OOP stuff quite a few years ago, when
NAG first added some of the OOP features to their f95 compiler. But
alas, although they added some of the OOP features, there were critical
parts that were not included. Play with it a little is all I could do. I
couldn't write any serious code, or even "play" very extensively. (Lack
of allocatable scalars was a show stopper; it seems so simple and isn't
even normally identified as an OOP feature per se, but you need it to do
much with OOP in F2003).

I'm told that NAG's 5.2 finally has the needed stuff, but it has been an
awful long time in coming... and still isn't yet here for anything but
Linux (I'm told "soon"). Some other compilers are also getting there.
But there is just no way that most people have spent much time
developing with compilers that adequately supported the f2003 OOP
features.

(I'd cite my formal comment on f2008, and maybe David Muxworthy's recent
article as well, but we've already been down that path.)

 

[Dennis Lee Bieber]

I think the OP has misunderstood how the Fortran 2003 C bindings work.
They don't convert C pointers to Fortran strings or Fortran arryas.
They convert C pointers to Fortran pointers.

Most of my FORTRAN experience is DEC/VMS F77, with a summer of
rudimentary F90... Interlanguage bindings weren't part of those
standards as I recall, it was all proprietary extensions (the DEC
%descr(), %loc(), %val(), %ref() functions... Of course, in C %val() was
a no-op, whereas %ref() was a no-op in FORTRAN -- except for strings
where it would prevent the descriptor from being built/passed <G>

 

[nmm1]

But C is not OOP. The ISO C bindings in Fortran are not ISO C++
bindings. This is for a reason: C++ does not have a standard ABI like
ISO C.

Nor does C. Almost everything that most people believe about C is
wrong, because C is not well-defined at any level, so there are
many twisty little C languages, all different.

Richard is perfectly correct that my point was OOP. C interoperability
does not apply to any derived type with type-bound procedures, which
include finalizers. Note that this ALSO forbids them from being
passed as data, even if the other language never uses the OOP features
of the type.


Regards,
Nick Maclaren.

 

[sturlamolden]

Nor does C.  Almost everything that most people believe about C is
wrong, because C is not well-defined at any level, so there are
many twisty little C languages, all different.

Richard is perfectly correct that my point was OOP.  C interoperability
does not apply to any derived type with type-bound procedures, which
include finalizers.  Note that this ALSO forbids them from being
passed as data, even if the other language never uses the OOP features
of the type.

You also made this claim regarding Fortran's C interop with strings:


"No, I mean things like 'Kilroy was here'. Currently, Fortran's C
interoperability supports only strings of length 1, and you have
to kludge them up as arrays. That doesn't work very well, especially
for things like function results."


This obviosuly proves you wrong:


subroutine foobar(fstr)
! this is the Fortran function we want to call from C
! it takes a variable length string as argument and
! print its length
character(*) :: fstr
write (*,*) len(fstr)
end subroutine



subroutine wrap_foobar(cstr) bind(c, name='foobar')
! a wrapper for foobar which we expose to C
use, intrinsic :: iso_c_binding

interface
function strlen(cstr) bind(c, name='strlen')
use, intrinsic :: iso_c_binding
integer(c_int) :: strlen
type(c_ptr), value :: cstr
end function strlen
end interface

type(c_ptr), value :: cstr
character(2147483647), pointer :: p_fstr
integer :: n

n = strlen(cstr)
call c_f_pointer(cstr, p_fstr)
call foobar(p_fstr(1:n))

end subroutine


I am not saying you are wrong regarding OOP derives types, but I'm not
taking your word for it. However, I am not wasting my time on Fortran
2003 OOP now, as it lacks compiler support. So I'll leave your claim
uncontested.

By the way, I am more than happy using Python for OOP and Fortran 95
for speed. As for numerical computing, I'd rather see better support
for array and matrix types in Cython than OOP in Fortran.


Regards,
Sturla Molden