Embedding perl in shared libs

[Bastian Friedrich]

Hi,

I am currently working on a project that uses the "perlembed" methods to run
perl functions from within a C program. The structure is similar to
Apache's "mod_perl":

Program A loads my library B with the "dlopen" POSIX system call. Library B
is linked against Perl (-lperl /path/to/Dynaloader.a). Library B
initializes a Perl interpreter C that parses a script D. In this script,
there are "use" statements for modules that in turn need binary extensions.
Perl itself uses the "dlopen" call itself to load these extensions.

dlopen takes two parameters: file path, and flags.

* When the first dlopen call (A -> B) is done _without_ "RTLD_GLOBAL"
flag, a segfault occurs during the parsing of the script.
* When linking program A with perl, everything is fine
* When using the RTLD_GLOBAL flag, everything is fine
* When script D does not load binary extensions, everything is fine.
* When statically linking library B to program A (instead of dlopening it),
everything is fine.

Obviously, there is some problem with the dlopen calls (

If you are interested in the topic, you can download a sample program from
http://www.iump.de/perl_via_dlopen.tar.gz

So my questions are:
* Is this behaviour "normal"? Is RTLD_GLOBAL in fact absolutely necessary in
that place?
* Did I really hit a perl bug??
* Do you know about any workarounds?

Thanks a lot, Regards,
Bastian

 

[Ben Morrow]

Hi,

I am currently working on a project that uses the "perlembed" methods to run
perl functions from within a C program. The structure is similar to
Apache's "mod_perl":

Program A loads my library B with the "dlopen" POSIX system call. Library B
is linked against Perl (-lperl /path/to/Dynaloader.a). Library B
initializes a Perl interpreter C that parses a script D. In this script,
there are "use" statements for modules that in turn need binary extensions.
Perl itself uses the "dlopen" call itself to load these extensions.

dlopen takes two parameters: file path, and flags.

* When the first dlopen call (A -> B) is done _without_ "RTLD_GLOBAL"
flag, a segfault occurs during the parsing of the script.
* When linking program A with perl, everything is fine
* When using the RTLD_GLOBAL flag, everything is fine
* When script D does not load binary extensions, everything is fine.
* When statically linking library B to program A (instead of dlopening it),
everything is fine.

Obviously, there is some problem with the dlopen calls (

If you are interested in the topic, you can download a sample program from
http://www.iump.de/perl_via_dlopen.tar.gz

So my questions are:
* Is this behaviour "normal"? Is RTLD_GLOBAL in fact absolutely necessary in
that place?

It seems pretty clear to me from dlopen(3):

| Optionally, RTLD_GLOBAL may be or'ed into flag, in which case the
| external symbols defined in the library will be made available for sym-
| bol resolution of subsequently loaded libraries.

That is, unless you use RTLD_GLOBAL, the symbols from -lperl (which were
pulled in either from or by libB.so, depending on whether libB is
statically or dynamically linked to libperl) won't be available to the
dynamically loaded extensions later. Since they require those symbols,
you get a segfault.

* Did I really hit a perl bug??
Nope.

* Do you know about any workarounds?

Use RTLD_GLOBAL. Is this a problem? If, for some reason, you really
don't want to load libB.so with RTLD_GLOBAL, then I guess that you could

1. build a shared libperl.so (to do this you need to rebuild perl
with -Duseshrplib),

2. *don't* link libB with -lperl, but instead

3. dlopen libperl.so from B with RTLD_GLOBAL.

Of course you would then have to dlsym perl_parse &c., and you may well
have *real* trouble getting the macros in EXTERN.h to behave. So you'd
be better off creating separate libBperl.so and libBrtld_local.so, and
having one dlopen the other.

Ben

 

[Bastian Friedrich]

Hi Ben,

thx a lot for your detailed explanation.

It seems pretty clear to me from dlopen(3):

I had read man dlopen, even more then once. Frankly, I had not fully
understood it (and probably still have not)...

| Optionally, RTLD_GLOBAL may be or'ed into flag, in which case the
| external symbols defined in the library will be made available for sym-
| bol resolution of subsequently loaded libraries.

That is, unless you use RTLD_GLOBAL, the symbols from -lperl (which were
pulled in either from or by libB.so, depending on whether libB is
statically or dynamically linked to libperl) won't be available to the
dynamically loaded extensions later. Since they require those symbols,
you get a segfault.

Program A does not need access to Perl's symbols. That's why I thought
RTLD_GLOBAL might in fact not be necessary. Obviously, I misunderstood the
GLOBAL flag.

Nope.

Good.

Use RTLD_GLOBAL. Is this a problem?

In a way. I'm not writing/maintaining the core program, I just develop a
module for it. Unfortunately, a second (standard) module segfaults when
loaded with RTLD_GLOBAL. If I want to load all modules with the same flags,
I only had the choice between a segfaulting perl and a segfaulting standard
module.

Currently, I introduced a configuration flag that lets users modify the
loading behaviour. By this, my perlish module may be loaded /with/
RTLD_GLOBAL, while others are not. We will see whether I'll stick with
that...

Anyway, I now have an answer to the core question "do I really need
RTLD_GLOBAL". Thx again!

Bastian