Problem with dlopen

[Unknown Soldier]

Hello,

I have a program that makes use of external plugins, implemented as
shared libraries loaded at runtime through a call to dlopen(). The
libraries export symbols which the main program collects through a call
to dlsym().

The problem is that the libraries refer to certain symbols from the main
program. But the linker at runtime fails to resolve these "undefined
symbols" in the shared library.

I think I may have misunderstood something in how dlopen() works - can
anyone give me some advice?

Thanks!

 

[Antoninus Twink]

Hello,

I have a program that makes use of external plugins, implemented as
shared libraries loaded at runtime through a call to dlopen(). The
libraries export symbols which the main program collects through a call
to dlsym().

The problem is that the libraries refer to certain symbols from the main
program. But the linker at runtime fails to resolve these "undefined
symbols" in the shared library.

I think I may have misunderstood something in how dlopen() works - can
anyone give me some advice?

Did you link the main program with -rdynamic? If not, only the symbols
the external library knows it needs at link-time will be exported.

 

[Walter Roberson]

I have a program that makes use of external plugins, implemented as
shared libraries loaded at runtime through a call to dlopen().

dlopen() is not part of standard C, and the exact operations of
dlopen() depend upon your operating system. Please contact a
development group that deals with your particular operating system.

If I recall correctly from my scanning, on -some- systems,
"backwards" referrals from a shared library can only be implemented
by having the shared library itself dlopen the "earlier" object
(e.g., in your case, the library might have to dlopen() the main
executable.) In other systems, a similar effect might be in place
but rather than having to "manually" dlopen() the earlier objects,
just having the shared library refer to the object in its headers
is enough to trigger a transitive dlopen(). And then there are other
systems that these restrictions don't apply to and it should Just Work,
and other systems on which it should Just Work but only if the
object backwards-referenced was defined as a global object. Different
systems, different rules, so you need to check the rules for -your-
system.

 

[dj3vande]

I think I may have misunderstood something in how dlopen() works - can
anyone give me some advice?

It smells like POSIX to me; if true, that means the people in
comp.unix.programmer probably could.


dave

 

[Unknown Soldier]

That did the trick, thanks a lot for the quick answer!

 

[Fei Liu]

Hello,

I have a program that makes use of external plugins, implemented as
shared libraries loaded at runtime through a call to dlopen(). The
libraries export symbols which the main program collects through a call
to dlsym().

The problem is that the libraries refer to certain symbols from the main
program. But the linker at runtime fails to resolve these "undefined
symbols" in the shared library.

I think I may have misunderstood something in how dlopen() works - can
anyone give me some advice?

Thanks!

Do you have some sample code to demonstrate this technique, this sounds
interesting.

Fei

 

[Antoninus Twink]

Do you have some sample code to demonstrate this technique, this sounds
interesting.

Here's a simple example with the shared library code referencing a
symbol from the main executable:


/* prog.c */

#include <stdio.h>
#include <stdlib.h>
#include <dlfcn.h>

int main(void)
{
void (*f)(void);
void *handle;

handle = dlopen ("./shared.so", RTLD_NOW);
if (!handle)
abort();

f = dlsym (handle, "f");
if (!f)
abort();
f();

if(dlclose(handle))
abort();
return 0;
}

void g(void)
{
puts("Calling g...");
}





/* shared.c */

#include <stdio.h>

extern void g(void);

void f(void)
{
puts("Calling f...");
g();
}




$ make prog LDFLAGS=-rdynamic LDLIBS=-ldl
cc -rdynamic prog.c -ldl -o prog
$ make shared.o
cc -c -o shared.o shared.c
$ ld -shared -o shared.so shared.o
$ ./prog
Calling f...
Calling g...

 

[Keith Thompson]

Here's a simple example with the shared library code referencing a
symbol from the main executable:

[...]

Why don't you post this in comp.unix.programmer?

 

[Richard]

Here's a simple example with the shared library code referencing a
symbol from the main executable:

[...]

Why don't you post this in comp.unix.programmer?

Because it was requested here and it's in C.

Why do you ask? It seems fairly obvious. You use Gnus. If you are not
interested then kill the thread. Further discussions can move to the
other NG when it starts to deepen.

 

[jxh]

...
[dlopen on library that uses symbols in main program fails]

Did you link the main program with -rdynamic? If not, only
the symbols the external library knows it needs at link-time
will be exported.

That did the trick, thanks a lot for the quick answer!

While it is fortunate the answer worked for you, it was by pure
luck since there is no way to actually know what system you are
working on. You did not provide enough information to really
provide a correct answer.

The "-rdynamic" flag may not have been available on the compiler
you were using. You may have needed to find a flag that served a
similar function.

There may have been no way of accomplishing this from your
compiler, and you would have then needed to read the
documentation on your linker to find out how to build a binary
properly to give you similar properties.

Or, you may have been required to export those parts of your code
that is needed by the dynamically loaded library into a different
library that the dynamically loaded library can link against.

These issues may not feel relevant to you now, but they should be
kept in mind for when you change jobs and you find yourself needing
to something very similar on a very alien development environment.

As you can see from the explanations provided, none of the solutions
(including the one that apparently worked for you) had anything to
do with C programming or the C language.

A technique that is sometimes used in C development to get around
such issues without linker tricks is to use explicit symbol
registration APIs. After using magic to load in a plugin, the
plugin should provide an entry point to accept symbol registration.
Let's call this entry point "init_symbols()".

struct external_symbols g_es;
struct internal_symbols g_is;

const struct internal_symbols *
init_symbols (const struct external_symbols *s)
{
g_es = *s;
return &g_is;
}

After the main program calls init_symbols, the plugin can refer
to g_es to get to the symbols exported by the main program, and
the main program can refer to the return value of the function
to refer to symbols exported by the plugin.

struct external_symbols {
int (*foo)(void);
int (*bar)(void);
};

struct internal_symbols {
int (*stage1)(void);
int (*stage2)(void);
};

Of course, the actual magic for loading a plugin and getting
to the entry point varies depending on the system (but in your
case they seem to presently be dlopen and dlsym).

-- James

 

[Fei Liu]

Here's a simple example with the shared library code referencing a
symbol from the main executable:


/* prog.c */

#include <stdio.h>
#include <stdlib.h>
#include <dlfcn.h>

int main(void)
{
void (*f)(void);
void *handle;

handle = dlopen ("./shared.so", RTLD_NOW);
if (!handle)
abort();

f = dlsym (handle, "f");
if (!f)
abort();
f();

if(dlclose(handle))
abort();
return 0;
}

void g(void)
{
puts("Calling g...");
}





/* shared.c */

#include <stdio.h>

extern void g(void);

void f(void)
{
puts("Calling f...");
g();
}




$ make prog LDFLAGS=-rdynamic LDLIBS=-ldl
cc -rdynamic prog.c -ldl -o prog
$ make shared.o
cc -c -o shared.o shared.c
$ ld -shared -o shared.so shared.o
$ ./prog
Calling f...
Calling g...

Very interesting, thanks a lot!

Fei