DLL unload question for embedded Perl on Windows

[cyl]

I ran into some problems when executing the sample code from perldoc
about embedding Perl in a C program. Here are my codes

---- embeddedperl.c begin ----
#include <EXTERN.h> /* from the Perl distribution */
#include <perl.h> /* from the Perl distribution */

EXTERN_C void boot_DynaLoader (pTHX_ CV* cv);

EXTERN_C void xs_init(pTHX)
{
char *file = __FILE__;
/* DynaLoader is a special case */
newXS("DynaLoader::boot_DynaLoader", boot_DynaLoader, file);
}

void runperl()
{
int ARGC = 2;
char *ARGV[]={"","test.pl"};
PerlInterpreter *my_perl;

PERL_SYS_INIT3(&ARGC,(char ***)&ARGV,NULL);
my_perl = perl_alloc();
perl_construct(my_perl);
PL_exit_flags |= PERL_EXIT_DESTRUCT_END;
perl_parse(my_perl, xs_init, ARGC, ARGV, (char **)NULL);
perl_run(my_perl);
perl_destruct(my_perl);
perl_free(my_perl);
PERL_SYS_TERM();

}

int main(int argc, char **argv, char **env)
{
int i=0;
for (i=0;i<2;i++)
runperl();
}
---- embeddedperl.c end ----

---- test.pl begin ----
use Cwd;

print cwd,"\n";
---- test.pl end ----

Here are the problems I got during execution

1. The loaded DLLs do not unload after the Perl interpreter is
shutdown
In my example, after perl_parse() Cwd.dll will be loaded. I
expected this dll will be unloaded after calling
perl_run() but it was not. How do I force all DLLs to be unloaded
after a script finishes?
2. perl_destruct() always throws exception. I have to comment out it
for my program to run. I suspect if it can run
without problem, maybe my 1st question can be solved.
3. After commenting out perl_destruct(), my program throws exception
after calling runperl() the 2nd time. It
actually died on perl_parse(). Since I wrap everything in a sub-
routine runperl() I thought everything starts from
scratch. However It seems not. I have no idea how come it is OK
the 1st time but not the 2nd.

Hope somebody can shed light on these. Thanks very much.

 

[cyl]

I would not expect Cwd.dll to be unloaded until after perl_free is
called. It is not normal for a perl interpreter to ever unload a loaded
extension dll.

I just want to have a clean environment like a new process starts. Is
there any method to achieve this?

I suspect this may have something to do with your misuse of SYS_INIT3
and SYS_TERM, but I don't know. If fixing that doesn't help, build perl
with -DDEBUGGING and see if you get more information.

I modified my code in this way

int main(int argc, char **argv, char **env)
{
int i=0;
PERL_SYS_INIT3(&argc,&argv,NULL);
for (i=0;i<2;i++)
runperl();
PERL_SYS_TERM();
}

and the result is a bit confusing. It ran fine on the machine that
build the executable but still threw exception on perl_destruct() on
another machine. The only difference I can tell is one machine has MS
Visual Studio 2005 and the other not.

This is expected. Your perl is built with threads (since you're on
WIn32) and you are effectively trying to use two different interpreters

Multi-thread is another problem I met but didn't mention. I'm using
Active Perl 5.8.8 but my program always crash with multi-thread.
Currently I just remove the thread functions and want to solve the
problems I mentioned first.

 

[Ilya Zakharevich]

I don't know. Have you checked to see whether perl_free unloads the dlls
or not? If it does then that's what you want.

AFAIK, the design of Perl DLLs was always that they are not unloadable.

Hope this helps,
Ilya

 

[Ilya Zakharevich]

That's more-or-less what I thought, but perldoc DynaLoader says

| dl_unload_file()
| Syntax:
|
| $status = dl_unload_file($libref)
|
| Dynamically unload $libref, which must be an opaque ’library
| reference’ as returned from dl_load_file. Returns one on success
| and zero on failure.

This is just an interface to the OS's implementation; it has practically
nothing to do with Perl's DLLs.

One may keep in mind a simple analogy: in C, one can free(); but if
you have an array of (pointers to) structures, it is not enough to
free() the arena where the array content is situated. One must know
the semantic of array elements, and do recursive free()ing (or
decrement of refcounts; or whatever is needed).

dl_unload_file() is just a dumb free(). It knows nothing about what
one should do with dangling pointers inside the arena, or pointers
which were stored in the area.

Hope this helps,
Ilya

 

[Ilya Zakharevich]

You snipped the important bit. DynaLoader will in fact call
dl_unload_file (if defined) on all previously-loaded extension dlls
during global destruction.

This must be a somewhat new development; if so, I have no idea why
this is supposed to work (definitely, in my Perl DLLs, I have no
provision for unloading...).

Quite so, which is why this can't (easily) be done *before* global
destruction.

.... and, AFAIU, *after* global destruction too. Perl DLLs have BOOT:
sections; they do not have UNBOOT: ones to release resources they
allocated.

Ilya

 

[cyl]

I got my problems fixed by building my own Perl even though I prefer
to use ActivePerl. Anyway it works for now. Since I need to use a DLL
to run the Perl interpreter, I don't have the arg/argv/env arguments.
However it still works with NULL passed. Not sure if there's any
potential risks. It works fine so far. As for unloading DLL problem, I
add a compiler flag DL_UNLOAD_ALL_AT_EXIT and Perl interpreter will
unload all DLLs after perl_destruct or perl_free (sorry I forgot it).
Thanks for all the valuable information. For a layman of Perl guts,
all the information mentioned help me a lot.

 

[cyl]

Ilya is right, though, that perl extensions are not designed to be
unloaded (since DL_UNLOAD_ALL isn't normally defined). You are likely to
get memory and other resource leaks from unloading and reloading
extensions that statically allocate resources.

Ben

So your suggestion is not to enable this flag? If so, will it make any
difference in the behavior of the same script executed several times?
I am asking because my script hanged when running the 2nd time. But it
happened when I used ActivePerl. Maybe I will recompile Perl again
without this flag if I got any resource leak issue.

 

[cyl]

Ilya is right, though, that perl extensions are not designed to be
unloaded (since DL_UNLOAD_ALL isn't normally defined). You are likely to
get memory and other resource leaks from unloading and reloading
extensions that statically allocate resources.

Ben

So your suggestion is not to enable this flag? If so, will it make any
difference in the behavior of the same script executed several times?
I am asking because my script hanged when running the 2nd time. But it
happened when I used ActivePerl. Maybe I will recompile Perl again
without this flag if I got any resource leak issue.