Perl callback from c, multiple threads, cloned interpreter

[Jeff]

Hello,

I'm having a problem passing a function parameter to a perl callback
function being call from C w/ a cloned interpreter. The reason for the
cloned interpreter is that the perl callback function is being called
from a different thread, and perl does not currently support multiple
threads concurrently calling into the same interpreter.

The below function works perfectly if I do not clone the interpreter.
But when I clone the interpreter, the function parameter passed on the
stack does not make its way to the function. When the perl function
attempts to use this varaible, it's always 0.

Is there a special step I need to take to setup the function parameter
on the stack w/ a cloned interpreter? I've tried passing a reference
to the variable, and this also does not work.

Any insight would be greatly appreciated.

Thanks,
Jeff

static void PerlCallBack(void *clientdata)
{

//clone perl interpreter and set it as context
perl_clone((PerlInterpreter*)Perl_get_context(),
CLONEf_CLONE_HOST|CLONEf_KEEP_PTR_TABLE);

PUSHMARK(SP) ;

//push integer, 2, onto stack as parameter for test purposes
XPUSHs(sv_2mortal(newSViv(2)));
PUTBACK ;

//call perl function passed via clientdata
perl_call_sv((SV*) clientdata, G_DISCARD) ;
LEAVE;

}

 

[Tassilo v. Parseval]

Also sprach Jeff:

I'm having a problem passing a function parameter to a perl callback
function being call from C w/ a cloned interpreter. The reason for the
cloned interpreter is that the perl callback function is being called
from a different thread, and perl does not currently support multiple
threads concurrently calling into the same interpreter.

The below function works perfectly if I do not clone the interpreter.
But when I clone the interpreter, the function parameter passed on the
stack does not make its way to the function. When the perl function
attempts to use this varaible, it's always 0.

Is there a special step I need to take to setup the function parameter
on the stack w/ a cloned interpreter? I've tried passing a reference
to the variable, and this also does not work.

Any insight would be greatly appreciated.

Thanks,
Jeff

static void PerlCallBack(void *clientdata)
{

//clone perl interpreter and set it as context
perl_clone((PerlInterpreter*)Perl_get_context(),
CLONEf_CLONE_HOST|CLONEf_KEEP_PTR_TABLE);

I am not actually an expert when it comes to working with several perl
interpreters. However, are you sure that the above comment correctly
describes what your code is doing? perl_clone() returns the new
interpreter. You are not doing anything with the return value.

I would assume, something like

PerlInterpreter *p = perl_clone(PERL_GET_CONTEXT,
CLONEf_CLONE_HOST|CLONEf_KEEP_PTR_TABLE);
PERL_SET_CONTEXT(p);

is needed.

PUSHMARK(SP) ;

//push integer, 2, onto stack as parameter for test purposes
XPUSHs(sv_2mortal(newSViv(2)));
PUTBACK ;

//call perl function passed via clientdata
perl_call_sv((SV*) clientdata, G_DISCARD) ;

The (P|p)erl_ form of the various functions should no longer be used.
It's simply call_sv().

Tassilo

 

[Jeff]

The perl_clone() functon creates a new interpreter and set's it as the
current context. There is no need to call PERL_SET_CONTEXT again.

Thanks for the input though.

 

[Tassilo v. Parseval]

Also sprach Jeff:

The perl_clone() functon creates a new interpreter and set's it as the
current context. There is no need to call PERL_SET_CONTEXT again.

I am still not entirely convinced that a mere perl_clone() is all you
have to do. It has to be made sure that in the current executing scope
there is a variable 'my_perl' that points to the actual interpreter.
Neither perl_context() nor my initial suggestion of PERL_SET_CONTEXT
ensure that. What happens if you do

dTHXa(perl_clone(aTHX, ...));

?

That will set pTHX which is #defined to be my_perl.

Tassilo

 

[Jeff]

The function call is executing in the new perl context. I can see this
in the debugger, and if it wasn't executing in the newly cloned
context, it would work.

I believe the problem is that the function reference I'm calling is
from the parent interpreter, and its CV value is pointing to the parent
interpreter, not the cloned interpreter.

Any ideas on how to fix this?

 

[Tassilo v. Parseval]

[ I am sure it would increase readability of this thread
if you quoted a bit of context. ]

Also sprach Jeff:

The function call is executing in the new perl context. I can see this
in the debugger, and if it wasn't executing in the newly cloned
context, it would work.

I believe the problem is that the function reference I'm calling is
from the parent interpreter, and its CV value is pointing to the parent
interpreter, not the cloned interpreter.

That sounds plausible.

Any ideas on how to fix this?

As always in such cases, it's now probably necessary to try some of the
non-public stuff found in the source that has a promising name.

You might want to try it with applying sv_dup() to the callback SV. Its
prototype:

SV * Perl_sv_dup(pTHX_ SV *sstr, CLONE_PARAMS* param);

'CLONE_PARAMS' is:

struct clone_params {
AV* stashes;
UV flags;
PerlInterpreter *proto_perl;
};

'flags' seem to be those of perl_clone(). Looking at the source of
sv_dup(), 'stashes' is probably irrelevant in your case. 'proto_perl'
doesn't seem to be used for anything either but it can't hurt to store
perl_clone()'s return value there.

Other than that, skimming through ext/threads/threads.xs could be a good
idea (Perl_ithread_create() in particular).

Tassilo

 

[Jeff]

Good call on sv_dup, that fixed the problem. I'm still going to take a
look through Perl_ithread_create though to see how perl handles this
situation.

Thanks for your help,
Jeff