Question about language setting

[Rainer Weikusat]

[...]

require XSLoader;

#line 1 /opt/perl/lib/5.16.3/amd64-freebsd/POSIX.pm
XSLoader::load("POSIX");
POSIX::setlocale("LC_ALL", "C");
delete $POSIX::{$_} for keys %POSIX::;

The #line line is necessary to prevent XSLoader from falling back to
DynaLoader (which loads vars.pm, which has a 'use VERSION' line). It
must give the correct path to POSIX.pm on your system, with /
separators. Clearing out POSIX:: is necessary to avoid a whole lot of
'subroutine redefined' warnings when you load POSIX properly.

The #line hack could be avoided by using DynaLoader::dl_load_file and
dl_find_symbol directly, but that still needs to know the full path to
POSIX.dll (and you can't load Config to find it properly). I was hoping
it would be possible to avoid needing to clear POSIX:: by just looking
up 'setlocale' and then unloading the DLL, but these days XSUBs are
static functions and the only external symbol in the DLL is boot_POSIX.

Hi Ben

Argument "LC_ALL" isn't numeric in subroutine entry at
d:/usr/lib/perl/lib/5.16.
0/OS2/POSIX.pm line 2.

I could of course hard code the value from the .h file

That's what you have to do (and use the number instead of a string)
because there's no way to get access to the symbolic names without
loading the Perl part. However, what I was thinking was actually rather
something like this:

-----------
/* includes */
#include "EXTERN.h"
#include "perl.h"
#include "XSUB.h"

#include <locale.h>

MODULE = set_lc_all

char *set_lc_all(what)
char *what
CODE:
RETVAL = setlocale(LC_ALL, what);
OUTPUT:
RETVAL
----------

This is code written in the Perl eXtenSion language. It can be turned
into C with xsubpp, compiled into a shared object/ DLL and the
individual funtion can then be made available via DynaLoader. This
requires a certain one-time effort (I don't think I can legally post the
code I'm using but all in all, it is a lot less than 100 LOC) but I
found it more than occasionally useful that I can combine Perl and C
freely.

 

[Rainer Weikusat]

[...]

"Perfection is the enemy of the good": I'm fine with using XS and all I
really want is 'implement a function (or some functions)' in XS/C and
link that together with an existing Perl

[...]

Yes, if you're writing your own XS (or XS-compatible C) it's not a
problem. However, turning that XS file into a 'proper' extension just
means adding a 3-line .pm and a 4-line Build.PL, so I'd generally do
that. (I never use h2xs.) That way it behaves and installs like a Perl
module, so I don't have to worry about installing custom .sos when I
copy stuff onto a production machine.

I'm usually building (pretty bare bones) Debian packages, so that's not
an issue. ATM, there are 51 one them, with the 52nd in development. The
initial attempt to manage all of this manually (on about a dozen
different installations spread all around the globe) broke down a few
years ago .

What *would* be really useful, though, is a proper FFI interface that
allowed you to dlopen an arbitrary .so and call functions in it. That's
what you were talking about before: if you tried to open libc.so with
dl_open_file and find setlocale with dl_find_symbol it would segfault
when you tried to call it, because setlocale isn't void setlocale(pTHX_
CV *).

This had helped here but I'm actually really writing C code and linking
that with Perl, the code I posted upthread was just the simplest example
I'm presently using (there's no way to get access to __builtin_clz by
using a library as this is really a 'built-in' gcc function which gets
replaced by the corresponding machine instruction for the target
architecture).

There is

http://search.cpan.org/~gaal/FFI-1.02/FFI.pm

(which I found by 'creatively' typing 'Perl FFI' in a Google search box)
but I'm unsure if I'd want to use that (would need to have a look at the
internals), considering that XS works nicely enough.

 

[Rainer Weikusat]

Quoth "Dave Saville" <[email protected]>:
[...]

require XSLoader;

my $dll = "$ENV{PERLLIB_PREFIX}/5.16.0/OS2/auto/POSIX/POSIX.dll";
my $lib = DynaLoader::dl_load_file($dll, 0);
push(@DynaLoader::dl_librefs, $lib);

my $boot = DynaLoader::dl_find_symbol($lib, "boot_POSIX");
push(@DynaLoader::dl_modules, "POSIX");

my $xs = DynaLoader::dl_install_xsub(
"POSIX::bootstrap", $boot, $dll);
push(@DynaLoader::dl_shared_objects, $dll);

$xs->();
{ no warnings "redefine";
*POSIX::bootstrap = sub { };
}

Considering that you don't do anything with XSLoader, should this
perhaps have been

require Dynaloader;

?

Also, the 5.10.0 POSIX.whatever should have XS__setlocale, considering
that the 5.10.1 one has it.

 

[Rainer Weikusat]

Quoth "Dave Saville" <[email protected]>:
[...]

require XSLoader;

my $dll = "$ENV{PERLLIB_PREFIX}/5.16.0/OS2/auto/POSIX/POSIX.dll";
my $lib = DynaLoader::dl_load_file($dll, 0);
push(@DynaLoader::dl_librefs, $lib);

[...]

Considering that you don't do anything with XSLoader, should this
perhaps have been

require Dynaloader;

?

Also, the 5.10.0 POSIX.whatever should have XS__setlocale, considering
that the 5.10.1 one has it.

The correct name is XS_POSIX_setlocale. For the sake of completeness,
here's an executable example (Tested on Linux. Will need to be changed
on platforms which don't use .so as extension for 'dynamically loadable
library files) showing how to import just the setlocale routine into the
current package without running any of the 'module init code',

---------------
BEGIN {
my ($path, $lib, $sym);

require DynaLoader;

# add the perl include path to the DynaLoader library path
# so that dl_findfile will find POSIX.so
#
local @DynaLoader::dl_library_path = (@DynaLoader::dl_library_path, @INC);
$path = DynaLoader::dl_findfile('auto/POSIX/POSIX.so');
unless ($path) {
print STDERR ("didn't find POSIX.so\n");
return;
}

# load the library
#
$lib = DynaLoader::dl_load_file($path, 0);
unless ($lib) {
print STDERR ("failed to load $path\n");
return;
}

# locate the setlocale interface routine
#
$sym = DynaLoader::dl_find_symbol($lib, 'XS_POSIX_setlocale');
unless ($sym) {
print STDERR ("didn't find XS_POSIX_setlocale\n");
return;
}

# install it into the current package
#
DynaLoader::dl_install_xsub('setlocale', $sym);

# print a 'German floating point number'
#
setlocale(1, 'de_DE'); # LC_ALL == 1
printf("%f\n", 2.5);
}
---------------

It should also be noted that locally extending an existing array with
some values couldn't work in the way it is done here without being able
to access the previous incarnation of some symbol while creating a new
one.

 

[Rainer Weikusat]

[...]

Dave is using 5.16, by which point XSUBs had changed to being static
symbols by default, with only the boot_Foo symbol externally visible.

Is there some rationale for that beyond "it breaks existing code"?

 

[Rainer Weikusat]

Perhaps you would like to ask that question again, leaving out the
implicit assumption that p5p are idiots?

A comment regarding that is available below the page break as this
question is really besides the point. I assume the XSUB change had some
technical objective, ie, someone thought something could be gained in
this way. But I can't imagine what (wich may be entirely my fault). The
wording in the corresponding changes document,

The EXPORT_XSUB_SYMBOLS: keyword is likely something you will never
need.
[http://perldoc.perl.org/5.16.1/perlxs.html]

suggests that whoever wrote that didn't believe anyone actually uses
DynaLoader for anything, yet I do. Hence, the immediate effect is that I
can't use 5.16 and onwards (not exactly an immediate danger) without
either changing all the affected code or (more likely) restoring the old
behaviour.

An immediate (although weak) argument for the old behaviour which comes
to mind would be: If the XSUBs are externally visible symbols, they can
be installed on demand. Especially for large modules providing access to
'diverse facilities', eg, POSIX, this should reduce the startup time of
code using the module, may reduce the total time needed to use some
parts of the module for a given situation (if only a few symbols are
ever used) and should use somewhat less memory.

Considering that I know that not every report made to the Perl
foundation in the course of some development grant is technically
accurate -- whether this happened because the guy who wrote them didn't
understand the matter himself or banked on the fact that the people
getting the reports won't understand it I don't know -- 'malice [aka
"some political agenda"] or stupidity' might be an interesting question
on its own right, but a rather tangential one: People who subscribe to
p5p are people, hence, they're bound to make mistakes and prone to
inventing technical justifications which don't really hold water in order
to push this or that pet agenda of them, just like all other people.

But beyond this general statement ('even the wisest man is no God'),
speculations about other people's hidden motives are (IMHO) unpleasant
and useless.

 

[Rainer Weikusat]

Perhaps you would like to ask that question again, leaving out the
implicit assumption that p5p are idiots?

As far as I know, the term 'idiot' ('idiotus', actually) was invented by
Cusanus to describe people he referred to as 'eigensinnig Wissende' in
German, whose main trait was supposed to be that they won't - under any
circumstances - accept anything from anyone else and who are thus bound
to progress slowly and circuitiously towards understanding the world but
who might eventually gain such understanding nevertheless.

 

[Dave Saville]

Quoth "Dave Saville" <[email protected]>:
[...]

require XSLoader;

my $dll = "$ENV{PERLLIB_PREFIX}/5.16.0/OS2/auto/POSIX/POSIX.dll";
my $lib = DynaLoader::dl_load_file($dll, 0);
push(@DynaLoader::dl_librefs, $lib);

my $boot = DynaLoader::dl_find_symbol($lib, "boot_POSIX");
push(@DynaLoader::dl_modules, "POSIX");

my $xs = DynaLoader::dl_install_xsub(
"POSIX::bootstrap", $boot, $dll);
push(@DynaLoader::dl_shared_objects, $dll);

$xs->();
{ no warnings "redefine";
*POSIX::bootstrap = sub { };
}

Considering that you don't do anything with XSLoader, should this
perhaps have been

require Dynaloader;

?

No. DynaLoader.pm loads Config.pm which loads vars.pm which tickles the
bug we're trying to avoid. XSLoader.pm bootstraps the XS part of
DynaLoader without loading the .pm, since the whole point of that module
is to dynaload without loading the relatively heavy Perl part of
DynaLoader.

Of course, you could always just run

DynaLoader::boot_DynaLoader("DynaLoader")
if defined &DynaLoader::boot_DynaLoader
&& !defined &DynaLoader::dl_error;

instead, since the whole point is that DynaLoader's bootstrap is
statically linked, so we don't need to load anything to make it
available.

Also, the 5.10.0 POSIX.whatever should have XS__setlocale, considering
that the 5.10.1 one has it.

Dave is using 5.16, by which point XSUBs had changed to being static

Yes *I* am but other users will almost certainly be using earlier
versions. I suppose one could switch on the perl version.

I appreciate all the help guys but I really think the simple answer is
to check the perl version for before 5.16.3 and for 2.5 ne "2.5" and
put out a message telling them to change the locale before running the
script and quit.

 

[Rainer Weikusat]

Perhaps you would like to ask that question again, leaving out the
implicit assumption that p5p are idiots?

A comment regarding that is available below the page break as this
question is really besides the point. I assume the XSUB change had some
technical objective, ie, someone thought something could be gained in
this way. But I can't imagine what (wich may be entirely my fault). The
wording in the corresponding changes document,

The EXPORT_XSUB_SYMBOLS: keyword is likely something you will never
need.
[http://perldoc.perl.org/5.16.1/perlxs.html]

suggests that whoever wrote that didn't believe anyone actually uses
DynaLoader for anything, yet I do.

I've located the corresponding p5p thread,

http://www.nntp.perl.org/group/perl.perl5.porters/2011/08/msg175781.html

but I'm still at a loss regarding what this is supposed to accomplish. A
couple of quotes from the thread and the original commit:

https://gitorious.org/chip-perl5/chip-perl5/commit/e64345f82d66a32f6da47acf482e7e6c9282b433

This is [...] generally preferable

the symbols are already all internal by default on Windows (and AIX)

I doubt that many (any?) existing modules intentionally export
XS symbols [...] It is much easier to call plain C code from a different module
than XS code, which really should only be called through Perl.

Exporting the C symbol corresponding to an XS sub body is a pretty bizarre
requirement

'perl -MGtk2 -eexit' is now twice as fast as before: 0.036s with 5.14.1,
0.018s

"A little less than 0.02s speedup for a program which loads a seriously
large module and does nothing" doesn't seem to be much of an improvement
to me, especially considering that Gtk2 would likely be used for an
interactive application and '0.02s during startup' is irrelevant noise
for that. The idea that this "bizarre requirement" enables on demand
creation of of XSUBs instead of creating them all at once (a benchmark
on how omitting boot_ from the Gtk2 load sequence in favour of this
affects execution time would be interesting) doesn't seem to have
crossed anyone's mind and the "It is much easier to call plain C code
[...]" strongly suggests that the guy who wrote this doesn't even know
that Perl provides an API for accessing invividual XSUBs.