Optionally avoid AutoLoader

[Marc Girod]

Hello,

I contribute to a family of CPAN packages which heavily use AutoSplit/
AutoLoader (ClearCase::Wrapper, ClearCase::Wrapper::MGi --
unfortunately, they interface a proprietary tool: IBM/Rational
ClearCase, so that you cannot really test the packages without a
ClearCase installation).

The autoload functionality is good and useful. I don't want to modify
it.
However, for development and debugging purposes, it would be useful to
temporarily override the autoloader.
Now, the modules are actually used through a command line driver (the
'cleartool wrapper'), which is part of ClearCase::Wrapper.
This package is rather stable, so the the one I want to develop and
debug is rather the (an)other one.
I could consider having an alternative driver, and installing it
beside the normal one.
What I don't really want is to have to modify part of the
ClearCase::Wrapper package while editing ClearCase::Wrapper::MGi.

My idea for the 'wrapdebug' driver would be to define there an
AUTOLOAD routine, and to inject its name into the space of
ClearCase::Wrapper (preempting AutoLoader::AUTOLOAD, which is 'used'
explicitely, in the standard way).

The problem I hit right away is that then, I'd need to access code
beyond the __END__ tag in the modules...

Is there a way to open that door? Did I miss an obvious corridor
before reaching this door?

Thanks,
Marc

 

[Marc Girod]

What does this new AUTOLOAD do? Can you not just call
AutoLoader::AUTOLOAD when you want to define the actual sub?

It would (I didn't write it yet: decided to check first...) do what
you described below: ignore the *.al files, and load instead the pm
files (not only ClearCase/Wrapper.pm, but also ClearCase/Wrapper/
MGi.pm), by-passing the __END__ tags.

It has to compute the paths itself (doesn't it?) because it will be
called before the $INC hash has been updated...

Well, AutoSplit has handily split that code up into *.al files for you,
which can simply be required as you need them. You could also open the
file in $INC{"ClearCase/Wrapper.pm"} and search for __END__ yourself,
then string-eval the rest of the file. Unfortnately once perl sees
__END__ in a .pm file, it closes the filehandle, so you can't avoid
opening a new file.

There are (at least) two problems with the autoload mechanism in
development mode:
- use strict and use warnings are ignored
- debugging is hard: one must navigate through the code to the
invocations, and step through the autoloader code: it is impossible to
put breakpoints because the code is first not loaded, and then not
loaded into the same (file) space (line numbers are corrected to match
the original source, but only valid in the *.al files).

Will string-eval produce strict/warning reports?

(If you look in %INC, be aware that coderef hooks in @INC produce paths
like "/loader/0xdeadbeef/...", which obviously aren't valid filenames.)

I guess I have to try before I understand this...
Thanks,
Marc

 

[Marc Girod]

Well, AutoSplit has handily split that code up into *.al files for you,
which can simply be required as you need them.

Actually, I first explored this idea, and it seems to works...
I.e. I looked at all the registered packages, and required all the
*.al files.
This relies on the fact that the installation directory is clean from
removed functions.

my (%pkg, %seen); #seen functions were overridden: skip the next
occurrences
$pkg{$_}++ for values %ClearCase::Wrapper::ExtMap;
for (reverse sort keys %pkg) {
s%::%/%g;
my $as = catfile($_, 'autosplit.ix');
my ($fas) = grep /\Q$as\E$/, keys %INC;
die "Could not find $as in \%INC" unless $fas;
my $dir = dirname($fas);
opendir DIR, $dir or die "Could not open $dir";
for (grep /\.al$/, readdir DIR) {
next if $seen{$_}++;
my $f = catfile($dir, $_);
require $f;
}
closedir DIR;
}

I'll have to practice a bit to see if it buys me the expected
advantages...
Looking at the breakpoint:

main:./wrapdebug:125): my $status;
DB<1> b 2882
Line 2882 not breakable.
DB<2> c 142
main:./wrapdebug:142): if (scalar @ARGV == 1 && $ARGV[0] eq '-
status') {
DB<3> b 2882
Line 2882 not breakable.
DB<4> x grep m%ClearCase/Wrapper/MGi/describe.al$%, keys %INC
0 '/home/emagiro/perl/lib/auto/ClearCase/Wrapper/MGi/describe.al'
DB<5> f describe.al
Choosing /home/emagiro/perl/lib/auto/ClearCase/Wrapper/MGi/describe.al
matching `describe.al':
1 # NOTE: Derived from blib/lib/ClearCase/Wrapper/MGi.pm.
2 # Changes made here will be lost when autosplit is run again.
3 # See AutoSplit.pm.
4 package ClearCase::Wrapper::MGi;
5
6 #line 2871 "blib/lib/ClearCase/Wrapper/MGi.pm (autosplit into blib/
lib/auto/ClearCase/Wrapper/MGi/describe.al)"
DB<6> b 2882
Line 2882 not breakable.
DB<7> v 2880

Argh... Now, the line numbers are *not* autocorrected...
Maybe only a minor nuisance...

Thanks,
Marc

 

[Marc Girod]

Maybe only a minor nuisance...

Unfortunately not...
The debugger really doesn't see beyond:

DB<10> v
3 # See AutoSplit.pm.
4 package ClearCase::Wrapper::MGi;
5
6 #line 2871 "blib/lib/ClearCase/Wrapper/MGi.pm (autosplit into blib/
lib/auto/ClearCase/Wrapper/MGi/describe.al)"
DB<10> b 18
Line 18 not breakable.

Marc

 

[Marc Girod]

Hi,

I acknowledge your reply to a previous message of mine.

Unfortunately not...
The debugger really doesn't see beyond: ....
Line 18 not breakable.

Maybe a judicious 'use #line' is what I miss here...
I thought of a problem with this approach: the strict and warnings
reports are issued while compiling, and the compilation took place
with autosplitting.
Hence, I'll have already missed them...
It is indeed autosplit, not autoload which I ought to by-pass (or to
alter)!

Marc

 

[Marc Girod]

Not 'use #line', just a line like

Sorry but then... it is already there: AutoSplit put it!
I have in describe.al (line 6):
#line 2871 "blib/lib/ClearCase/Wrapper/MGi.pm (autosplit into blib/lib/
auto/ClearCase/Wrapper/MGi/describe.al)"

Now, this seems (in this context) to create a problem for the debugger
(perl -d).
- I cannot put a breakpoint beyond this point, using either the
original or the corrected line number
- The 'v' command refuses to show any code
- However, 'c' happily continues...

No it didn't. AutoSplit treats the code to be split purely as text, it
doesn't compile it.

OK. Only, in the normal development lifecycle, which I'd wish to
disturb as little as possible for backwards compatibility reasons,
autosplit is invoked as part of the 'make' --a bit in an obscure/
implicit way: I can only find in the generated Makefile:

# --- MakeMaker tool_autosplit section:
# Usage: $(AUTOSPLITFILE) FileToSplit AutoDirToSplitInto
AUTOSPLITFILE = $(ABSPERLRUN) -e 'use AutoSplit; autosplit($
$ARGV[0], $$ARGV[1], 0, 1, 1)' --

....which matches the AutoSplit documentation:

Typical use of AutoSplit in the perl MakeMaker utility is via
the command-
line with:

perl -e 'use AutoSplit; autosplit($ARGV[0], $ARGV[1], 0, 1,
1)'

Will the splitting take place if the build fails?
As it would because of a strict/warnings report...
I cannot understand why/how these messages are lost (in my
experience...).
Looking at the AutoLoader::AUTOLOAD code, it seems to croak such
messages:

if ($@){
$@ =~ s/ at .*\n//;
my $error = $@;
require Carp;
Carp::croak($error);
}

In any case, if you pass the code through 'eval' or
'require' it will be compiled again, regardless of what has happened to
it in the past.

OK. This is what I do now, in my tentative code (following your
suggestion).

Thanks a lot for your continued attention!
Marc

 

[Marc Girod]

Now, this seems (in this context) to create a problem for the debugger
(perl -d).

If I do, at the debugger prompt, and after the point where I have
loaded all the .al files:

DB<8> f MGi.pm
Choosing blib/lib/ClearCase/Wrapper/MGi.pm (autosplit into blib/lib/
auto/ClearCase/Wrapper/MGi/_Checkcs.al) matching `MGi.pm':
1 2 3 4 5 6 7 8 9 10

I don't quite understand this output.
However, what actually got loaded is the first .al file (in the
alphabetic order): _Checkcs.al, and there:

DB<12> v 191
188 189 190 191 sub _Checkcs {
192: use File::Basename;
193: use Cwd;
194: my ($v) = @_;
195: $v =~ s/^(.*?)\@\@.*$/$1/;
196: my $dest = dirname($v);
197: $dest .= '/' unless $dest =~ m%/$%;

This 191 comes from the cpp #line directive in the file:

#line 191 "blib/lib/ClearCase/Wrapper/MGi.pm (autosplit into blib/lib/
auto/ClearCase/Wrapper/MGi/_Checkcs.al)"

The first output showed line numbers, which until line 191, are
'empty'...?
Is it possible to load the other fragments?

Marc

 

[Marc Girod]

I don't use the debugger.

Well, I do. In fact, the debugger plays an important role in my using
Perl.
Anyway, I have to support users of the modules.

What happened when you tried it?

I can load the .al files by name, but then, I cannot even read the
code (hence not put breakpoints).
I couldn't find a syntax to load any other file but the first
(_Checkcs.al), which I load as 'MGi.pm'.

Marc

 

[Marc Girod]

Did you try 'f _Checkcs.al' and so on, as I suggested? It seems to work
OK to me:

Yes I did, and it didn't work for me...

Note that the very first thing I did was *call* POSIX::chdir, so the .al
could be autoloaded.

Well, I didn't do that, but I ran my code which requires all the
files:

ClearCase-Wrapper> perl -d wrapdebug -noautoload des -s lbtype:MG

Loading DB routines from perl5db.pl version 1.33
Editor support available.

Enter h or `h h' for help, or `man perldebug' for more help.

....
DB<1> c 145
main:wrapdebug:145): my $status;
DB<2> f _Checkcs.al
Choosing blib/lib/ClearCase/Wrapper/MGi.pm (autosplit into blib/lib/
auto/ClearCase/Wrapper/MGi/_Checkcs.al) matching `_Checkcs.al':
1 2 3 4 5 6 7 8 9 10

At that point, I don't see the code.
Now indeed, if I exert it (not that exact piece but say the one I ran
on the debug line):

DB<8> c 73
main:ne_cmd(wrapdebug:73): my $rc = eval { $cmd->(@ARGV) };
DB<9> s
main:ne_cmd(wrapdebug:73): my $rc = eval { $cmd->(@ARGV) };
DB<9>
ClearCase::Wrapper::MGi::describe(blib/lib/ClearCase/Wrapper/MGi.pm
(autosplit into blib/lib/auto/ClearCase/Wrapper/MGi/describe.al):
2874):
2874: my $desc = ClearCase::Argv->new(@ARGV);
DB<9> v
2871 sub describe {
2872: use strict;
2873: use warnings;
2874==> my $desc = ClearCase::Argv->new(@ARGV);
2875: $desc->optset(qw(CC WRAPPER));
2876: $desc->parseCC(qw(g|graphical local l|long s|short
2877 fmt=s alabel=s aattr=s ahlink=s ihlink=s
2878 cview version=s ancestor
2879 predecessor pname type=s cact));
2880: $desc->parseWRAPPER(qw(parents|par9999=i family=i));
DB<9> b 2874

.... I do get there, and it does work.
I believe that this is equivalent to your running CORE::chdir.
The problem is that this is a chicken and egg issue: if I know to
navigate to the code, I don't need the breakpoint...

It seems to me that the whole idea of breakpoints is to be able to
place them *before* I know whether I'll get there...
I already new to navigate to the code through the autoloader...

I have to think of it...
Is there any benefit at that stage?

Thanks again,
marc

 

[Marc Girod]

You do, though . The debugger is showing you the first ten lines of
that logical file, which are all empty. The #line declaration means the
actual code starts much further down.

Yes, I know, but the debugger doesn't show it, unless I *navigate* to
it (as I explained).

Try this:

    - open ClearCase/Wrapper/MGi.pm in your editor,
    - find the 'describe' function in the __END__ section,

Yes, it is line 2871, as in the #line directive.
Or if you prefer, 2874, the first line of runnable code (an
assignment).

    - note the line number where it starts,
    - run the debugger, and type
        f describe.al
        l <line number in MGi.pm where &describe starts>
        l

DB<4> l 2871

DB<5> l 2874

If you've loaded all the .als, I would expect you could also just use

    l ClearCase::Wrapper::MGi::describe

which should take you to the right file and line directly.

Interesting: *that* works...

DB<6> l ClearCase::Wrapper::MGi::describe
Switching to file 'blib/lib/ClearCase/Wrapper/MGi.pm (autosplit into
blib/lib/auto/ClearCase/Wrapper/MGi/describe.al)'.
2871 sub describe {
2872: use strict;
2873: use warnings;
2874: my $desc = ClearCase::Argv->new(@ARGV);
2875: $desc->optset(qw(CC WRAPPER));
2876: $desc->parseCC(qw(g|graphical local l|long s|short
2877 fmt=s alabel=s aattr=s ahlink=s ihlink=s
2878 cview version=s ancestor
2879 predecessor pname type=s cact));
2880: $desc->parseWRAPPER(qw(parents|par9999=i family=i));

Any benefit in what? The AutoLoader? I would say not. If you want to do
delay-loading, there are simpler and less magical ways of doing it.

No, I meant a benefit of preloading all the .al files to restore
debuggability.
With the above note, there is one, indeed.
So far, I couldn't see any.

The point is you typically know where you want to put a breakpoint but
have no clue how you exactly reach this point. That's what you hope
the debugger will tell!

Thanks!
For somebody who doesn't use the debugger, you manage to help me
who use it every day... I feel humble.
Marc

 

[Marc Girod]

Given the 'Switching to...' below, I deduce that you were in the wrong
file. Perhaps 'f describe.al' picked up ClearCase/Wrapper/describe.al
instead of .../MGi/describe.al, or something?

It doesn't seem so:

DB<3> f describe.al
Choosing /home/emagiro/perl/lib/auto/ClearCase/Wrapper/MGi/describe.al
matching `describe.al':
1 # NOTE: Derived from blib/lib/ClearCase/Wrapper/MGi.pm.
2 # Changes made here will be lost when autosplit is run again.
3 # See AutoSplit.pm.
4 package ClearCase::Wrapper::MGi;
5
6 #line 2871 "blib/lib/ClearCase/Wrapper/MGi.pm (autosplit into blib/
lib/auto/ClearCase/Wrapper/MGi/describe.al)"

Referencing the symbol seems to tie things up in a way that wasn't
done before...

Marc

 

[Marc Girod]

'f' takes a regex, so try 'f autosplit.*describe.al' followed by 'l
2871'.

Bingo. Now everything matches.

Thanks!
Marc