Need advice on debugging malloc bug (XML::Parser::Expat)

[kj]

I have run out of things to try to fix a particular bug, and I'm
hoping some of you may be able to give me some ideas.

The bug happens only when the program is run inside the Perl debugger
(which I do often; the Perl debugger is an integral part of my
development environment).

To reproduce the bug, the program must first invoke
XML:arser::Expat:arseString to parse some XML. Once the parsing
finishes, if I attempt to match a capture-containing regexp to a
string obtained from the match, one of a few bad things happen:
either the capture (e.g. $1) will contain garbage; or the program
will die with a "panic: malloc" error; or the program will hang;
or the program will die with an "Out of memory!" error.

The following snippet illustrates the sequence I just described
(but before you "try this at home", read below):

use strict;
use warnings;
use XML:arser::Expat;

my $xml = '<?xml version="1.0"?><Foo></Foo>';
my $content = '';
my $parser = XML:arser::Expat->new();

$parser->setHandlers(Start => sub { $content .= ">@_<" });

XML:arser::Expat:arseString($parser->{Parser}, $xml);

print "$content\n"; # shows that the parsing happend
$content =~ /([a-z]+)/; # BOOM!
print "OK\n";
__END__

% perl -d expat_bug.pl
....

XML:arser::Expat=HASH(0x8342150) Foo<

panic: malloc at expat_bug.pl line 15.
at expat_bug.pl line 15


In this case, the outcome was the "panic: malloc" error, but this
is not always the case. Innocuous-seeming changes in the source
code can lead to one of the other outcomes I listed.

Unfortunately, I have not been able to reproduce this bug anywhere
outside our system (Linux SuSE). (In other words, if I don't fix
this problem, probably no one will.)

I have re-built and re-installed perl, XML:arser, and libexpat,
and still the problem persists.

I looks like the root of the bug is that some part of memory is
being overwritten by mistake (a rather typical occurrence with C
programs).

What tool would you recommend to catch something like this?

Any other suggestions would be much appreciated.

kj

PS For the sake of completeness, I give details of my setup below.
Also note that the details below are only one of the several setups
I've tried. (E.g. I've compiled and tried different versions of
perl, with different options, etc.)

% uname -ar
Linux luna 2.6.11.4-21.10-smp #1 SMP Tue Nov 29 14:32:49 UTC 2005 i686 i686 i386 GNU/Linux
% perl -V

Summary of my perl5 (revision 5 version 8 subversion 6) configuration:
Platform:
osname=linux, osvers=2.6.9, archname=i586-linux-thread-multi
uname='linux g226 2.6.9 #1 smp tue jun 28 14:58:56 utc 2005 i686 i686 i386 gnulinux '
config_args='-ds -e -Dprefix=/usr -Dvendorprefix=/usr -Dinstallusrbinperl -Dusethreads -Di_db -Di_dbm -Di_ndbm -Di_gdbm -Duseshrplib=true -Doptimize=-O2 -march=i586 -mcpu=i686 -fmessage-length=0 -Wall -g -Wall -pipe'
hint=recommended, useposix=true, d_sigaction=define
usethreads=define use5005threads=undef useithreads=define usemultiplicity=define
useperlio=define d_sfio=undef uselargefiles=define usesocks=undef
use64bitint=undef use64bitall=undef uselongdouble=undef
usemymalloc=n, bincompat5005=undef
Compiler:
cc='cc', ccflags ='-D_REENTRANT -D_GNU_SOURCE -DTHREADS_HAVE_PIDS -DDEBUGGING -fno-strict-aliasing -pipe -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64',
optimize='-O2 -march=i586 -mcpu=i686 -fmessage-length=0 -Wall -g -Wall -pipe',
cppflags='-D_REENTRANT -D_GNU_SOURCE -DTHREADS_HAVE_PIDS -DDEBUGGING -fno-strict-aliasing -pipe'
ccversion='', gccversion='3.3.5 20050117 (prerelease) (SUSE Linux)', gccosandvers=''
intsize=4, longsize=4, ptrsize=4, doublesize=8, byteorder=1234
d_longlong=define, longlongsize=8, d_longdbl=define, longdblsize=12
ivtype='long', ivsize=4, nvtype='double', nvsize=8, Off_t='off_t', lseeksize=8
alignbytes=4, prototype=define
Linker and Libraries:
ld='cc', ldflags =''
libpth=/lib /usr/lib /usr/local/lib
libs=-lnsl -ldl -lm -lcrypt -lutil -lpthread -lc
perllibs=-lnsl -ldl -lm -lcrypt -lutil -lpthread -lc
libc=, so=so, useshrplib=true, libperl=libperl.so
gnulibc_version='2.3.4'
Dynamic Linking:
dlsrc=dl_dlopen.xs, dlext=so, d_dlsymun=undef, ccdlflags='-Wl,-E -Wl,-rpath,/usr/lib/perl5/5.8.6/i586-linux-thread-multi/CORE'
cccdlflags='-fPIC', lddlflags='-shared'

Characteristics of this binary (from libperl):
Compile-time options: DEBUGGING MULTIPLICITY USE_ITHREADS USE_LARGE_FILES PERL_IMPLICIT_CONTEXT
Locally applied patches:
SPRINTF0 - fixes for sprintf formatting issues - CVE-2005-3962
Built under linux
Compiled at Dec 17 2005 03:23:29
%ENV:
PERL5LIB="/home/jones/local/lib/perl5"
PERL5_CPANPLUS_CONFIG="/home/jones/.cpanplus/config"
PERL_RL="Perl"
@INC:
/home/jones/local/lib/perl5/5.8.6/i586-linux-thread-multi
/home/jones/local/lib/perl5/5.8.6
/home/jones/local/lib/perl5/i586-linux-thread-multi
/home/jones/local/lib/perl5
/usr/lib/perl5/5.8.6/i586-linux-thread-multi
/usr/lib/perl5/5.8.6
/usr/lib/perl5/site_perl/5.8.6/i586-linux-thread-multi
/usr/lib/perl5/site_perl/5.8.6
/usr/lib/perl5/site_perl
/usr/lib/perl5/vendor_perl/5.8.6/i586-linux-thread-multi
/usr/lib/perl5/vendor_perl/5.8.6
/usr/lib/perl5/vendor_perl

 

[John Bokma]

The bug happens only when the program is run inside the Perl debugger
(which I do often; the Perl debugger is an integral part of my
development environment).

I hardly use it (once in a year is enough )

 

[Andreas Puerzer]

kj schrieb:

[bug description kept for completeness]

I have run out of things to try to fix a particular bug, and I'm
hoping some of you may be able to give me some ideas.

The bug happens only when the program is run inside the Perl debugger
(which I do often; the Perl debugger is an integral part of my
development environment).

To reproduce the bug, the program must first invoke
XML:arser::Expat:arseString to parse some XML. Once the parsing
finishes, if I attempt to match a capture-containing regexp to a
string obtained from the match, one of a few bad things happen:
either the capture (e.g. $1) will contain garbage; or the program
will die with a "panic: malloc" error; or the program will hang;
or the program will die with an "Out of memory!" error.

The following snippet illustrates the sequence I just described
(but before you "try this at home", read below):

use strict;
use warnings;
use XML:arser::Expat;

my $xml = '<?xml version="1.0"?><Foo></Foo>';
my $content = '';
my $parser = XML:arser::Expat->new();

$parser->setHandlers(Start => sub { $content .= ">@_<" });

XML:arser::Expat:arseString($parser->{Parser}, $xml);

print "$content\n"; # shows that the parsing happend
$content =~ /([a-z]+)/; # BOOM!
print "OK\n";
__END__

% perl -d expat_bug.pl
....
main:expat_bug.pl:6): my $xml = '<?xml version="1.0"?><Foo></Foo>';

XML:arser::Expat=HASH(0x8342150) Foo<

panic: malloc at expat_bug.pl line 15.
at expat_bug.pl line 15

Unfortunately, I have not been able to reproduce this bug anywhere
outside our system (Linux SuSE). (In other words, if I don't fix
this problem, probably no one will.)

Bug reproduced here on WindowsXP, AS Perl 5.8.4, Expat 2.34;
I could not reproduce it with Cygwin Perl 5.8.7, Expat 2.34!

Strangely, I don't get any of the different panic messages you describe, it just
segfaults silently every time I use the debugger on it. I still don't quite
understand why it segfaults with the AS Perl, but not with the Cygwin Perl. (Ok,
to be honest, I don't understand why it segfaults at all... ;->)

I have re-built and re-installed perl, XML:arser, and libexpat,
and still the problem persists.

I looks like the root of the bug is that some part of memory is
being overwritten by mistake (a rather typical occurrence with C
programs).

Just being curious, but how do you come to this conclusion?

What tool would you recommend to catch something like this?

Normally I would recommend to use the debugger..., which unfortunately doesn't
seem to be helpful here, right?!

Any other suggestions would be much appreciated.

Some Ideas:

The setup you showed used 5.8.6, my Cygwin, which seems unaffected, is 5.8.7,
maybe it's a known &, in newer versions at least, resolved issue? Have you
searched the bug-database? ( or already tried it with 5.8.7 or 5.8.8 to no avail?)

Another thing I noticed:

/([a-z]+)/ # original RE, segfaults
/[a-z]+/ # works fine!
/([\w]+)/ # segfaults
/(\w+)/ # works fine!

To me it seems like the Captures are only half the problem, the bug only shows
when there are also character classes used.

kj

Andreas Pürzer