How to avoid Out of Memory Errors when dealing with a large XML file?

[Saqib Ali]

Hi.

I'm reading a large (57 MB) XML file Using XML::XPath::XMLParser()

I keep getting this error:

"Callback called exit at XML/XPath/Node/Element.pm at line 144 during
global destruction."

I'm using Windows XP. So I watched the task-management memory meter
during the execution of this process. The PERL process chewed up a lot
of the "available memory". But when the process died, it still showed
about 216MB available memory.

Is there anything I can do to work-around this problem? From reading
responses to other similar questions, the only option may be to use a
XML stream parser instead of one that builds the entire DOM tree
internally.

 

[Chris Nehren]

Hi.

I'm reading a large (57 MB) XML file Using XML::XPath::XMLParser()

I keep getting this error:

"Callback called exit at XML/XPath/Node/Element.pm at line 144 during
global destruction."

I'm using Windows XP. So I watched the task-management memory meter
during the execution of this process. The PERL process chewed up a lot
of the "available memory". But when the process died, it still showed
about 216MB available memory.

Firstly, as someone or something has inflicted Windows on you, are you
using Strawberry Perl like every sane Windows user (bit of an oxymoron,
that)? Second, calling it "PERL" makes you look either ignorant or
trollish, so don't do that.

Is there anything I can do to work-around this problem? From reading
responses to other similar questions, the only option may be to use a
XML stream parser instead of one that builds the entire DOM tree
internally.

Yes, you'll want a SAX parser. Strawberry Perl ships with XML::LibXML,
which provides a SAX interface via XML::LibXML::SAX.

 

[Saqib Ali]

FYI, someone posted a workaround here: http://osdir.com/ml/perl-beginners/2011-01/msg00182.html
It worked for me.


I'm using Perl provided by ActiveState.