process large file

[Jimmy Zhang]

I am having some trouble processing some large file (40mb) in Java. For the
problem I have, I have tried to use SAX, but doesn't
find it suitable (well, coding just becomes a little complicated). So DOM is
better but a little overhead on memory. Can someone share with me their
experiences in dealing with their situations?

Cheers,
JZ

 

[Toivo Lainevool]

I am having some trouble processing some large file (40mb) in Java. For the
problem I have, I have tried to use SAX, but doesn't
find it suitable (well, coding just becomes a little complicated). So DOM is
better but a little overhead on memory. Can someone share with me their
experiences in dealing with their situations?

One good option is to use a pull parser. It has a simpler interface
than SAX, but doesn't have the memory overhead of DOM.

See http://www.extreme.indiana.edu/xgws/xsoap/xpp/ or
http://www.xmlpull.org/

Toivo Lainevool
http://www.XMLPatterns.com - Develop effective DTDs and XML Schema
documents for your XML using structural design patterns.

 

[Alexey Shirshov]

Hello, Toivo!
You wrote on 7 Apr 2004 12:09:05 -0700:


[Sorry, skipped]

TL> One good option is to use a pull parser. It has a simpler interface
TL> than SAX, but doesn't have the memory overhead of DOM.

TL> See http://www.extreme.indiana.edu/xgws/xsoap/xpp/ or
TL> http://www.xmlpull.org/

Intresting, that MS calls it cursor model processing (XPathNavigator), and
based on SAX calls push/pull model (XmlWriter/XmlReader).

With best regards, Alexey Shirshov.

 

[Jimmy Zhang]

How much memory does XPathNavigator consume? I assume it loads everything in
memory like DOM.

 

[Alexey Shirshov]

Hello, Jimmy!
You wrote on Thu, 08 Apr 2004 23:40:00 GMT:

JZ> How much memory does XPathNavigator consume? I assume it loads
JZ> everything in memory like DOM.

Well, XPathNavigator is just an interface (actually abstract class) and we
cann't talk about it performance. The important thing is an implementation
of this class. XmlDocument, which represents DOM implements it in
DocumentXPathNavigator class.
You can create the instance of this class via CreateNavigator method.
Another implementation you can get via CreateNavigator of the XPathDocument
class.
First implementation uses DOM as underlying data model, while the second -
XPath data model.
I think, for very large documents the XPathDocument will be much faster.


[Sorry, skipped]


With best regards, Alexey Shirshov.