Elementtree and CDATA handling

A

alainpoint

I am experimenting with ElementTree and i came accross some
(apparently) weird behaviour.
I would expect a piece of XML to be read, parsed and written back
without corruption (except for the comments and PI which have purposely
been left out). It isn't however the case when it comes to CDATA
handling.
I have the following code:
text="""<html><head>
<title>Document</title>
</head>
<body>
<script type="text/javascript">
//<![CDATA[
function matchwo(a,b)
{
if (a < b && a > 0) then
{
return 1
}
}
//]]>
</script>
</body>
</html>
"""

from elementtree import ElementTree
tree = ElementTree.fromstring(text)
ElementTree.dump(tree)

Running the above piece of code yields the following:

<html><head>
<title>Document</title>
</head>
<body>
<script type="text/javascript">
//
function matchwo(a,b)
{
if (a &lt; b &amp;&amp; a &gt; 0) then
{
return 1
}
}
//
</script>
</body>
</html>

There are two problems: the //<![CDATA[ has disappeared and the <, >
and && have been replaced by their equivalent entities (CDATA should
have prevented that).
I am no XML/HMTL expert, so i might be doing something wrong...
Thank you for helping

Alain
 
F

Fredrik Lundh

There are two problems: the //<![CDATA[ has disappeared and the <, >
and && have been replaced by their equivalent entities (CDATA should
have prevented that).
I am no XML/HMTL expert, so i might be doing something wrong...
Click to expand...

you're confusing the external representation of something with the internal
data model.

consider this:

the above are six ways to write the same string literal in Python. all these result
in a five-character string containing the letters "h", "e", "l", "l", and "o". if you type
the above at a python prompt, you'll find that Python echoes the strings back as
'hello' in all six cases.

in XML, entities, character references, and CDATA sections are three different
way to represent reserved characters. once you've loaded the file, they all "dis-
appear".

</F>
 
T

Terry Reedy

Fredrik Lundh said:
you're confusing the external representation of something with the
internal
data model.

consider this:


the above are six ways to write the same string literal in Python.
Click to expand...

Minor nit: I believe 'hell' + 'o' is two string literals and a runtime
concatenation operation. Perhaps you meant 'hell' 'o', without the '+',
which I believe is joined to one literal at parsing or compilation time.
all these result
in a five-character string containing the letters "h", "e", "l", "l", and
"o".
if you type the above at a python prompt,
you'll find that Python echoes the strings back as
'hello' in all six cases.
Click to expand...

Nit aside, this is a valuable point that bears repeating. Another example
of one internal versus multiple external that confuses many is the
following:
1.1 == 1.1000000000000001 # True

The mapping of external to internal is many-to-one for both strings and
floats and therefore *cannot* be exactly inverted! (Or round-tripped.) So
Python has to somehow choose one of the possible external forms that would
generate the internal form.

Terry J. Reedy
 
A

and-google

Alain said:
I would expect a piece of XML to be read, parsed and written back
without corruption [...]. It isn't however the case when it comes
to CDATA handling.
Click to expand...

This is not corruption, exactly. For most intents and purposes, CDATA
sections should behave identically to normal character data. In a real
XML-based browser (such as Mozilla in application/xhtml+xml mode), this
line of script would actually work fine:
if (a &lt; b &amp;&amp; a &gt; 0) {
Click to expand...

The problem is you're (presumably) producing output that you want to be
understood by things that are not XML parsers, namely legacy-HTML web
browsers, which have special exceptions-to-the-rule like "<script>
elements don't contain markup" that are not present in XML.

ElementTree is a data binding that strives to simplify the XML
processing experience, and as such it folds CDATA sections down to
plain characters - this is usually easier for programmers to deal with.
Such a feature is considered normal in XML processing, and is the
default for, eg. DOM Level 3 implementations.

If, instead, you want to keep track of where the CDATA sections are,
and output them again without change, you'll need to use an
XML-handling interface that supports this feature. Typically, DOM
implementations do - the default Python minidom does, as does pxdom.
DOM is a more comprehensive but less friendly/Python-like interface for
XML processing.

There are a few other obstacles you may meet if you are outputting XML
for use by a non-XML parser (legacy browsers):

- entity references - &eacute; etc. The HTML entities are not
built into XML so to read them at all you'll need a parser that
reads the external DTD subset (and a suitable !DOCTYPE). Even then
they'll be converted to text, if that matters. (pxdom, optionally,
can keep them as entity references regardless of whether their
content is known);

- empty elements - <img/> etc. An XML serialiser won't know how to
output this is a browser-compatible way. (The next release of pxdom
has an option to do so.)

If you're generating output for legacy browsers, you might want to just
use a 'real' HTML serialiser.
 
U

uche.ogbuji

"If, instead, you want to keep track of where the CDATA sections are,
and output them again without change, you'll need to use an
XML-handling interface that supports this feature. Typically, DOM
implementations do - the default Python minidom does, as does pxdom.
DOM is a more comprehensive but less friendly/Python-like interface for
XML processing. "

Amara in CVS makes it easy to perform the output part of this:

text="""<html><head>
<title>Document</title>
</head>
<body>
<script type="text/javascript">
//<![CDATA[
function matchwo(a,b)
{
if (a < b && a > 0) then
{
return 1
}
}

//]]>
</script>
</body>
</html>
"""

from amara.binderytools import bind_string
doc = bind_string(text)
print doc.xml(cdataSectionElements=[u'script'])

Output:

<?xml version="1.0" encoding="UTF-8"?>
<html><head>
<title>Document</title>
</head>
<body>
<script type="text/javascript"><![CDATA[
//
function matchwo(a,b)
{
if (a < b && a > 0) then
{
return 1
}
}

//
]]></script>
</body>
</html>

Unfortunately, in cooking up this example I did find a bug in the Amara
1.0b1 release that requires a workaround. I should be releasing 1.0b2
this weekend, which fixes this bug (among other fixes and
improvements).

"If you're generating output for legacy browsers, you might want to
just
use a 'real' HTML serialiser. "

Amara does provide for this, e.g.:

from amara.binderytools import bind_string
doc = bind_string(text)
print doc.xml(method=u"html")

Which automatically and transparently brings to bear the full power of
the XSLT HTML output method.

--
Uche Ogbuji Fourthought, Inc.
http://uche.ogbuji.net http://fourthought.com
http://copia.ogbuji.net http://4Suite.org
Use CSS to display XML, part 2 -
http://www-128.ibm.com/developerworks/edu/x-dw-x-xmlcss2-i.html
XML Output with 4Suite & Amara -
http://www.xml.com/pub/a/2005/04/20/py-xml.htmlUse XSLT to prepare XML
for import into OpenOffice Calc -
http://www.ibm.com/developerworks/xml/library/x-oocalc/
Schema standardization for top-down semantic transparency -
http://www-128.ibm.com/developerworks/xml/library/x-think31.html
 
F

Fredrik Lundh

Terry said:
Minor nit: I believe 'hell' + 'o' is two string literals and a runtime concatenation operation.
Click to expand...

I guess I should have written "constant".

on the other hand, while the difference might matter for current python
interpreter implementations, it doesn't really matter for the user. after
the operation, they end up with a string object that doesn't contain what
they wrote.

</F>
 

Ask a Question

Want to reply to this thread or ask your own question?

You'll need to choose a username for the site, which only take a couple of moments. After that, you can post your question and our members will help you out.

Ask a Question

Similar Threads

simple ElementTree based parser that allows entity definition map 0
!CDATA 4
About as basic "Newbie-Question" that you can get. 3
Script stops working when using variables to save time typing... 4
lxml/ElementTree and .tail 30
How to save JSON Data to a file using fetch() api? 2
Final chapter of "Learn PHP, MySQL and JavaScript" 3
Why doesn't the function get called? 1

Members online

No members online now.
Total: 56 (members: 1, guests: 55)
Robots: 67

Forum statistics

Threads
473,999
Messages
2,570,243
Members
46,838
Latest member
KandiceChi

Latest Threads

Top