112062 - Node.namespaceURI values incorrect for "empty" xmlns values

Status: RESOLVED INVALID

(Core :: XML, defect)

Description

[Ian Graham]

From Bugzilla Helper:
User-Agent: Mozilla/5.0 (Windows; U; Win98; en-US; rv:0.9.6+) Gecko/20011126
BuildID:    2001112603

namespaceURI properties for XML element nodes are not 
properly set when the namespace URI string is 'empty' -- they are
set to null, instead of to an empty string.

The correct behavior is as follows: If <foo> is the root element 
for a document, then the following should be the case:

<foo>          --  node.namespaceURI=null (null for the namespaceURI)  
<foo xmlns=""> --  node.namespaceURI=""   (an 'empty' string for
                   namespaceURI)

This is an artefact of the DOM specification [From DOM 2 (core)
spec, Sect 1,1,8, paragraph 3]:

    "Note that because the DOM does no lexical checking, the empty 
    string will be treated as a real namespace URI in DOM Level 2 
    methods. Applications must use the value null as the namespaceURI 
    parameter for methods if they wish to have no namespace."

This wording is unchanged in the current DOM 3 draft. 

This is kinda an 'edge' behavior, as the XML namespace specification 
[Section 5.2] states the following:

    "The default namespace can be set to the empty string. This has 
    the same effect, within the scope of the declaration, of there 
    being no default namespace."

So that although the DOM requires that <foo> and <foo xmlns=""> be
different (one having namespaceURI=null and the other 
namespaceURI=""), the namespace spec says that the two are
equivalent, and declare that there is 'no' default namespace.

To illustrate the bug, I will attach a ZIP file consisting of an 
XML doc which loads a script that processes the XML tree and 
writes out namespace-decorated element and attribute names.

Comment 1

[Ian Graham]

Attachment: ZIP file with XML doc and script that illustrate the bug

Attached example XML and script.

Comment 2

[Johnny Stenback (:jst)]

Hmm, sounds like we screw this up in the XML parsing code and not in the DOM,
over to XML...

Comment 3

[Heikki Toivonen (remove -bugzilla when emailing directly)]

I think this is one of those: should we return null or empty string cases. I
think this can be fixed in nsNodeInfo::GetNamespaceURI(), but I haven't tested
yet. Should we go through all of those null vs. "" cases at the same time?

Comment 4

[Heikki Toivonen (remove -bugzilla when emailing directly)]

Actually jst is right: the sink & namespaces interaction does not take into
account empty namespace declaration, but treats empty the same as no namespace.
We parse things correctly, though: xmlns="" correctly resets the default
namespace to nothing. The discrepancy between what is required of parsing and
DOM means that this is a bit more work than I anticipated. (Changing only
nsNodeInfo::GetNamespaceURI() would make us always return empty string instead
of null.) Futuring for now since this is such an edge case, and it seems
returning null would cover typical cases correctly.

Comment 5

[Heikki Toivonen (remove -bugzilla when emailing directly)]

Attachment: testcase

Comment 6

[Hixie (not reading bugmail)]

I strongly believe that this is a bug in the DOM spec. If <foo> and <foo
xmlns=""> are exactly equivalent, then they should be equivalent at the DOM
level too, IMHO.

Is there any reason why we shouldn't push back to the DOM WG to get this
changed? Or at least to make always returning null a valid behaviour?

Comment 7

[Ian Graham]

I believe this was considered between DOM1 and DOM2, but rejected ... don't 
know why, but it was probably a S/W compatibility reason.

THe "Names in XML 1.1" spec change things somewhat, with normative text such as 
(Section 5.1):

  "The attribute value in a namespace declaration for a prefix may be empty.
  This has the effect, within the scope of the declaration, of removing any 
  association of the prefix with a namespace name. "

THis may change things for DOM3, but I suspect the current DOM implementation 
should comply with the admittedly weird DOM1/2 model. 

Comment 8

[Hixie (not reading bugmail)]

We should get DOM2 errata'ed. That's what we've done to the parts of CSS2 that
made no sense.

Comment 9

[Ian Graham]

Well, I suppose the DOM-WG perspective would be that DOM 1/2 is consistently 
specified and unambiguously implementable -- and thus given existing compliant 
implementations not 'errata-able' in the same way as CSS2 (where parts were not 
well-specified, and were open to conflicting and incompatible 
interpretations).  

I think it's worth a try though, if only to keep awareness of this issue up-
front. 

Comment 10

[Hixie (not reading bugmail)]

Well, since DOMs can be written on top of infosets, and infosets won't store the
information required here (right?), I'd say its not necessarily unambiguously
implementable by everyone.

Comment 11

[Ian Graham]

Well DOM1/2 were designed before the infoset, so I don't think you can use that 
argument to modify those DOM specs. OTOH, the DOM3 (http://www.w3.org/TR/DOM-
Level-3-Core/core.html) core draft makes heavy reference to the infoset, and 
appears to be designed to support 'both' the pre-infoset and post-infoset views 
of XML .... 

The more pragmatic question, of course, is whether or not existing DOM code 
takes advantage of the difference between null and empty-string namespaceURI 
parameter values.  I don't know the answer to that -- in my own (limited 
experience case), I've generally had to code around the problem 
if (x.namespaceURI=null | x.namespaceURI="") {.....

Comment 12

[Hixie (not reading bugmail)]

Any code which takes _advantage_ of this "feature" is, IMHO, broken, since it
would violate the namespaces-in-XML spec.

Comment 13

[Ian Graham]

I agree that this is bad practice (perhaps an anti-feature ;-) ) but since this 
behavior is normatively defined in the DOM 1/2 spec, Mozilla should support 
it.  I think of this as one of those nasty legacy things .... and hope it goes 
away in DOM 3. 

Comment 14

[Hixie (not reading bugmail)]

The point of standards compliance is not to follow the specs to the letter, it's
to implement interfaces that can be reliably used by authors without having to
check what environment they are running in first. I don't see any value in
implementing this "feature" as is, and I don't see any reason why the DOM WG
should not errata it. It seems to me to be a pretty clear oversight, which would
probably have been caught at CR if the DOM2 specs had had a serious CR stage.

Another example of a poorly designed DOM2 spec is DOM2 CSS, and that interface
is being seriously overhauled to remove the errors, even those that are
ambiguous and even already implemented, in order to give authors a good API to
work with.

I really think we should push back to the WG on this issue.

Comment 15

[Ian Graham]

Well, you can try but I suspect the response will be that 'this is being fixed 
in DOM3 core' ....  

I don't know the history of discussion on point, and even after an hour of 
pouring over the public list archives couldnt' find any discussion of it, 
although there must be something in there.  If you have access to the internal 
WG lists, perhaps there's something there ... 

I poked around the crimson parser code (version at http://www.kbs.uni-
hannover.de/~firma3/source/org/apache/crimson/tree/XmlDocumentBuilderNS.java.htm
l) and noted the following chunk of code:

// Translate a SAX empty string to mean no namespaceURI
   if ("".equals(namespaceURI)) {
        namespaceURI = null;
   }

which implies that others have decided to ignore this odd part of the spec...  
I just don't know how general this is, nor how dependent any code might be on 
the current DOM 1/2 model. 

The evidence (lack of bug reports to mozilla) would seem to imply that the 
current mozilla/DOM behavior is acceptable. 

Comment 16

[Hixie (not reading bugmail)]

Well, if DOM3 is changing this, doesn't that rather imply DOM2 is wrong? :-)

Comment 17

[Ian Graham]

Umm... well yes. But at least well specified wrongness.  I certainly wish DOM1/2
had been specified differently. But I'm used to coding around well-specified,
but bad, choices, so I'd prefer consistent bad functionality across all
implementations over an inconsistent mix of bad and good.

Perhaps I'm a bit too cynical ;-)

Comment 18

[Hixie (not reading bugmail)]

What I mean is if DOM2 and DOM3 are going to be mutually incompatible in this
regard, doesn't it make more sense to errata DOM2 to match what DOM3 will say.
The CSSWG in the past has "undefined" aspecs of CSS2 because they were either
woefully badly implemented or stupidly defined, and we knew that CSS3 was going
to change the way they were defined.

Comment 19

[Cesar Eduardo Barros]

"Note that because the DOM does no lexical checking, the empty string will be
treated as a real namespace URI in DOM Level 2 methods. Applications must use
the value null as the namespaceURI parameter for methods if they wish to have no
namespace."

I think that means that when you are calling a DOM Level 2 method, "" and null
are different values, which means you can create a element with a namespace URI
of "" using DOM. Nowhere it says that the parser should treat no xmlns and
xmlns="" differently. In fact, the XML Namespaces spec says that xmlns="" and no
xmlns are the same, which is probably why they put that note in the DOM spec (to
tell people to use null and not "", because using "" would result in unexpected
behavior).

The key part seems to be "in DOM Level 2 methods", which means the distinction
only exists in the DOM interfaces. I see no conflict between both standards.

Comment 20

[Alex Vincent [:WeirdAl]]

Looks like the DOM WG has agreed with Hixie.

http://www.w3.org/2000/11/DOM-Level-2-errata#core-27

The section of the DOM specification which this bug has been filed against (as
mentioned in comment 0 and comment 19) should now read:

Applications should use the value null as the namespaceURI parameter for methods
if they wish to have no namespace. In programming languages where empty strings
can be differentiated from null, the way empty strings are treated, when given
as a namespace URI to a DOM Level 2 method, is implementation dependent. This is
true even though the DOM does no lexical checking of URIs.

By the words "implementation dependent," I think we can resolve this bug as
INVALID (but it's still a very useful bug to have filed!)

Comment 21

[Hixie (not reading bugmail)]

Yay for the DOM WG!

Comment 22

[Ian Graham]

And to think they (well, me actually) said it couldn't be done.

I don't know how it got done, but congrats to any and all involved!