540854 - XMLHttpRequest truncates UTF-8 encoded strings containing Code Point 0x00

Status: RESOLVED FIXED

(Core :: DOM: Core & HTML, defect)

Description

[johann.borck]

User-Agent:       Mozilla/5.0 (X11; U; Linux i686; en-US) AppleWebKit/532.5 (KHTML, like Gecko) Chrome/4.0.249.43 Safari/532.5
Build Identifier: Gecko/20091216 Firefox 3.5.6

when sending an UTF-8 encoded string using XMLHttpRequest (method POST), that contains the Code Point 0, the content is truncated at the position of the 0x00-byte. The Content-Length is set accordingly: if the first byte in the string is a 0x00, the content-length will also be 0. 

Reproducible: Always

Steps to Reproduce:
1. send a utf-8 encoded string, that contains Codepoint 0 via XMLHttpRequest
2. use wireshark or something else to observe the bytes actually sent.
Actual Results:  
the string is not being transmitted.

Expected Results:  
the complete string should be transmitted.

are you using 0-terminated C-strings for this? OMG.

Comment 1

[Johnathan Nightingale [:johnath]]

-> Core

Component's going to be a wild guess, but it'll get more attention there, anyhow.

Comment 2

[Olli Pettay [:smaug][[e-mail]]]

A testcase would be great there.

(but I thinks where the problem is)

Comment 3

[Olli Pettay [:smaug][[e-mail]]]

er, I think I know where the problem is.

Comment 4

[johann.borck]

(In reply to comment #2)
> A testcase would be great there.

Sure, is there a recommended (existing) environment, like a scriptable server for this kind of test?

Comment 5

[Olli Pettay [:smaug][[e-mail]]]

Well, if you could upload the client side of the test?
a *minimal* testcase please.

Comment 6

[Olli Pettay [:smaug][[e-mail]]]

Is this what you mean here?
http://mozilla.pettay.fi/xhr_upload/utf8_send.html

Comment 7

[johann.borck]

yes, that's it, it also fails if the 0 is somewhere in the string:
function getUTF8() {
    return String.fromCharCode(1,2,0,1,2,0);
}

Comment 8

[Olli Pettay [:smaug][[e-mail]]]

Attachment: patch

I think we want this.

Btw, Opera has this same bug.

Comment 9

[johann.borck]

definitely, very nice.
webkit/google chrome/(?safari?) also do, though only if 0 is the first byte.

Comment 10

[Olli Pettay [:smaug][[e-mail]]]

I did test Safari 4 and Chrome and having 0 as the first byte works there

Comment 11

[johann.borck]

(In reply to comment #10)
> I did test Safari 4 and Chrome and having 0 as the first byte works there

os x or windows? I only tested chrome on linux

Comment 12

[johann.borck]

hm, seems I was wrong. it works now. Strange, I thought I noticed the bug on chrome first.. never mind.

Comment 13

[Olli Pettay [:smaug][[e-mail]]]

http://hg.mozilla.org/mozilla-central/rev/dd7fe7248c58