Last Comment Bug 413974 - [FIX]Firefox appends charset to Content-Type: multipart/form-data in XmlHttpRequest, PHP can't parse it
: [FIX]Firefox appends charset to Content-Type: multipart/form-data in XmlHttpR...
Status: RESOLVED FIXED
:
Product: Core
Classification: Components
Component: DOM (show other bugs)
: Trunk
: x86 Other
: P2 normal (vote)
: ---
Assigned To: Boris Zbarsky [:bz]
:
Mentors:
: 415876 (view as bug list)
Depends on:
Blocks:
  Show dependency treegraph
 
Reported: 2008-01-25 05:06 PST by Michal Krause
Modified: 2013-04-04 13:53 PDT (History)
10 users (show)
pavlov: blocking1.9+
bzbarsky: in‑testsuite+
See Also:
Crash Signature:
(edit)
QA Whiteboard:
Iteration: ---
Points: ---
Has Regression Range: ---
Has STR: ---


Attachments
Fix (14.04 KB, patch)
2008-02-21 00:04 PST, Boris Zbarsky [:bz]
no flags Details | Diff | Review
Same without the extra test in the makefile (14.02 KB, patch)
2008-02-21 00:06 PST, Boris Zbarsky [:bz]
cbiesinger: review+
jonas: review+
jonas: superreview+
mbeltzner: approval1.9b4+
Details | Diff | Review

Description Michal Krause 2008-01-25 05:06:49 PST
User-Agent:       Mozilla/5.0 (X11; U; Linux i686; cs; rv:1.9b2) Gecko/2007121016 Firefox/3.0b2
Build Identifier: Mozilla/5.0 (X11; U; Linux i686; cs; rv:1.9b2) Gecko/2007121016 Firefox/3.0b2

Firefox 3.0b2 appends charset to Content-Type header in XmlHttpRequest even if content type is multipart/form-data. Header in form

Content-Type: multipart/form-data; boundary=-----------------------------AaB03x; charset=UTF-8

is not successfully parsed by PHP (at least in version 5.2) which makes some applications broken with Firefox 3.0b2 (upload plugin for TiddlyWiki, for example).

Temporary solution is to force charset field in Content-Type header before boundary - than PHP can parse this header. It may be bug in PHP request parser (but does it make sense to have charset in Content-Type: multipart/form-data?), but there is probably no chance that all PHP installations will be fixed in near future so it's probably easier to workaround it in Firefox?

There is no such problem in Firefox 2.x.

Example page and PHP script:

============ HTML
<html>
    <head>
        <script type="text/javascript">
            function upload(withCharset) {
                var xmlhttp = (window.XMLHttpRequest ? new XMLHttpRequest : (window.ActiveXObject ? new ActiveXObject("Microsoft.XMLHTTP") : false));
                if (!xmlhttp) {
                    return false;
                }
                xmlhttp.open("POST", "http://localhost/test.php");
                xmlhttp.onreadystatechange = function() {
                    if (xmlhttp.readyState == 4) {
                        alert(xmlhttp.responseText);
                    }
                };
                var boundary = "-----------------------------AaB03x";
                if (withCharset) {
                    xmlhttp.setRequestHeader("Content-Type", "multipart/form-data; charset=UTF-8; boundary="
                            + boundary);
                } else {
                    xmlhttp.setRequestHeader("Content-Type", "multipart/form-data; boundary="
                            + boundary);
                }

                var fileContent = "xxxx";
                var content = "--" + boundary + "\r\n";
                content += "Content-disposition: form-data; name=\"userfile\"; filename=\"file.txt\"\r\n";
                content += "Content-Type: text/plain; charset=UTF8\r\n";
                content += "Content-Length: " + fileContent.length + "\r\n\r\n";
                content += fileContent;
                content += "\r\n--" + boundary + "--\r\n";

                xmlhttp.send(content);
                return true;
            }
        </script>
    </head>

    <body>
        <a href="#" onclick="upload(0); return false;">upload file with AJAX</a><br />
        <a href="#" onclick="upload(1); return false;">upload file with AJAX (with charset)</a>
    </body>

</html>

============= PHP script

<? var_dump($_FILES); ?>

Reproducible: Always

Steps to Reproduce:
1. create PHP script with content noted above and place it to website with PHP support
2. create HTML page with content noted above and change URL used in xmlhttp.open to URL of PHP script
3. open HTML page in Firefox 3.0b2
Actual Results:  
click on first link: alert will display "array(0) {}"

click on second link: alert will display something like this:
array(1) {
  ["userfile"]=>
  array(5) {
    ["name"]=>
    string(8) "file.txt"
    ["type"]=>
    string(10) "text/plain"
    ["tmp_name"]=>
    string(14) "/tmp/phpACkzbd"
    ["error"]=>
    int(0)
    ["size"]=>
    int(4)
  }
}


Expected Results:  
click on both links should display something like this:
array(1) {
  ["userfile"]=>
  array(5) {
    ["name"]=>
    string(8) "file.txt"
    ["type"]=>
    string(10) "text/plain"
    ["tmp_name"]=>
    string(14) "/tmp/phpACkzbd"
    ["error"]=>
    int(0)
    ["size"]=>
    int(4)
  }
}
Comment 1 Boris Zbarsky [:bz] 2008-01-25 19:09:46 PST
So the problem is that if the charset comes before the boundary param PHP is OK, and if it comes after the boundary param PHP shows its "MIME spec, what MIME spec?" colors once more?

I guess we can try to move the charset param around... blizzard, shaver sez you can follow this up with the PHP folks?
Comment 2 Boris Zbarsky [:bz] 2008-01-25 19:50:41 PST
So one option here is to have extractCharsetFromContentType set the charset position to right after the winning type if that winning type has no charset param.  That would guarantee that the charset= comes before the boundary=.  Christian, what do you think of that approach?
Comment 3 Mike Schroepfer 2008-01-27 09:43:02 PST
Moving to blocking p1 because of web compat
Comment 4 Damon Sicore (:damons) 2008-01-28 17:43:26 PST
Assigning to Jonas.
Comment 5 Jonas Sicking (:sicking) 2008-01-28 17:44:40 PST
The solution in comment 2 sounds like a good one FWIW.

Why can't people just parse their MIME correctly :(
Comment 6 Michal Krause 2008-01-29 02:27:17 PST
It seems that PHP parses headers according to obsoleted RFC 1867 where comma is mentioned as separator (section 6, Examples). Parsing of header is not too smart - it looks for substring "boundary" and then try to trim everything from next comma to end of line. This is the reason, why charset before boundary doesn't bother.
Comment 7 Boris Zbarsky [:bz] 2008-01-29 07:45:27 PST
Uh... the examples section there is non-normative, and doesn't actually follow the MIME RFCs relevant at the time...  I'd certainly hope no one wrote a parser based on the examples section.  :(  Clearly I hope in vain....
Comment 8 Phil Ringnalda (:philor) 2008-02-05 23:25:22 PST
*** Bug 415876 has been marked as a duplicate of this bug. ***
Comment 9 Boris Zbarsky [:bz] 2008-02-05 23:32:50 PST
Another, possibly simpler and maybe more correct, solution is to just not append a charset at all for multipart/form-data, since the type isn't actually defined to have such a parameter.  It's the responsibility of the data creator to set the charset on each part individually...

Ideally there would be a simple way to tell from the type whether it wants a charset param, but there isn't.  So we'd have to just hardcode the "multipart/form-data".
Comment 10 Ian Bugeja 2008-02-06 23:04:58 PST
I managed to solve it by simply adding the charset myself before the boundary... it seems that php was taking the charset as part of the boundary string

the below should fix the trick

http_request.setRequestHeader("Content-Type", "multipart/form-data; charset=utf-8; boundary=---------------------------" + border);
Comment 11 Boris Zbarsky [:bz] 2008-02-07 17:00:53 PST
*** Bug 416178 has been marked as a duplicate of this bug. ***
Comment 12 Alexander Klimetschek 2008-02-13 05:39:16 PST
Please not that the duplicate (and closed) Bug 416178 is more general: it affects normal XHR requests of form posts that have the standard application/x-www-form-urlencoded encoding. The appended charset= breaks many webservers and is thus considered a bad-thing-on-the-web(tm), although the HTTP spec allows it.

There is *no* workaround for it and this bug will break some Ajax apps out there.
Comment 13 Damon Sicore (:damons) 2008-02-20 17:45:41 PST
Jonas, any progress on this one?
Comment 14 Jonas Sicking (:sicking) 2008-02-20 18:43:05 PST
Smaug: You have the cycles for this one?
Comment 15 Boris Zbarsky [:bz] 2008-02-21 00:04:05 PST
Created attachment 304679 [details] [diff] [review]
Fix
Comment 16 Boris Zbarsky [:bz] 2008-02-21 00:06:55 PST
Created attachment 304680 [details] [diff] [review]
Same without the extra test in the makefile
Comment 17 Jonas Sicking (:sicking) 2008-02-26 02:20:07 PST
Comment on attachment 304680 [details] [diff] [review]
Same without the extra test in the makefile

Looks good. Feel free to land, but it wouldn't hurt to get biesis input too at some point.
Comment 18 Boris Zbarsky [:bz] 2008-02-26 20:50:12 PST
The network stuff really needs review from biesi before I'd be comfortable landing it.
Comment 19 Christian :Biesinger (don't email me, ping me on IRC) 2008-02-27 12:57:29 PST
+                charsetStart =  flatStr.Length();

should have just one space
Comment 20 Boris Zbarsky [:bz] 2008-02-27 13:08:06 PST
Comment on attachment 304680 [details] [diff] [review]
Same without the extra test in the makefile

Requesting approval.  I think we want to fix this for the beta...  Risk should be pretty low, win is reasonably big.
Comment 21 Mike Beltzner [:beltzner, not reading bugmail] 2008-02-27 21:37:06 PST
Comment on attachment 304680 [details] [diff] [review]
Same without the extra test in the makefile

a1.9b4=beltzner
Comment 22 Boris Zbarsky [:bz] 2008-02-28 20:24:50 PST
Fixed.
Comment 23 Alexander Klimetschek 2008-03-11 12:33:30 PDT
Does not fix bug 416178.

Note You need to log in before you can comment on or make changes to this bug.