1752761 - multipart/form-data fetch cannot be parsed with Response.formData()

Status: RESOLVED FIXED

(Core :: DOM: Networking, defect, P2)

Description

[szustkarol]

Attachment: simple_client_and_server.zip

User Agent: Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:96.0) Gecko/20100101 Firefox/96.0

Steps to reproduce:

1. Create a simple web server, that serves multipart/form-data as response. Might be of any type but You might want to use the attached python script to reproduce it fast.
2. Create a client that uses fetch to request a multipart/form-data result from the server. You might want to use the attached client.html file.
3. call fetch() to get the response from the server
4. Optionally, call .formData on the response

Also, I tried to send the same request (same headers) with Postman, and the bug does not occur.

The attached file is the simplest example I was able to reproduce. I have also encountered this problem when creating a Scala server:
https://stackoverflow.com/questions/70917217/play-framework-build-a-temporaryfile-from-a-file-stored-in-ram

Actual results:

Instead of returning the actual response content, the browser for some reason acquries the base64 (utf-8) encoded version of it (check the network -> request -> response tab and compare the result with Chromium).
This is by itself a bug, but it also makes it impossible to use the Response.formData method on the result, yielding "TypeError: Could not parse content as FormData.".

Expected results:

One should be able to use the .formData on the result, and it should not be encoded in base64 - NOT sending the data in base64 is the idea behind multipart replies in the first place. Check how the sample site behaves on Chromium.

Comment 1

[szustkarol]

Forgot to mention, but using .text() on the response yields valid result.

Comment 2

[BugBot [:suhaib / :marco/ :calixte]]

The Bugbug bot thinks this bug should belong to the 'Core::Networking' component, and is moving the bug to that component. Please revert this change in case you think the bot is wrong.

Comment 3

[Valentin Gosu [:valentin] (he/him)]

The form data parser failure is coming from here
The reason for the failure on this specific test case, is that in PushOverBoundary the line starts not with -- but with \r\n--
I'm not sure if the extra \r\n is coming from the python server implementation, or if fetch should skip it.
Considering Chrome works, I assume it's the second. 🙂

Comment 4

[Valentin Gosu [:valentin] (he/him)]

The devtools issue is separate from the issue here. I think the netmonitor by default shows "unknown" binary content as base64.
You may open a different bug in the netmonitor component for supporting multipart/form-data payloads.

Comment 5

[Valentin Gosu [:valentin] (he/him)]

Attachment: Bug 1752761 - response.formData() throws error r=#necko

This is caused by an extra \r\n at the begining of the content.

Comment 6

[szustkarol]

I did check it, and after removing the added \r\n Chrome doesn't parse the response, but Firefox doesn't too.

In the sample server just replace the line 16 and 20, to respond with single \r\n:
self.wfile.write(bytes("Content-Type: text/plain\r\n", "utf-8"))

Comment 7

[Valentin Gosu [:valentin] (he/him)]

The problem here is line 12

self.send_header("Content-type", "multipart/form-data; boundary=---XXX\r\n")
There should be no \r\n at the end, because send_header also adds a \r\n. So that \r\n goes into the response body making it start with a \r\n.

Comment 8

[szustkarol]

Have You been able to successfully run the sample client & server after removing the \r\n from line 12 then?
Because for me the problem persists after altering the server (i.e. formData() still throws an error).

Comment 9

[szustkarol]

Hey It's been a year, maybe we should have an update on this bug?

Comment 10

[Valentin Gosu [:valentin] (he/him)]

Feel free to take the bug if you want.

Comment 11

[Kershaw Chang [:kershaw]]

Hi Reporter,

Could you try to download this test build and see if you can still reproduce this issue?
I've tested the sample server locally and I can see the response was parsed successfully.

Thanks.

Comment 12

[Kershaw Chang [:kershaw]]

Attachment: Bug 1752761 - Fix FormDataParser issue due to extra \r\n, r=#necko

Comment 13

[BugBot [:suhaib / :marco/ :calixte]]

Redirect a needinfo that is pending on an inactive user to the triage owner.
:smayya, since the bug has recent activity, could you have a look please?

For more information, please visit BugBot documentation.

Comment 14

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/39a6d30ab71c
Fix FormDataParser issue due to extra \r\n, r=sunil

Comment 15

[pstanciu]

https://hg.mozilla.org/mozilla-central/rev/39a6d30ab71c