1686765 - Implement spec-compliant newline normalization in form payloads

Status: RESOLVED FIXED

(Core :: DOM: Forms, task)

Description

[Andreu Botella]

Spec PRs: https://github.com/whatwg/html/pull/6282, https://github.com/whatwg/html/pull/6287 (still under review)
Test PRs: https://github.com/web-platform-tests/wpt/pull/27142, https://github.com/web-platform-tests/wpt/pull/26740 (still under review)

The HTML spec is changing the way form payloads are handled to be more specific and match implementations better, in particular around newline normalization and escaping in multipart/form-data payloads.

https://github.com/whatwg/html/pull/6282 specifies the exact escaping implementations must use for multipart/form-data forms: percent-encoding newlines and double quotes. This goes against Firefox's behavior of replacing newlines with a space and escaping double quotes with a backslash. See the tests FileAPI/file/send-file-form-controls.html and FileAPI/file/send-file-form-punctuation.html.

https://github.com/whatwg/html/pull/6287 changes the way newline normalization works: for multipart/form-data, the spec used to require that form entries which originated from forms were normalized (names and string values, but not filenames or file contents), but not entries added through other means (such as by populating a FormData object from JS, or by modifying the entry list from JS through the formdata event). Now all form entries must be normalized. See FileAPI/file/send-file-formdata-controls.html.

Comment 1

[Anne (:annevk)]

John, is this something you want to fix (at least the changes that are no longer under review)? (And perhaps you're also interested in giving feedback on the proposed changes to these algorithms.)

Comment 2

[Andreu Botella]

This doesn't seem like a hard fix, so I'd like to take it up if that's okay. I would appreciate jdai's review here and on the WHATWG PR, though.

Comment 3

[John Dai[:jdai]]

I'll take a look. Keep NI for tracking.

Comment 4

[Andreu Botella]

It seems like the HTML pull request is close to being merged, so John, if you have any feedback, please go ahead and give it.

Comment 5

[Andreu Botella]

Attachment: Bug 1686765 - Update newline normalization in form payloads. r=jdai

This commit also changes the way escapes work in multipart/form-data names and filenames.

Comment 6

[Andreu Botella]

jdai: Could you review the revision?

Comment 7

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/beece7a29ecd
Update newline normalization in form payloads. r=smaug

Comment 8

[Cosmin Sabou [:CosminS]]

Backed out for mochitest failures on test_ext_webrequest_upload.html.

Push with failures: https://treeherder.mozilla.org/jobs?repo=autoland&resultStatus=testfailed%2Cbusted%2Cexception%2Cusercancel&fromchange=beece7a29ecd5a96c1db3a8b19e9556ac0676530&tochange=1660127d222e1ddf9e7e89bff2c92eb73f471305&selectedTaskRun=V_DNRIbVRZu3Y0uSqN-Haw.0

Failure log: https://treeherder.mozilla.org/logviewer?job_id=340614157&repo=autoland

Backout link: https://hg.mozilla.org/integration/autoland/rev/1660127d222e1ddf9e7e89bff2c92eb73f471305

Comment 9

[Andreu Botella]

This revision had to land together with https://phabricator.services.mozilla.com/D108679, since that revision adds the changes needed to the multipart/form-data parser in the WebExtensions API which are needed to get the same result as these changes on the serializer. See bug 1697292. I had linked both revisions, but maybe I got that wrong.

Comment 10

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/3e05faaa217d
Update newline normalization in form payloads. r=smaug

Comment 11

[Narcis Beleuzu [:NarcisB]]

https://hg.mozilla.org/mozilla-central/rev/3e05faaa217d

Comment 12

[Rachel Andrew]

Added to release notes.