1467010 - Add Accept-Encoding: identity to Range requests

Status: RESOLVED WORKSFORME

(Core :: Networking: HTTP, enhancement, P3)

Description

[Jake Archibald [:jakea]]

Spec change: https://github.com/whatwg/fetch/pull/751
Tests: https://github.com/web-platform-tests/wpt/pull/11291

Turns out a significant number of servers ignore Range headers unless "Accept-Encoding: identity" is set. This spec change applies that header to all requests with a Range header.

Firefox current omits the Accept-Encoding header for media requests, which feels risky.

Perhaps related: https://bugzilla.mozilla.org/show_bug.cgi?id=613159

Comment 1

[Daniel Stenberg [:bagder]]

I find that an unfortunate restriction in those servers, but I suppose that doesn't change anything... =)

Comment 2

[Jake Archibald [:jakea]]

I agree, but it prevents a pretty big gotcha.

We could in future provide an opt-out of this. It would require some way for developers and spec authors to say "give me the body before decoding", along with a way to set the `Accept-Encoding` header.

Comment 3

[Anne (:annevk)]

Folks are hitting this in the wild by the way: https://stackoverflow.com/questions/55914486/issue-making-range-requests-in-some-browsers.

Comment 4

[Tom S. (please needinfo tschuster)]

Firefox fails a bunch of tests because of this: https://wpt.fyi/results/fetch/range?label=experimental&label=master&aligned.

Comment 5

[BugBot [:suhaib / :marco/ :calixte]]

The bug has a release status flag that shows some version of Firefox is affected, thus it will be considered confirmed.

Comment 6

[Thomas Wisniewski [:twisniewski]]

We are already doing this: https://searchfox.org/mozilla-central/source/netwerk/protocol/http/nsHttpChannel.cpp#793-798

The remaining WPT failures listed in comment 4 will mostly pass after the patch in bug 1784880 lands, with the failure in general.any.serviceworker.html being shared among browsers right now, and the timeouts/crashing in sw.https.window.html seeming unrelated (the tests pass for me locally, so it's not likely to be related to this).