1670675 - Brotli compression does not work (at least on http://localhost/ ?)

Status: RESOLVED FIXED

(Core :: DOM: Networking, defect, P3)

Description

[Jukka Jylänki]

Attachment: build_webgl_release_brotli.zip

STR:

1. Download attached file
2. Run start_adhoc_server.bat , which spawns an ad hoc web server on http://localhost:6931/
3. Visit the site

Observe that the page does not load, but fails on error

Uncaught SyntaxError: illegal character
build_webgl_release_brotli.framework.js.br:1:1
Uncaught ReferenceError: unityFramework is not defined
    onload http://localhost:9001/CallJSFromCSharp/build_webgl_release_brotli/Build/build_webgl_release_brotli.loader.js:1
    d http://localhost:9001/CallJSFromCSharp/build_webgl_release_brotli/Build/build_webgl_release_brotli.loader.js:1
    d http://localhost:9001/CallJSFromCSharp/build_webgl_release_brotli/Build/build_webgl_release_brotli.loader.js:1
    u http://localhost:9001/CallJSFromCSharp/build_webgl_release_brotli/Build/build_webgl_release_brotli.loader.js:1
    createUnityInstance http://localhost:9001/CallJSFromCSharp/build_webgl_release_brotli/Build/build_webgl_release_brotli.loader.js:1
    createUnityInstance http://localhost:9001/CallJSFromCSharp/build_webgl_release_brotli/Build/build_webgl_release_brotli.loader.js:1
    onload http://localhost:9001/CallJSFromCSharp/build_webgl_release_brotli/:73
    EventHandlerNonNull* http://localhost:9001/CallJSFromCSharp/build_webgl_release_brotli/:72
build_webgl_release_brotli.loader.js:1:3167

The ad hoc web server delivers all files that have suffix '.br' with "Content-Encoding: br".

Some observations:

1. Firefox does not send HTTP Request Header Accept-Encoding: br along with the request, which suggests that Firefox does not support Brotli here. Maybe it is due to the ad hoc server not being HTTPS? Note that Chrome (and Chrome-based Edge) and Safari all support this. If HTTPS is the issue, can http://localhost:<any_port>/ and http://127.0.0.1:<any_port>/ domains be treated special and allowed to support Brotli?

2. Even when Firefox does not support Brotli, it still attempts to load the received files as bare uncompressed content(!) That is, Firefox well knows when receiving the file that it had Content-Encoding: br, so it is compressed with some unknown or unsupported compression. Why does it still attempt to parse the received file?

That is, instead of an error

Uncaught SyntaxError: illegal character
build_webgl_release_brotli.framework.js.br:1:1

Firefox should print something along the lines of

Uncaught UnsupportedEncodingError: Content-Encoding: 'br' is only supported over HTTPS protocol.

instead of pretending the file did not have any compression at all?

3. Please add a boolean pref support.brotli.over.insecure.http so that developers can use Firefox to develop Brotli-enabled pages. Needing to do HTTPS just to be able to locally develop is an insane ask for developers. Paired with (1.) above that would make Brotli development less painful.

Or perhaps the "Brotli is HTTPS only" is a misinterpretation here, and the root cause of the bug is something else? (has Brotli support regressed?)

Comment 1

[Valentin Gosu [:valentin] (he/him)]

You are correct that brotli is HTTPS only.
https://hacks.mozilla.org/2015/11/better-than-gzip-compression-with-brotli/#comment-19069

I'll try to take a look at what fails there.
You're right that we should be better at handling this issue.
I'll try to take a look at why things are breaking tomorrow.

Comment 2

[Valentin Gosu [:valentin] (he/him)]

Attachment: Bug 1670675 - Add test for loading brotli encoded files over unsecured HTTP r=#necko

Comment 3

[Valentin Gosu [:valentin] (he/him)]

(In reply to Jukka Jylänki from comment #0)

3. Please add a boolean pref support.brotli.over.insecure.http so that developers can use Firefox to develop Brotli-enabled pages. Needing to do HTTPS just to be able to locally develop is an insane ask for developers. Paired with (1.) above that would make Brotli development less painful.

You can use the network.http.accept-encoding pref and set it to gzip, deflate, br
I'm submitting a patch to make sure the pref works well, and to document this better for future uses.
Thanks!

Comment 4

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/fe1cffc8ac28
Add test for loading brotli encoded files over unsecured HTTP r=necko-reviewers,kershaw

Comment 5

[Andreea Pavel [:apavel]]

https://hg.mozilla.org/mozilla-central/rev/fe1cffc8ac28