Closed Bug 1921583 Opened 1 year ago Closed 2 months ago

Implement brotli for Compression/DecompressionStream

Categories

(Core :: DOM: Streams, enhancement, P3)

Product:
Core
Component:
DOM: Streams
Type:
enhancement

Tracking

()

Status:
RESOLVED FIXED
Milestone:
147 Branch
Tracking Flags:
Tracking Status
relnote-firefox --- 147+
firefox146 --- wontfix
firefox147 --- fixed

People

(Reporter: saschanaz, Assigned: saschanaz)

References

(Regressed 1 open bug)

Details

(Keywords: dev-doc-complete, parity-safari)

Attachments

(5 files)

Bug 1921583 - Part 2: Support brotli in DecompressionStream r=smaug
48 bytes, text/x-phabricator-request
Details | Review
Bug 1921583 - Part 1: Refactor decompression tests to share chunk data r=smaug
48 bytes, text/x-phabricator-request
Details | Review
Bug 1921583 - Part 3: Build brotli encoder r=jfkthame
48 bytes, text/x-phabricator-request
Details | Review
Bug 1921583 - Part 4: Support brotli in CompressionStream r=smaug
48 bytes, text/x-phabricator-request
Details | Review
Bug 1921583 - Part 5: Refactor compression tests to cover brotli r=smaug
48 bytes, text/x-phabricator-request
Details | Review

Per today's resolution in TPAC WHATWG/WebPerf joint meeting.

We should also add some telemetry to see the usage ratio (between comp/decomp for each format)

Summary: Implement brotli Compression/DecompressionStream → Implement brotli for Compression/DecompressionStream
See Also: → 1947431

:saschanaz, do you have any idea about when this feature will be available ?
I'm asking because it seems that brotli decompression will be a part of the pdf 2.0 specs:
https://pdfa.org/brotli-compression-coming-to-pdf/

Flags: needinfo?(krosylight)

Err, I was thinking maybe we can go straight to zstd, do you have some info why they chose brotli? Maybe the decision was made a few years ago already?

We got zstd support internally for the translation module and hasn't been doing much for brotli yet (which doesn't mean we have a strong reason not to ship brotli).

Flags: needinfo?(krosylight)

Hi. I'm the guy that did the investigation for the introduction of Brotli to PDF.

We considered a range of different options, (including LZMA, Brotli, ZStd), and in our testing found that Brotli was as good, if not better, than all the others at the different compression/decompression settings we tried.

The fact that a Brotli decoder will likely be included in most web environments (not least because it is required for WOFF2 fonts, I believe) is reinforcement to this decision.

Adding multiple new compression methods is not really a good idea for us, as we want PDF to continue to be readable in resource constrained environments.

(In reply to Calixte Denizet (:calixte) from comment #1)

:saschanaz, do you have any idea about when this feature will be available ?
I'm asking because it seems that brotli decompression will be a part of the pdf 2.0 specs:
https://pdfa.org/brotli-compression-coming-to-pdf/

Hi Calixte, how urgent is this for your team? Trying to adjust the priority.

Flags: needinfo?(cdenizet)

:Robin, do you know when BrotliDecode will be in the PDF 2.0 specs ?
:saschanaz, for now it isn't in the PDF specs, Acrobat doesn't implement the feature, so I'd say it isn't urgent but if you could add it to the 2025 roadmap, it'd be nice.
Anyway, on the pdf.js' side, we've some work too in order to make sure we're able to get the data asynchronously from a BrotliDecode stream.

Flags: needinfo?(cdenizet) → needinfo?(robin)

We are currently preparing the official Brotli Decode in PDF documentation. An informal document is available to members of the PDF association, but is not currently public.

Basically, if you use "BrotliDecode" in all the places that you currently read "FlateDecode" you won't go far wrong.

Development versions of MuPDF and Ghostscript both have support for the current version so files can be generated for testing, and generated files can be tested.

There is no guarantee that the spec won't change before publication (In particular, I was interested in doing a new custom dictionary for PDF compression, but this hasn't shown any meaningful benefits so far), but I suspect you'll be safe just going with the obvious.

If you're interested in participating more thoroughly with this and other developments coming down the line, you'd be well advised to consider joining the PDFA.

Flags: needinfo?(robin)

I should have said "We have no exact timeline for this as yet. It needs to go through the ISO fast-track process, but I have no personal experience on how long that will take."

Thanks all! P3 for now.

Priority: -- → P3

(In reply to Robin Watts from comment #3)

We considered a range of different options, (including LZMA, Brotli, ZStd), and in our testing found that Brotli was as good, if not better, than all the others at the different compression/decompression settings we tried.

Can you share your measurement code, and the corpus you tried it on? I find this claim surprising, but it's hard to design an experiment that allows for a fair comparison.

The fact that a Brotli decoder will likely be included in most web environments (not least because it is required for WOFF2 fonts, I believe) is reinforcement to this decision.

zstd is going to be on the Web in the coming months to years (https://github.com/whatwg/compression/issues/54).

Flags: needinfo?(robin)

(In reply to Paul Adenot (:padenot) from comment #9)

Can you share your measurement code, and the corpus you tried it on? I find this claim surprising, but it's hard to design an experiment that allows for a fair comparison.

Sure. If you'd like to be involved, then just join the PDFA. Then I can give you access to my code where I tried LZMA vs Brotli vs ZStd.

zstd is going to be on the Web in the coming months to years (https://github.com/whatwg/compression/issues/54).

But not necessarily everywhere that PDF renderers need to be implemented. It seems likely that PDF will want to support WOFF/WOFF2 fonts in some future version, so Brotli is probably going to be required within PDF implementations at some point.

It's bad enough to insist that every PDF renderer in the world needs to implement another decoder; we can't in good conscience insist that they all need to implement every decoder people might want. So we picked 'the best' one (for at least the values of 'best' that I tested).

Honestly, we'd need some pretty compelling evidence to relitigate this now. And the place to present that evidence is in the PDFA committee process. Ideally 6 months ago :) We'd love to see you there.

Whatever option we pick, someone will complain that we picked the wrong one.

Flags: needinfo?(robin)

:saschanaz, it seems that it's becoming more and more relevant to have this feature implemented in Firefox at least in the pdf.js context:
https://github.com/mozilla/pdf.js/issues/20290#:~:text=Several,possible,-%2E

Could we prioritize it ?

Flags: needinfo?(krosylight)

Ah, thanks for heads up. I'll find some time.

Assignee: nobody → krosylight
Depends on: 1994230
Flags: needinfo?(krosylight)

Out of curiosity, will pdf.js also use CompressionStream for brotli? Maybe to save PDF?

Flags: needinfo?(cdenizet)

Yes it's already possible to add some elements (images, texts, ...) which are saved and then compressed.
We are currently working on the possibility to split and merge pdfs, and having a good compression tool would be amazing.

Flags: needinfo?(cdenizet)
Attachment #9520735 - Attachment description: WIP: Bug 1921583 - Support brotli in compression streams → Bug 1921583 - Part 2: Support brotli in DecompressionStream r=smaug
Created web-platform-tests PR https://github.com/web-platform-tests/wpt/pull/55599 for changes under testing/web-platform/tests

https://hg.mozilla.org/mozilla-central/rev/3cb78154635b

Status: NEW → RESOLVED
Closed: 2 months ago
status-firefox146: --- → fixed
Resolution: --- → FIXED
Target Milestone: --- → 146 Branch
Upstream PR merged by moz-wptsync-bot

Reopened for unpublished changesets.

Status: RESOLVED → REOPENED
Resolution: FIXED → ---
Attachment #9522045 - Attachment description: WIP: Bug 1921583 - Part 4: Support brotli in CompressionStream r=smaug → Bug 1921583 - Part 4: Support brotli in CompressionStream r=smaug

Jonathan, I added you as a reviewer of brotli dependency because you were the last reviewer, but are you still the right one or should I look for someone else?

Flags: needinfo?(jfkthame)

Yeah - I'll take a look, sorry for not getting to this right away.

(Aside: One thing to be aware of when it comes to compression is that IIRC brotli can be pretty slow compared to other compressors; it's optimized for good compression-ratio and decompression performance, but compression is expensive.)

Flags: needinfo?(jfkthame)

True, but given PDF decided to use brotli we have not much choice 😔 (given we also need to edit PDF)

Created web-platform-tests PR https://github.com/web-platform-tests/wpt/pull/56058 for changes under testing/web-platform/tests

Is this worth calling out in the Fx147 relnotes? Please nominate if yes.

status-firefox146: affected → wontfix
Flags: needinfo?(krosylight)
Flags: in-testsuite+
Target Milestone: 146 Branch → 147 Branch
Upstream PR merged by moz-wptsync-bot

Release Note Request (optional, but appreciated)
[Why is this notable]: adding web platform feature for brotli compression
[Affects Firefox for Android]: Yes
[Suggested wording]: Both CompressionStream and DecompressionStream now can handle the brotli format.
[Links (documentation, blog post, etc)]:

relnote-firefox: --- → ?
Flags: needinfo?(krosylight)

Increased the installer size on Windows non-shippable opt builds by 333KB.
Probably expected from the nature of the bug (moar stuff is being compiled now).

Added to the Fx147 relnotes.

relnote-firefox: ? → 147+
Regressions: 2002244

FF147 MDN docs work for this can be tracked in https://github.com/mdn/content/issues/42251

Keywords: dev-doc-needed → dev-doc-complete
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Creator:
Created:
Updated:
Size: