1643810 - Examine using higher compression level for update generation

Status: RESOLVED FIXED

(Release Engineering :: Release Automation: Updates, enhancement)

Description

[Chris AtLee [:catlee]]

We're currently not specifying a compression level for xz for complete or partial updates. The default level is -6.

I've experimented with some updates from 77.0.1, and it looks like -7e is a good level, especially combined with disabling the x86 BCJ filter. This gives us a ~6% reduction in partial size.

According to the man page, the amount of memory required for decompressing -7e is 17 MB, vs 9MB for -6.

Here are the file sizes for some of the combinations I've tested:

11724543 firefox-76.0.1-77.0.1.partial.mar-T1-7e-nobcj
11727127 firefox-76.0.1-77.0.1.partial.mar-T1-6e-nobcj
12005603 firefox-76.0.1-77.0.1.partial.mar-T1-8e
12012531 firefox-76.0.1-77.0.1.partial.mar-T1-7e
12036267 firefox-76.0.1-77.0.1.partial.mar-T1-6e
12040147 firefox-76.0.1-77.0.1.partial.mar-T1-9e
12193523 firefox-76.0.1-77.0.1.partial.mar-nobcj
12483095 firefox-76.0.1-77.0.1.partial.mar-plain
12483095 firefox-76.0.1-77.0.1.partial.mar-T1
12483095 firefox-76.0.1-77.0.1.partial.mar-T1-6
12483641 firefox-76.0.1-77.0.1.partial.mar  # the original
12492119 firefox-76.0.1-77.0.1.partial.mar-T0
12499743 firefox-76.0.1-77.0.1.partial.mar-T1-7
12500639 firefox-76.0.1-77.0.1.partial.mar-T1-9
12516203 firefox-76.0.1-77.0.1.partial.mar-T1-8

I didn't sign these, but that only adds a few hundred bytes to the result. Compare the original file to the plain/T1/T1-6 variants to see the difference signing makes.

Comment 1

[Chris AtLee [:catlee]]

For complete updates, we have these results:

4759217 firefox-77.0.1.complete.mar-T1-9e
54820253 firefox-77.0.1.complete.mar-T1-9
54892749 firefox-77.0.1.complete.mar-T1-8e
54950313 firefox-77.0.1.complete.mar-T1-8
55142857 firefox-77.0.1.complete.mar-T1-7e
55207069 firefox-77.0.1.complete.mar-T1-7
55532917 firefox-77.0.1.complete.mar-T1-6e
55580665 firefox-77.0.1.complete.mar-plain
55580665 firefox-77.0.1.complete.mar-T1
55580665 firefox-77.0.1.complete.mar-T1-6
55762485 firefox-77.0.1.complete.mar-T0
55762961 firefox-77.0.1.complete.mar
56044473 firefox-77.0.1.complete.mar-T1-7e-nobcj
56428173 firefox-77.0.1.complete.mar-T1-6e-nobcj
56451077 firefox-77.0.1.complete.mar-nobcj

The difference isn't as noticeable here. If we force single threaded compression and use -7e, then we can reduce our complete update size by about 1%.

Comment 2

[Chris AtLee [:catlee]]

Attachment: Bug 1643810: Use -T1 -7e for update generation r=sfraser

Comment 3

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/a6db77be7818
Use -T1 -7e for update generation r=sfraser

Comment 4

[Alexandru Michis [:malexandru]]

https://hg.mozilla.org/mozilla-central/rev/a6db77be7818