1231379 - Disable compression in omni.ja for Firefox

Status: RESOLVED WONTFIX

(Firefox Build System :: General, defect)

Description

[Chris AtLee [:catlee]]

Disabling compression for omni.ja in Firefox (desktop) decreases installer and update size significantly. Users will get faster download and update times.

I'd like to try disabling this on Nightly to verify that there are no startup performance regressions.

Comment 1

[Chris AtLee [:catlee]]

Attachment: disable omni.ja compression for desktop

Comment 2

[Chris AtLee [:catlee]]

some background on why I think this will be a win...

I've been looking for ways to shrink Firefox updates. Smaller updates mean faster downloads for our users, better update rates, and also smaller bills for our CDNs. One approach I identified a few years ago [1] is to disable the zip compression used in our omni.ja files [2].

The data I've collected so far seems to indicate we would see startup performance wins (2-3%) in the case of warm caches.

The installer and update size improvements are more clear:
- Windows complete updates shrink by ~9%
- Windows partial updates shrink by ~40% on release, ~10% on nightly
- Windows full installers shrink by ~15%

The reason installer and update sizes shrink that they're generally using more efficient compression than zip is. So using 7zip to compress all of a decompressed omni.ja is smaller than using 7zip to compress a compressed omni.ja.

Partial updates also can be much smaller because it's more efficient to do binary diffs between uncompressed files than between compressed files.

I've been digging around in some of the bugs about optimized omni.ja and readahead. optimized omni.ja's were landed in http://hg.mozilla.org/mozilla-central/rev/dde9bc692553 (2010-08-18), and and had some fixes later in http://hg.mozilla.org/mozilla-central/rev/87ff6a0cc45e (2012-07-12)

Taras enabled readhead for omni.ja quite a bit later in https://hg.mozilla.org/mozilla-central/rev/f8e71f41c30f (2013-03-21), and it got some fixes in https://hg.mozilla.org/mozilla-central/rev/d7fa77615273 (2013-05-21).

Could it be that the readahead support that landed in 2013 makes the IO costs for loading more data relatively cheap?

Some other relevant bugs:
https://bugzilla.mozilla.org/show_bug.cgi?id=559961
https://bugzilla.mozilla.org/show_bug.cgi?id=711811
https://bugzilla.mozilla.org/show_bug.cgi?id=772841
https://bugzilla.mozilla.org/show_bug.cgi?id=810151
https://bugzilla.mozilla.org/show_bug.cgi?id=873640

[1] http://atlee.ca/blog/posts/firefox-update-sizes.html
[2] https://bugzilla.mozilla.org/show_bug.cgi?id=772868

Comment 3

[Joel Maher ( :jmaher ) (UTC -8)]

looking forward to this landing, I will look for talos changes!

Comment 4

[Mike Hommey [:glandium]]

Comment on attachment 8697000 [details] [diff] [review]
disable omni.ja compression for desktop

Review of attachment 8697000 [details] [diff] [review]:
-----------------------------------------------------------------

::: toolkit/mozapps/installer/upload-files.mk
@@ +681,5 @@
>  
>  ifneq (android,$(MOZ_WIDGET_TOOLKIT))
>  OPTIMIZEJARS = 1
> +ifneq (gonk,$(MOZ_WIDGET_TOOLKIT))
> +DISABLE_JAR_COMPRESSION = 1

Please add a ifdef NIGHTLY_BUILD

Comment 5

[Chris AtLee [:catlee]]

Attachment: MozReview Request: Bug 1231379 - Disable omni.ja compression on desktop platforms r=glandium

Bug 1231379: Disable omni.ja compression on desktop platforms

Comment 6

[Chris AtLee [:catlee]]

Comment on attachment 8698415 [details]
MozReview Request: Bug 1231379 - Disable omni.ja compression on desktop platforms r=glandium

Review request updated; see interdiff: https://reviewboard.mozilla.org/r/27985/diff/1-2/

Comment 7

[Chris AtLee [:catlee]]

Comment on attachment 8698415 [details]
MozReview Request: Bug 1231379 - Disable omni.ja compression on desktop platforms r=glandium

fixed typo; carried over r+ from previous patch

Comment 8

[Chris AtLee [:catlee]]

https://reviewboard.mozilla.org/r/27985/#review25087

Comment 9

[Chris AtLee [:catlee]]

Comment on attachment 8698415 [details]
MozReview Request: Bug 1231379 - Disable omni.ja compression on desktop platforms r=glandium

Review request updated; see interdiff: https://reviewboard.mozilla.org/r/27985/diff/2-3/

Comment 10

[Mike Hommey [:glandium]]

https://reviewboard.mozilla.org/r/27985/#review25153

Comment 11

[Mike Hommey [:glandium]]

Comment on attachment 8698415 [details]
MozReview Request: Bug 1231379 - Disable omni.ja compression on desktop platforms r=glandium

Damn mozreview.

Comment 12

[Pulsebot]

https://hg.mozilla.org/integration/mozilla-inbound/rev/3e08a6a2299b

Comment 13

[Carsten Book [:Tomcat]]

https://hg.mozilla.org/mozilla-central/rev/3e08a6a2299b

Comment 14

[Mark Finkle (:mfinkle) (use needinfo?)]

Chris - Does this also result in a smaller APK for Android? If not, we may only want this patch for Desktop.

Comment 15

[Mark Finkle (:mfinkle) (use needinfo?)]

Oops! I see it's Desktop-only.

Comment 16

[Chris AtLee [:catlee]]

Did you want it for Android as well? I'm not sure if it would result in a smaller APK or not...but my understanding was that this would have a higher perf impact on Android due to how APK loading works, so I made it explicitly desktop only.

Comment 17

[Mike Hommey [:glandium]]

The files in the omni.ja are compressed. The omni.ja itself is uncompressed in the APK. This allows to read(/uncompress) individual files from the omni.ja without uncompressing the entire omni.ja.

Not compressing the files in the omni.ja means the APK *will* be bigger (much bigger).
Not compressing the files in the omni.ja and then compressing the omni.ja in the APK means the APK would possibly be smaller, but that reading files in the omni.ja will add noticeable latency the first time and suck more memory.

Comment 18

[Mark Finkle (:mfinkle) (use needinfo?)]

What Mike said. I didn't want to mess with it on Android unless we had a specific plan. Let's not mess with it for now. We did a bunch of work in the past, which likely is still the best approach for now.

Comment 19

[Mayur Patil]

[bugday-20160323]

Status: RESOLVED,FIXED -> UNVERIFIED

Comments:
STR: Not clear.
Developer specific testing

Component: 
Name			Firefox
Version			46.0b9
Build ID		20160322075646
Update Channel	        beta
User Agent		Mozilla/5.0 (Windows NT 6.1; WOW64; rv:46.0) Gecko/20100101 Firefox/46.0
OS			Windows 7 SP1 x86_64

Expected Results: 
Developer specific testing

Actual Results: 
As expected

Comment 20

[Chris AtLee [:catlee]]

https://treeherder.mozilla.org/#/jobs?repo=try&revision=fe6092fd065f

Comment 21

[Chris AtLee [:catlee]]

This never rode the trains past nightly.

Comment 22

[Chris AtLee [:catlee]]

https://treeherder.mozilla.org/#/jobs?repo=try&revision=bc5451702ee7

Comment 23

[Chris AtLee [:catlee]]

This got re-landed in bug 1362377 in Firefox 55