Closed Bug 1231379 Opened 4 years ago Closed 3 years ago

Disable compression in omni.ja for Firefox

Categories

(Firefox Build System :: General, defect)

defect
Not set

Tracking

(Not tracked)

RESOLVED WONTFIX

People

(Reporter: catlee, Assigned: catlee)

References

(Blocks 1 open bug)

Details

Attachments

(2 files)

Disabling compression for omni.ja in Firefox (desktop) decreases installer and update size significantly. Users will get faster download and update times.

I'd like to try disabling this on Nightly to verify that there are no startup performance regressions.
Assignee: nobody → catlee
some background on why I think this will be a win...

I've been looking for ways to shrink Firefox updates. Smaller updates mean faster downloads for our users, better update rates, and also smaller bills for our CDNs. One approach I identified a few years ago [1] is to disable the zip compression used in our omni.ja files [2].

The data I've collected so far seems to indicate we would see startup performance wins (2-3%) in the case of warm caches.

The installer and update size improvements are more clear:
- Windows complete updates shrink by ~9%
- Windows partial updates shrink by ~40% on release, ~10% on nightly
- Windows full installers shrink by ~15%

The reason installer and update sizes shrink that they're generally using more efficient compression than zip is. So using 7zip to compress all of a decompressed omni.ja is smaller than using 7zip to compress a compressed omni.ja.

Partial updates also can be much smaller because it's more efficient to do binary diffs between uncompressed files than between compressed files.

I've been digging around in some of the bugs about optimized omni.ja and readahead. optimized omni.ja's were landed in http://hg.mozilla.org/mozilla-central/rev/dde9bc692553 (2010-08-18), and and had some fixes later in http://hg.mozilla.org/mozilla-central/rev/87ff6a0cc45e (2012-07-12)

Taras enabled readhead for omni.ja quite a bit later in https://hg.mozilla.org/mozilla-central/rev/f8e71f41c30f (2013-03-21), and it got some fixes in https://hg.mozilla.org/mozilla-central/rev/d7fa77615273 (2013-05-21).

Could it be that the readahead support that landed in 2013 makes the IO costs for loading more data relatively cheap?

Some other relevant bugs:
https://bugzilla.mozilla.org/show_bug.cgi?id=559961
https://bugzilla.mozilla.org/show_bug.cgi?id=711811
https://bugzilla.mozilla.org/show_bug.cgi?id=772841
https://bugzilla.mozilla.org/show_bug.cgi?id=810151
https://bugzilla.mozilla.org/show_bug.cgi?id=873640

[1] http://atlee.ca/blog/posts/firefox-update-sizes.html
[2] https://bugzilla.mozilla.org/show_bug.cgi?id=772868
looking forward to this landing, I will look for talos changes!
Attachment #8697000 - Flags: review?(mh+mozilla)
Comment on attachment 8697000 [details] [diff] [review]
disable omni.ja compression for desktop

Review of attachment 8697000 [details] [diff] [review]:
-----------------------------------------------------------------

::: toolkit/mozapps/installer/upload-files.mk
@@ +681,5 @@
>  
>  ifneq (android,$(MOZ_WIDGET_TOOLKIT))
>  OPTIMIZEJARS = 1
> +ifneq (gonk,$(MOZ_WIDGET_TOOLKIT))
> +DISABLE_JAR_COMPRESSION = 1

Please add a ifdef NIGHTLY_BUILD
Attachment #8697000 - Flags: review?(mh+mozilla) → review+
Bug 1231379: Disable omni.ja compression on desktop platforms
Comment on attachment 8698415 [details]
MozReview Request: Bug 1231379 - Disable omni.ja compression on desktop platforms r=glandium

Review request updated; see interdiff: https://reviewboard.mozilla.org/r/27985/diff/1-2/
Attachment #8698415 - Attachment description: MozReview Request: Bug 1231379: Disable omni.ja compression on desktop platforms → MozReview Request: date: Fri Oct 16 09:55:58 2015 -0400
Comment on attachment 8698415 [details]
MozReview Request: Bug 1231379 - Disable omni.ja compression on desktop platforms r=glandium

fixed typo; carried over r+ from previous patch
Attachment #8698415 - Flags: review+
Comment on attachment 8698415 [details]
MozReview Request: Bug 1231379 - Disable omni.ja compression on desktop platforms r=glandium

Review request updated; see interdiff: https://reviewboard.mozilla.org/r/27985/diff/2-3/
Attachment #8698415 - Attachment description: MozReview Request: date: Fri Oct 16 09:55:58 2015 -0400 → MozReview Request: Bug 1231379 - Disable omni.ja compression on desktop platforms r=glandium
Attachment #8698415 - Flags: review?(mh+mozilla)
Comment on attachment 8698415 [details]
MozReview Request: Bug 1231379 - Disable omni.ja compression on desktop platforms r=glandium

Damn mozreview.
Attachment #8698415 - Flags: review?(mh+mozilla) → review+
Depends on: 1233214
https://hg.mozilla.org/mozilla-central/rev/3e08a6a2299b
Status: NEW → RESOLVED
Closed: 4 years ago
Resolution: --- → FIXED
Target Milestone: --- → Firefox 46
Chris - Does this also result in a smaller APK for Android? If not, we may only want this patch for Desktop.
Flags: needinfo?(catlee)
Oops! I see it's Desktop-only.
Flags: needinfo?(catlee)
Did you want it for Android as well? I'm not sure if it would result in a smaller APK or not...but my understanding was that this would have a higher perf impact on Android due to how APK loading works, so I made it explicitly desktop only.
Flags: needinfo?(mark.finkle)
The files in the omni.ja are compressed. The omni.ja itself is uncompressed in the APK. This allows to read(/uncompress) individual files from the omni.ja without uncompressing the entire omni.ja.

Not compressing the files in the omni.ja means the APK *will* be bigger (much bigger).
Not compressing the files in the omni.ja and then compressing the omni.ja in the APK means the APK would possibly be smaller, but that reading files in the omni.ja will add noticeable latency the first time and suck more memory.
What Mike said. I didn't want to mess with it on Android unless we had a specific plan. Let's not mess with it for now. We did a bunch of work in the past, which likely is still the best approach for now.
Flags: needinfo?(mark.finkle)
[bugday-20160323]

Status: RESOLVED,FIXED -> UNVERIFIED

Comments:
STR: Not clear.
Developer specific testing

Component: 
Name			Firefox
Version			46.0b9
Build ID		20160322075646
Update Channel	        beta
User Agent		Mozilla/5.0 (Windows NT 6.1; WOW64; rv:46.0) Gecko/20100101 Firefox/46.0
OS			Windows 7 SP1 x86_64

Expected Results: 
Developer specific testing

Actual Results: 
As expected
Blocks: 1275998
Blocks: 1303172
This never rode the trains past nightly.
Status: RESOLVED → REOPENED
Resolution: FIXED → ---
Target Milestone: Firefox 46 → ---
Depends on: 1340157
Status: REOPENED → RESOLVED
Closed: 4 years ago3 years ago
Resolution: --- → WONTFIX
This got re-landed in bug 1362377 in Firefox 55
Component: Build Config → General
Product: Firefox → Firefox Build System
You need to log in before you can comment on or make changes to this bug.