673391 - Jetpack addons are big

Status: RESOLVED FIXED

(Add-on SDK Graveyard :: General, enhancement, P2)

Description

[Mike Hommey [:glandium]]

Take ttaubert's version freeze extension:
$ git clone https://github.com/ttaubert/version-freeze

The bulk of its code is:
let delegate = {
  onTrack: function (window) {
    let doc = window.document;
    let winType = doc.documentElement.getAttribute("windowtype");

    if ("Browser:About" == winType) {
      let version = doc.getElementById("version");
      version.setAttribute("value", "5.0.1");
    }
  },

  onUntrack: function () {}
};

exports.main = function () {
  let winUtils = require("window-utils");
  new winUtils.WindowTracker(delegate);
};

which is 439 bytes.

Package it:
$ cfx xpi

The resulting xpi is... 237KB.

Even with the option that removes parts of the SDK:
$ cfx xpi --strip-xpi

The resulting xpi is... 67KB.

I bet the equivalent restartless extension would be a few kilobytes.

If I strip down lib/main.js to contain the following:
exports.main = function () {
};

The resulting stripped xpi is... 46KB.

Comment 1

[Myk Melez [:myk] [@mykmelez]]

This is a top priority for us, but it's going to involve a number of discrete fixes, so we're going to track this using a feature page.  Assigning to dcm for creation of that feature page.  In the meantime, marking this as depending on the most immediate low-hanging fruit for reducing addon size (making --strip-xpi the default).

Comment 2

[Alexandre Poirot [:ochameau]]

Attachment: Strip licence header on cfx xpi

I took some time to see what is eating so many bytes:
(Numbers enclosed by parenthesis are file size, the first number is raw size, the second one is compressed size)

1/ The loader:
  /bootstrap.js (20/6.4)
  /components/harness.js (17/6.1)
  /api-utils/lib/cuddlefish.js (5/1.6)
  /api-utils/lib/securable-module.js (15/4.3)
We end up having 57/18KB dedicated to the loader.
If we take ongoing work from Irakli on loader simplification (bug 674492),
we strip it down to 17/6KB! So that the simplest "export.main = fucntion (){}" addon would be 34KB instead of 46KB.

2/ Licences header removal:
We have multiple small modules where licence headers can have the same size that the code. With quick fix proposed in this patch, I remove all these headers before shipping files in xpi. 
I'm able to save 17KB on the simplest example to end up with 29KB.
We may just remove licence header from files or simplify it and put a unique licence file.

3/ harness-options.json:
This file is pretty big but has a good compression rate 12/2KB.
We may gain some extra bytes by splitting review/meta(packages manifests) data from loader data (mandatory one for execution).
And we have many unused fields that should be only created when we start using them: docsSHA256, jsSHA256.

Comment 3

[Wes Kocher (:KWierso) (Not reading bugmail; email directly if needed)]

(Pushing all open bugs to the --- milestone for the new triage system)

Comment 4

[Gregg Lind (Fx Strategy and Insights - Shield - Heartbeat )]

Did that comment stripper ever land?  I am not sure who to find the 'new triage system'.

Comment 5

[Wes Kocher (:KWierso) (Not reading bugmail; email directly if needed)]

(In reply to Gregg Lind (User Research - Test Pilot) from comment #4)
> Did that comment stripper ever land?

I don't think so. We don't include unused modules by default anymore, but that doesn't do anything to reduce the size of the modules themselves.

The next big change will be to move the loader from within each individual addon to be a built-in part of Firefox, which will drop a bit of the size off of each addon.

After that, there's talk of moving the modules themselves into Firefox so that only the addon's own code needs to be packaged up separately, but that's not in the plans until later this year, last I checked.

(In reply to Gregg Lind (User Research - Test Pilot) from comment #4)
> I am not sure who to find the 'new
> triage system'.

The new triage system is just a change in the way we mark the priority of all of the various bugs. We don't give every bug a target milestone because we're on a rolling release. (We'll assign a milestone if something NEEDS to be in that release like a major bug fix to a broken feature.) We just use the Priority field to show what priorities we assign to the bugs.

Comment 6

[Wes Kocher (:KWierso) (Not reading bugmail; email directly if needed)]

Comment on attachment 555442 [details] [diff] [review]
Strip licence header on cfx xpi

Gerv, asking you for feedback on this patch to strip the license headers from the various SDK modules going into the final generated XPI. Can we do this?

I don't remember offhand if the SDK's LICENSE gets copied over to the generated XPI. Would we need it to if the modules don't include the license themselves?

Comment 7

[Wes Kocher (:KWierso) (Not reading bugmail; email directly if needed)]

Mossop brought up that the new MPL2 headers are significantly smaller than the previous headers, so the benefit of stripping them is probably much less than when Alex posted comment 2...

Comment 8

[Alexandre Poirot [:ochameau]]

Yes, licence is now smaller, and we are now checking md5sum (or whatever sum) of SDK modules when uploading xpi to AMO. It doesn't mean we can't do that, but we will have to tune the validator!

I think that we should do an update, like what I did in comment 2 and see what is the current state. What files justify to be still 100k+ for simple addon.

Obviously landing into mozilla-central will drop significantly the size of xpi.

Comment 9

[Gervase Markham [:gerv]]

Optimization is about measuring, changing and measuring again. 10 months ago we measured, but recently I made a change to reduce the size of the headers by 10x. Let's measure again before we make another change, as ochameau suggests :-)

Gerv

Comment 10

[Gervase Markham [:gerv]]

Having said that, stripping licence headers as part of a compilation step is valid as long as the final XPI obeys the terms of the MPL 2 in terms of providing access to the original source and a copy of the licence file. (See the MPL 2 for exact details.)

Gerv

Comment 11

[Gregg Lind (Fx Strategy and Insights - Shield - Heartbeat )]

Fwiw, an empty addon for me is 52k, most of which is the loader files.  Given the complications of stripping out all comments (regexen aren't enough, since they can embed in strings, etc.), moving the loader seems like the largest-value win :)

Thanks for reviewing this all!  

(motivating example:  I had someone complain about using Addon-sdk due to 'large addon sizes', 'memory leaks' (fixed in 1.6!), and worries about the api moving)

Comment 12

[Jeff Griffiths (:canuckistani) (:⚡︎)]

Gregg: when the loader lands in Firefox, the empty add-on will be waaaay smaller. This situation will get even better when we land addon-kit in Firefox.

Comment 13

[Jeff Griffiths (:canuckistani) (:⚡︎)]

...and the loader has landed and shipped. Closing this. If you're looking for a dramatically smaller xpi that is compatible with Firefox 21+, use the '--strip-sdk' argument for cfx.