915721 - Gecko multi-locale builds should contain only b2g and select toolkit strings

Status: RESOLVED FIXED

(Firefox OS Graveyard :: General, defect)

Description

[Axel Hecht [:Pike]]

Attachment: add chrome overrides, and don't build toolkit l10n as part of the chrome-% target

+++ This bug was initially created as a clone of Bug #792077 +++

To include more localizations in our b2g multi-locale builds, we want to get rid of a bunch of toolkit cruft.

I intend to just copy over the file list we have on Fennec, knowing that it's a superset of what we need. Given how long it took us to figure out what we need on fennec, I'm taking good rather than perfect. We'll save tons of space already.

I'm attaching the simple patch, didn't try to do builds with that yet.

Comment 1

[Axel Hecht [:Pike]]

Attachment: Screenshot showing localized security error

Took me a bit to get a desktop build to work, but I got it to work. Here's a screenshot.

Comment 2

[Axel Hecht [:Pike]]

Comment on attachment 803765 [details] [diff] [review]
add chrome overrides, and don't build toolkit l10n as part of the chrome-% target

Review of attachment 803765 [details] [diff] [review]:
-----------------------------------------------------------------

Brad suggested that Wes might be a good reviewer of this. This is mostly just a port of what we do on Android, and to include the pages we mostly need for the browser app.

Comment 3

[Axel Hecht [:Pike]]

https://hg.mozilla.org/integration/mozilla-inbound/rev/42242953371b

Comment 4

[Dave Hylands [:dhylands]]

Hmm. I did an unagi build against mozilla-inbound

I started with the checkout before the one in this patch:

hg update -r ce65d48e182e

hg summary reported:
parent: 147691:ce65d48e182e 
 Bug 850364 - Implement setRangeText for HTMLInputElement. r=ehsan
branch: default
commit: 48 unknown (clean)
update: 69 new changesets (update)
mq:     19 unapplied

After building:
-rw-r--r--  1 dhylands dhylands 160501684 Sep 18 14:06 out/target/product/unagi/system.img
-rw-r--r--  1 dhylands dhylands   4980488 Sep 18 14:06 omni.ja

I then did:

hg update -r 42242953371b

hg summary reported:
parent: 147692:42242953371b 
 bug 915721, package limited set of localized files for b2g, r=wesj
branch: default
commit: 48 unknown (clean)
update: 68 new changesets (update)
mq:     19 unapplied

and after the build:
-rw-r--r--  1 dhylands dhylands 160526260 Sep 18 14:08 out/target/product/unagi/system.img
-rw-r--r--  1 dhylands dhylands   5002405 Sep 18 14:08 omni.ja

so after the patch, the size of omni.ja increased. I went back and did both builds again just to make sure and got the exact same results.

Comment 5

[Axel Hecht [:Pike]]

Have you been building the multi-locale gecko, and/or multi-locale gaia?

I can see how a single-locale gecko would get a slight bump in size, but the multi-locale ones should surely drop in size.

Running chrome-es inside b2g/locales would be part of the multi-locale gecko, or some mozharness equivalent of that.

Comment 6

[Dave Hylands [:dhylands]]

(In reply to Axel Hecht [:Pike] from comment #5)
> Have you been building the multi-locale gecko, and/or multi-locale gaia?
> 
> I can see how a single-locale gecko would get a slight bump in size, but the
> multi-locale ones should surely drop in size.
> 
> Running chrome-es inside b2g/locales would be part of the multi-locale
> gecko, or some mozharness equivalent of that.

I honestly don't know. I'm just building whatever the default is when I get a developer tree.

How do you enable multilocale builds?

Comment 7

[Axel Hecht [:Pike]]

If you did the multi-locale gecko, you'd know ;-). https://developer.mozilla.org/en-US/docs/Mozilla/Firefox_OS/Building#Gecko has docs on it. I confess, I'm using other entry points, so I can't say how word-by-word those docs are.

Comment 8

[Wes Kocher (:KWierso) (Not reading bugmail; email directly if needed)]

https://hg.mozilla.org/mozilla-central/rev/42242953371b

Comment 9

[Dave Hylands [:dhylands]]

OK - after building a multilocale gecko build I see the following changes:

before:
-rw-r--r--  1 dhylands dhylands 161451956 Sep 18 18:49 out/target/product/unagi/system.img
-rw-r--r--  1 dhylands dhylands   7491895 Sep 18 18:49 omni.ja

after:
-rw-r--r--  1 dhylands dhylands 159477684 Sep 18 18:27 out/target/product/unagi/system.img
-rw-r--r--  1 dhylands dhylands   5521151 Sep 18 18:27 omni.ja

So a savings of roughly 1.9 Mb in the size of the omni.ja file (and the system.img file).

My multilocale build had the following languages included:
  ar ca cs de el es-ES fr hr hu nl pt-BR ro ru sk sr tr zh-TW

If I'm understanding things properly, the savings should be roughly the same regardless of which languages were included as long as it was a multilocale build right?

Comment 10

[Axel Hecht [:Pike]]

Thanks for getting those numbers, feels good to be on the winning side.

The impact per locale should be more-or-less constant, word-lengths and encodings aside. The only caveat on the numbers is that the gecko locales are currently just three, es-ES, pl, pt-BR. There's some incompat to gaia locales due to the size impact of it.

Would you be able to get us numbers on how much the image size increases post-patch per gecko language?

Comment 11

[Dave Hylands [:dhylands]]

(In reply to Axel Hecht [:Pike] from comment #10)
> Thanks for getting those numbers, feels good to be on the winning side.
> 
> The impact per locale should be more-or-less constant, word-lengths and
> encodings aside. The only caveat on the numbers is that the gecko locales
> are currently just three, es-ES, pl, pt-BR. There's some incompat to gaia
> locales due to the size impact of it.
> 
> Would you be able to get us numbers on how much the image size increases
> post-patch per gecko language?

Just to clarify, you'd like before/after sizes with MOZ_CHROME_MULTILOCALE set to each of es-ES, pl, and pt-BR ?

If so, then that should be fairly easy to do.

Comment 12

[Dave Hylands [:dhylands]]

export MOZ_CHROME_MULTILOCALE='es-ES'

before:
-rw-r--r-- 1 dhylands dhylands   5116905 Sep 19 10:54 out/target/product/unagi/system/b2g/omni.ja
-rw-r--r-- 1 dhylands dhylands 159076276 Sep 19 10:55 out/target/product/unagi/system.img

after:
-rw-r--r-- 1 dhylands dhylands   5030427 Sep 19 10:57 out/target/product/unagi/system/b2g/omni.ja
-rw-r--r-- 1 dhylands dhylands 158990260 Sep 19 10:57 out/target/product/unagi/system.img

delta: about 85 Kb

export MOZ_CHROME_MULTILOCALE='pl'

before:
ls -l out/target/product/unagi/system.img out/target/product/unagi/system/b2g/omni.ja
-rw-r--r-- 1 dhylands dhylands   5126075 Sep 19 11:02 out/target/product/unagi/system/b2g/omni.ja
-rw-r--r-- 1 dhylands dhylands 159084468 Sep 19 11:03 out/target/product/unagi/system.img

after:
-rw-r--r-- 1 dhylands dhylands   5032554 Sep 19 11:12 out/target/product/unagi/system/b2g/omni.ja
-rw-r--r-- 1 dhylands dhylands 158990260 Sep 19 11:12 out/target/product/unagi/system.img

delta: about 91 Kb

export MOZ_CHROME_MULTILOCALE='pt-BR'

before:
-rw-r--r-- 1 dhylands dhylands   5125911 Sep 19 11:16 out/target/product/unagi/system/b2g/omni.ja
-rw-r--r-- 1 dhylands dhylands 159084468 Sep 19 11:16 out/target/product/unagi/system.img

after:
-rw-r--r-- 1 dhylands dhylands   5032798 Sep 19 13:25 out/target/product/unagi/system/b2g/omni.ja
-rw-r--r-- 1 dhylands dhylands 158990260 Sep 19 13:25 out/target/product/unagi/system.img

delta: about 91 Kb

Comment 13

[Preeti Raghunath(:Preeti)]

Axel

Please comment on user experience.

Comment 14

[Axel Hecht [:Pike]]

Attachment: 2013-10-09-19-50-46.png

We need the gecko localization for network error messages, see for example this security dialog of a current build for the German localization.

Similar for other network error messages, see the dependency tree of https://bugzilla.mozilla.org/showdependencytree.cgi?id=893802&hide_resolved=1.

Comment 15

[Ryan VanderMeulen [:RyanVM]]

https://hg.mozilla.org/releases/mozilla-aurora/rev/ed4d1b23e356