987582 - compress many of the system fonts to reduce ROM size

Status: RESOLVED FIXED

(Firefox OS Graveyard :: GonkIntegration, defect)

Description

[Jonathan Kew [:jfkthame]]

With the Gecko support from bug 987357, we can consider installing fonts in compressed (.woff) format instead of raw .ttf or .otf files. This could save us several megabytes in total.

We'll need to be alert for possible performance impact due to the need to decompress the fonts into RAM on first access. For this reason, I propose to initially try compressing a bunch of the fonts that are likely to be less-used (the serif family CharisSIL, and many non-Latin fonts).

Comment 1

[Jonathan Kew [:jfkthame]]

Attachment: [PR] compress a bunch of lesser-used fonts

This compresses many of the fonts from moztt. I omitted Fira for now, because (a) the .otf files are not as compressible as most .ttfs, so there's less to be gained, and (b) these are the fonts most likely to be used right away during startup, so any perf impact might be most noticeable.

We should also consider compressing many of the fonts from frameworks/base/data/fonts in the same way (we could get big gains on Chinese and Japanese), but I'm not sure how we manage those files.

Comment 2

[Michael Wu [:mwu]]

Comment on attachment 8396275 [details] [review]
[PR] compress a bunch of lesser-used fonts

I would rather not compress fonts on devices that don't need it. How about creating a variable that lets us switch between installing the compressed and uncompressed fonts?

Comment 3

[Michael Wu [:mwu]]

(In reply to Jonathan Kew (:jfkthame) from comment #1)
> We should also consider compressing many of the fonts from
> frameworks/base/data/fonts in the same way (we could get big gains on
> Chinese and Japanese), but I'm not sure how we manage those files.

Let's just fork and use our own copy in moztt if we want to change them. I think you can just remove the fallback font from the packages list and have it install normally through the copy files list.

Comment 4

[Kai-Zhen Li [:seinlin][:kli]]

Attachment: Tarako with Bengali

I try this patch together with the patch in bug 987357 on v1.3t, but the charater can't be rendered correctly. 

The language used is Bengali and tried both external/moztt/Noto/NotoSansBengali-Regular.woff and external/moztt/Lohit/lohit-bengali-ttf-2.5.3/Lohit-Bengali.woff. 

Before, it does work correctly with external/moztt/Lohit/lohit-bengali-ttf-2.5.3/Lohit-Bengali.ttf.

Do we need to do more for useing .woff fonts?

Comment 5

[Michael Wu [:mwu]]

(In reply to Kai-Zhen Li from comment #4)
> Do we need to do more for useing .woff fonts?

Oh, I think the 1.3 branch might not have freetype 2.5.2, which is required for woff support. https://bugzilla.mozilla.org/show_bug.cgi?id=966795 added woff support.

Comment 6

[Kai-Zhen Li [:seinlin][:kli]]

(In reply to Michael Wu [:mwu] from comment #5)
> (In reply to Kai-Zhen Li from comment #4)\
> 
> Oh, I think the 1.3 branch might not have freetype 2.5.2, which is required
> for woff support. https://bugzilla.mozilla.org/show_bug.cgi?id=966795 added
> woff support.

Michael, Thanks!

Yes, in 1.3t freetype 2.5.0.2 is used. I follow bug 966795 and bug 985806 and back-port freetype 2.5.2 to v1.3t successfully, now the woff font can be rendered correctly.

Comment 7

[Jonathan Kew [:jfkthame]]

Comment on attachment 8396275 [details] [review]
[PR] compress a bunch of lesser-used fonts

Updated the PR with a flag MOZ_PRODUCT_COMPRESS_FONTS that can be set to 'true' to request that a number of the MozTT fonts are shipped in .woff format; otherwise we keep them in the original .ttf form.

Is this the kind of thing you had in mind? It appears to behave as intended: if I include
  MOZ_PRODUCT_COMPRESS_FONTS := true
in my device's main .mk file, I get the .woff files in the resulting image; and if not, I get .ttfs.

I didn't do anything with the fonts from frameworks/base/data yet, but if you think this is the right direction to be going, I'll follow up with that.

Comment 8

[Michael Wu [:mwu]]

(In reply to Jonathan Kew (:jfkthame) from comment #7)
> Is this the kind of thing you had in mind? It appears to behave as intended:
> if I include
>   MOZ_PRODUCT_COMPRESS_FONTS := true
> in my device's main .mk file, I get the .woff files in the resulting image;
> and if not, I get .ttfs.
> 

Yup, this is what I had in mind. I suppose we could try to get more sophisticated about which fonts to compressed based on what the target market is, but this is more than good enough to start with.

BTW I had asked Kai-Zhen to experiment with backporting woff support to Tarako. The v1.3t branch is pretty different. It sounds like compressing fonts doesn't give us enough of a savings, but I'm not entirely sure what set of fonts were compressed.

Comment 9

[Kai-Zhen Li [:seinlin][:kli]]

With patch from attachment 8396275 [details] [review], on tarako when MOZ_PRODUCT_COMPRESS_FONTS=true, 6538752 Bytes can be saved in ROM.

I'll do a simple test on tarako to compare the difference of memory and cpu usage between with ttf and wit woff. Update the result later.

Comment 10

[Kai-Zhen Li [:seinlin][:kli]]

I did two tests to compare the differences of memory and cpu usage between with ttf and with woff fonts.

Test result: http://goo.gl/XYZMbV (There are significant differences in Browser's memory usage and CPU time.)

Details about the test:

Test 1 - ttf (using ttf fonts)
Test 2 - woff (all ttf fonts are replaced with woff fonts)

Test 1 and 2 are with the same steps:
- reboot phone
- after boot to homescreen, open browser, search bbc bengala and open a bengali news page
- back to homescreen, scroll left and right some times
- open browser again, browse another bengali news page

The following fonts under /system/bin, .ttf or .woff
CharisSILCompact-B.ttf
CharisSILCompact-BI.ttf
CharisSILCompact-I.ttf
CharisSILCompact-R.ttf
DroidSerif-Bold.ttf
DroidSerif-BoldItalic.ttf
DroidSerif-Italic.ttf
DroidSerif-Regular.ttf
FiraMonoOT-Bold.otf
FiraMonoOT-Regular.otf
FiraSansOT-Bold.otf
FiraSansOT-BoldItalic.otf
FiraSansOT-Light.otf
FiraSansOT-LightItalic.otf
FiraSansOT-Medium.otf
FiraSansOT-MediumItalic.otf
FiraSansOT-Regular.otf
FiraSansOT-RegularItalic.otf
Lohit-Bengali.ttf

Comment 11

[Jonathan Kew [:jfkthame]]

(In reply to Michael Wu [:mwu] from comment #8)
> (In reply to Jonathan Kew (:jfkthame) from comment #7)
> > Is this the kind of thing you had in mind? It appears to behave as intended:
> > if I include
> >   MOZ_PRODUCT_COMPRESS_FONTS := true
> > in my device's main .mk file, I get the .woff files in the resulting image;
> > and if not, I get .ttfs.
> > 
> 
> Yup, this is what I had in mind. I suppose we could try to get more
> sophisticated about which fonts to compressed based on what the target
> market is, but this is more than good enough to start with.

Right - e.g. if there's going to be a Bengali-specific tarako version that includes only the basic Fira/Droid fonts plus the Bengali font, I'd suggest leaving Bengali uncompressed in that build.

I've squashed the PR to a single commit for merging; marking checkin-needed.

We can follow up with a second bug to bring additional fonts such as Japanese from frameworks/base into moztt and handle them the same way.

> BTW I had asked Kai-Zhen to experiment with backporting woff support to
> Tarako. The v1.3t branch is pretty different. It sounds like compressing
> fonts doesn't give us enough of a savings, but I'm not entirely sure what
> set of fonts were compressed.

Comment 9 says that over 6MB was saved, which seems pretty significant to me, but I have no idea what the actual target or constraints are. I think that must have been an "all fonts included" build, as I'd only expect savings of around 4MB for the cut-down list of fonts in comment 10.

Comment 12

[Ryan VanderMeulen [:RyanVM]]

Master: https://github.com/mozilla-b2g/moztt/commit/5b93c7150acac5f657675b91889d828cc2b532e3

Comment 13

[Jonathan Kew [:jfkthame]]

Attachment: [PR] followup to compress fonts we're using from Android

I have not yet done a test build using this, so handle with care. :)

Comment 14

[Jonathan Kew [:jfkthame]]

Re-opening this bug to finish the job by compressing the Android fonts as well; see above.

Comment 15

[Michael Wu [:mwu]]

Comment on attachment 8398651 [details] [review]
[PR] followup to compress fonts we're using from Android

Looks good.

Since we're going to get rid of Roboto Real Soon Now, I guess there's no point in making it work. (though I'll have to make it work for Tarako)

Comment 16

[Michael Wu [:mwu]]

BTW it looks like removing DroidSansFallback from the product packages list also removes the DroidSansEthiopic font on ICS. On JB, DroidSansEthiopic and DroidSansTamil are removed.

Comment 17

[Jonathan Kew [:jfkthame]]

Huh, interesting. Well, we shouldn't really need DroidSansTamil, as we have Lohit (though this may affect which font people actually see when fallback kicks in), but I think we should restore DroidSansEthiopic - unless we decide to substitute a different Ethiopic font at some point, but that would be a separate decision.

Comment 18

[Michael Wu [:mwu]]

Attachment: Link to Github pull-request: https://github.com/mozilla-b2g/moztt/pull/36

Here's the v1.3t backport. Switching to compressed fonts saves about 11mb.

Comment 19

[Michael Wu [:mwu]]

Requesting blocking for 11mb system image savings on tarako. See also bug 987357 comment 7 .

Comment 20

[Michael Wu [:mwu]]

https://github.com/mozilla-b2g/moztt/commit/89895cfeb5df68dd30841dfe5b25aecf5d480bb1

Comment 21

[Jonathan Kew [:jfkthame]]

I've updated the PR to restore DroidSansEthiopic to the build (compressed, if that option is enabled). Re-flagging for r? with that change.

Comment 22

[Andreas Gal :gal]

11MB saved is nuts. Both of you deserve a medal.

Comment 23

[Jonathan Kew [:jfkthame]]

Re-setting checkin-needed for the second PR (https://github.com/mozilla-b2g/moztt/pull/35) to be merged on trunk.

Comment 24

[David Baron :dbaron: (⌚️UTC-4, no longer working on Mozilla)]

Was there ever a conclusion on whether this means we're using malloc where we were using mmap before (bug 987357 comment 2)?  Does that affect our ability to handle low-memory situations (in that we could page out an mmap, but not a malloc)?

Comment 25

[Ryan VanderMeulen [:RyanVM]]

(In reply to Jonathan Kew (:jfkthame) from comment #13)
> Created attachment 8398651 [details] [review]
> [PR] followup to compress fonts we're using from Android

Master: https://github.com/mozilla-b2g/moztt/commit/a425b5eaf4949ab17a1670442fc2b17a82c84a74

Comment 26

[[:fabrice] Fabrice Desré]

(In reply to Ryan VanderMeulen [:RyanVM UTC-4] from comment #25)
> (In reply to Jonathan Kew (:jfkthame) from comment #13)
> > Created attachment 8398651 [details] [review]
> > [PR] followup to compress fonts we're using from Android
> 
> Master:
> https://github.com/mozilla-b2g/moztt/commit/
> a425b5eaf4949ab17a1670442fc2b17a82c84a74

Ryan, do you mind uplifting that one to 1.3T ?

Comment 27

[Kai-Zhen Li [:seinlin][:kli]]

Fabrice, I thin it is already in 1.3T.
https://github.com/mozilla-b2g/moztt/commit/228cc186b31ef99df5a015fff1b67f4392463f56

Comment 28

[Ryan VanderMeulen [:RyanVM]]

Doesn't cherry-pick cleanly, so I assume Kai-Zhen is correct. Otherwise, someone will need to resolve whatever conflicts there are.

Comment 29

[[:fabrice] Fabrice Desré]

(In reply to Kai-Zhen Li from comment #27)
> Fabrice, I thin it is already in 1.3T.
> https://github.com/mozilla-b2g/moztt/commit/
> 228cc186b31ef99df5a015fff1b67f4392463f56

ok. Then please make sure to flag the bug correctly...