1426451 - Debug builds should Assert if a FTL (fluent) file is loaded but only a small percentage of the strings are referenced

Status: RESOLVED WONTFIX

(Core :: Internationalization, enhancement, P3)

Description

[Jared Wein [:jaws] (please needinfo? me)]

As we are working on porting the Firefox preferences to using Fluent, we may end up with a single Fluent file with over 900 strings. We want to make sure that other parts of the Preferences UI (subdialogs) do not load this Fluent file just to reference one or two strings. In this case, shared strings should be placed in a shared FTL file that the subdialog and the larger Preferences UI could load.

To help prevent this situation, after a Fluent file is loaded, we could wait some amount of time (perhaps after a couple idle callbacks or more) while keeping track of which strings are referenced. If less than 5% of the FTL strings are referenced we could throw a debug assertion.

Comment 1

[Zibi Braniecki [:zbraniecki][:gandalf]]

I'm wondering how to approach this bug.

There will be cases where we load a file with 10 strings and use only 1-2 of them, so we would have to make a warning if we use less than 10% of strings?

Or maybe it'd only affect files with more than X strings when they're loaded and less then Y% strings are used?

:jaws, :pike, :flod - thoughts?

Comment 2

[Axel Hecht [:Pike]]

Looking at the size of the files we have right now:

     211 toolkit/locales/en-US/toolkit/intl/languageNames.ftl
     253 browser/locales/en-US/browser/pageInfo.ftl
     279 toolkit/locales/en-US/toolkit/intl/regionNames.ftl
     282 browser/locales/en-US/browser/menubar.ftl
     314 toolkit/locales/en-US/toolkit/about/aboutSupport.ftl
     481 toolkit/locales/en-US/toolkit/about/aboutAddons.ftl
    1131 browser/locales/en-US/browser/preferences/preferences.ftl

Those are line counts, string counts are somewhere half-to-third of that. The sizes might be inconsistent, most are from my local tree, but menubar isn't.

langs and regions are OK, and designed to be used in a way of getting all strings, and then use one.

The only practical problem might become menubar.ftl. But that's loaded and cached on startup. So maybe it's actually better to re-use the known copy than to make us load more files on startup?

We also seem to be pretty successful to not throw all our strings in one basket.

Which is a long way of saying, maybe we don't need to invest in tooling here?

Comment 3

[Francesco Lodolo [:flod]]

Number of messages according to Pontoon

84  browser/locales/en-US/browser/menubar.ftl
88  browser/locales/en-US/browser/pageInfo.ftl
161 toolkit/locales/en-US/toolkit/about/aboutAddons.ftl
207 toolkit/locales/en-US/toolkit/intl/languageNames.ftl
218 toolkit/locales/en-US/toolkit/about/aboutSupport.ftl
272 toolkit/locales/en-US/toolkit/intl/regionNames.ftl
356 browser/locales/en-US/browser/preferences/preferences.ftl

I think we're doing a good job in keeping things separate as we create/migrate new content, but a patch reusing an existing file would easily fly under the radar.

Only looking at percentage per file doesn't seem like a good approach (importing a file with 15 strings and using one might still be OK). Would it be possible to look at the whole bundle size, and assert if the percentage of used strings out of the bundle is under X %?

Comment 4

[Jared Wein [:jaws] (please needinfo? me)]

Yeah I agree that we might not need this anymore.

Comment 5

[Zibi Braniecki [:zbraniecki][:gandalf]]

Only looking at percentage per file doesn't seem like a good approach (importing a file with 15 strings and using one might still be OK). Would it be possible to look at the whole bundle size, and assert if the percentage of used strings out of the bundle is under X %?

Yes. The question is if it's worth the effort. If you think it is, I'm happy to leave this bug open, otherwise I'll close it.

Comment 6

[Francesco Lodolo [:flod]]

I don't think it is worth the effort at the moment.