Closed Bug 1464156 Opened 4 years ago Closed 27 days ago

Fluent shouldn't automatically fall back to en-US when some files are missing for a context

Categories

(Core :: Internationalization: Localization, enhancement, P3)

enhancement

Tracking

()

RESOLVED DUPLICATE of bug 1743659

People

(Reporter: flod, Unassigned)

References

(Blocks 1 open bug, )

Details

Attachments

(5 obsolete files)

In case of Preferences in 62, we're including over 10 files. If one of them is missing, we're automatically falling back to English (en-US), risking to drop hundreds of localized strings in the process.

We should try to display as much localized content as possible, as long as the performance hit is acceptable.
Priority: -- → P2
For the record, I just got it by this bug: we added a new file in bug 1497694, and now preferences for Italian are falling back entirely to English (9 missing vs several hundreds translated). I was going to file a new bug, then remembered about this one.

I'm trying to wrap my head around what we actually want the l10n registry to do. Pathological example:

foo.ftl bar.ftl baz.ftl qux.ftl
distro X X
app X ?
platform X X X

baz.ftl may or may not exist, for the sake of understanding the expected behaviour, let's start with "it exists".

The first bundle is easy, I think:

(distro, foo), (platform, bar), (app, baz), (distro, qux)

Second and third get a little less clear to me. And to prove my point, let's assume that either foo or qux would be a customized branding file. You want all messages in the second bundle to still refer to the customized branding, right?

OTH, if those were all just regular files without message or term references, there would be no reason to generate another bundle with either foo of qux from distro.

I don't think we can optimize for the latter reliably?

So then the second would be

(app, foo), (platform, bar), (app, baz), (distro, qux)

And then the third could be either of

(app, foo), (platform, bar), (app, baz), (platform, qux)
(platform, foo), (platform, bar), (app, baz), (distro, qux)

I should really mock this as a test case and see what the current code does. But I'd also like to know what we want the code to do.

Attached file Bug 1464156, more extensive tests (obsolete) —

Depends on D30024

Depends on D30026

Attachment #9062916 - Attachment description: Bug 1464156, use cached iterators in bundle generation → Bug 1464156, refactor bundle generation

The code path is a bit trickier for async, as that returns promises that
will resolve to false for missing resources.
The complexity here balances with the intent to have parallel IO.
For sync IO, we can rely on fetchFile return false on missing resources,
so we don't need to retry on just existing resources.

Depends on D30028

Attachment #9062915 - Attachment is obsolete: true
Duplicate of this bug: 1561016

Depends on D30024

Attachment #9062913 - Attachment is obsolete: true
Attachment #9062916 - Attachment is obsolete: true
Attachment #9072214 - Attachment is obsolete: true
Attachment #9076072 - Attachment is obsolete: true

I've done a couple of attempts here, but I have to fold this. At the point of trying to integrate changes from the registry into the localization classes, I realized that I have no idea what is important and what's not, and I can't start the complete perf testing from scratch.

See Also: → 1586984
See Also: → 1627860

We're getting this more and more, as we migrate content on startup, and me jumping to run migrations simply doesn't scale, also considering the lack of replacement in case I'm not available.

What are the chances to actually scope and prioritize this?

With bug 1660392 stabilizing soon, I would like to start discussing possible paths forward for this.

I filed an issue against l10nregistry-rs to discuss possible avenues - https://github.com/zbraniecki/l10nregistry-rs/issues/8

Blocks: 1660392
No longer blocks: 1660392
Depends on: 1660392

The original idea for this bug has been to address it by heuristics (see https://github.com/zbraniecki/l10nregistry-rs/issues/8).

But recently introduced by Erik optional resource (bug 1743659) concept should address the most common source of this problem, which is a recently added resource.
I suggest we use optional resource for new resources added to large contexts (such as Preferences) for a couple of releases, until all locales carry it, and then switch the optional to false.

There's still a potential value in introducing heuristics for deciding when a bundle should be produced, but using memoization and algorithms seems more nuanced and I'd suggest to wait a bit to see how optional resource + process solves the issue.

If the problem prevails, I suggest to reopen this bug.

Status: NEW → RESOLVED
Closed: 27 days ago
Resolution: --- → DUPLICATE
Duplicate of bug: 1743659

(In reply to Zibi Braniecki [:zbraniecki][:gandalf] from comment #12)

I suggest we use optional resource for new resources added to large contexts (such as Preferences) for a couple of releases, until all locales carry it, and then switch the optional to false.

I don't think this process scales: it relies on developers's discretion to mark resources as optional, and requires work to clean up afterwards. And even cleaning up is not trivial: when do you stop marking a resource as optional? Some locales might not catch up for multiple releases, and their product still be almost fully localized.

BTW, I'm out of office, and I'm quite sure that preferences in Nightly are currently in English because of bug 1744466.

I don't think this process scales:

The one proposed in this bug is based on heuristics. Heuristic approach is likely to be more discrete, less explicit, giving nobody control over the experience and likely resulting in nuanced differences of experience continuously trying to balance between covering one edge scenario and another.

optional allows code maintainers, and l10n-drivers decide which resources are allowed to be missing, which has the benefit of minimizing surprises. I'd appreciate if we at least tried to use it before we dive into attempting a significantly more complex and complicated solutions that result in wider range of user experiences and potential regressions.

If you see another solution to the problem space, please share.

(In reply to Zibi Braniecki [:zbraniecki][:gandalf] from comment #14)

I'd appreciate if we at least tried to use it before we dive into attempting a significantly more complex and complicated solutions that result in wider range of user experiences and potential regressions.

I don't mind this approach, but I don't believe that's reflected in closing this bug as duplicate of bug 1743659.

This bug would be fixed by an automated solution, it's now become lower priority, and it will be either fixed or wontfixed.

Ah, makes sense. Ok! Reopening.

The target for optional imho should be first:

  • New files added to Preferences
  • Any emergency files added to ESR/late-Beta etc.
Component: Internationalization → Internationalization: Localization
Priority: P2 → P3
You need to log in before you can comment on or make changes to this bug.