1557713 - [fluent] Synchronous IO on content process doesn't work

Status: RESOLVED DUPLICATE of bug 1621024

(Core :: Internationalization, defect, P2)

Description

[:Gijs (he/him)]

STR:

1. open current nightly either with an attached terminal/stderr/stdout, or:
   1a. open the browser content toolbox (note: not the browser toolbox, not the regular devtools, the browser content toolbox).
2. open https://www.bbc.co.uk/sport/tennis/48555919
3. enter reader mode

ER:
no errors

AR:

[fluent] Request for keys failed because no resource bundles got generated.  keys: [{"id":"language-name-en"}].  resourceIds: ["toolkit/intl/languageNames.ftl"].

I don't really understand why because the string is right here: https://searchfox.org/mozilla-central/source/toolkit/locales/en-US/toolkit/intl/languageNames.ftl#48 .

I assume this has been broken since bug 1509583 or one of the related bugs but haven't run mozregression .

(This is triggered because Narrate asks for the locale name through Services.intl here: https://searchfox.org/mozilla-central/source/toolkit/components/narrate/NarrateControls.jsm#288 )

Comment 1

[Axel Hecht [:Pike]]

I can reproduce this, but then I get lost in the weeds of the l10n registry. Building a rebase of bug 1464156, to see what that does.

Comment 2

[Axel Hecht [:Pike]]

OK, so the problem is that in this context, the URLPreloader used by Cu.readUTF8URI isn't initialized.

Comment 3

[Axel Hecht [:Pike]]

... async works, and once I did an async, also the sync IO works. I think. Debugging this is a bit weird.

Comment 4

[Axel Hecht [:Pike]]

kmag, you mentioned on IRC to make sure that our IO should go the the preloader.

Any suggestions how to fix this bug while keeping the use of preloader?

Comment 5

[Kris Maglione [:kmag]]

The URL preloaded only works in the parent process. If we need to do sync loads in the content process for some reason, they need to happen another way.

Comment 6

[Axel Hecht [:Pike]]

Adjusting the title.

Now, I also realize why sync worked after async, the Resource from the async attempt is cached in the registry, and returned in the sync code paths afterwards. That's rather Heisenbug :-(

Comment 7

[Zibi Braniecki [:zbraniecki][:gandalf]]

If we need to do sync loads in the content process for some reason, they need to happen another way.

Can you point us at what's the other way? We're currently using https://searchfox.org/mozilla-central/source/intl/l10n/L10nRegistry.jsm#746-754

Comment 8

[Kris Maglione [:kmag]]

Well, there isn't exactly a good option. We intentionally make sync IO hard, because we want people to avoid it.

The usual solution is to create a channel, call the sync open() method, and read it (probably with NetUtil.readInputStreamToString). Usually you can do it in a single read operation, since .available should return the full size of the file for file: and local jar: streams.

However, you should test this with langpack extensions, since I'm not 100% sure how well sync reads from those work in the child process.

Also, in the parent process, you should continue using the URL preloader.

Comment 9

[BugBot [:suhaib / :marco/ :calixte]]

The priority flag is not set for this bug.
:m_kato, could you have a look please?

For more information, please visit auto_nag documentation.

Comment 10

[:Gijs (he/him)]

Can't we make the APIs involved here all async and just not support sync IO in the child?

Comment 11

[Zibi Braniecki [:zbraniecki][:gandalf]]

Can't we make the APIs involved here all async and just not support sync IO in the child?

Not really. ECMA402 is synchronous, and mozIntl is intended to be a temporary chrome-only implementation of upcoming ECMA402.

In particular, the bits that currently use Fluent Sync API will be moved to Intl API as we implement Intl.DisplayNames API in bug 1557727.

I'd recommend against diverging from the target API in mozIntl.

Comment 12

[:Gijs (he/him)]

So concretely, what does this bug break, now that it's been morphed...? Any and/or the first access to the region/language FTL equivalents in the content process, e.g. also the uses from FormAutoFill and various other things? Or only specific APIs? Or something else? (ni gandalf)

Can we make the data available from the parent to the child via shared memory or something like that? (ni kmag)

Comment 13

[Kris Maglione [:kmag]]

(In reply to :Gijs (he/him) from comment #12)

Can we make the data available from the parent to the child via shared memory or something like that? (ni kmag)

We've talked about using shared memory for processed fluent resources like we currently do for string bundles. I think that's going to be necessary one way or the other, but I'm not sure it entirely solves this problem. There will always be the possibility that a resource will be needed synchronously before the shared data is available in the child process.

For the shared memory string bundles, we fall back to synchronously loading the string bundles when that happens, and then replacing them with the shared memory versions when those become available. We'd presumably need to do something similar for fluent.

Comment 14

[j.j.]

Is this fixed now per bug 1586216 comment 23 ?

Comment 15

[:Gijs (he/him)]

(In reply to j.j. from comment #14)

Is this fixed now per bug 1586216 comment 23 ?

Dave would know... When I try the steps in comment #0 I now see:

[Exception... "Component returned failure code: 0x80520012 (NS_ERROR_FILE_NOT_FOUND) [nsIChannel.open]"  nsresult: "0x80520012 (NS_ERROR_FILE_NOT_FOUND)"  location: "JS frame :: resource://gre/modules/L10nRegistry.jsm :: this.L10nRegistry.loadSync :: line 772"  data: no]

which is odd because it looks like a permanent error, but it does not recur if I reload reader mode.

The URL in question is resource://app/localization/en-US/toolkit/intl/languageNames.ftl, and in fact on my local build there is no toolkit folder in resource://app/localization/en-US/. Same in beta. So I think something else is wrong here.

Comment 16

[Axel Hecht [:Pike]]

(In reply to :Gijs (he/him) from comment #15)

(In reply to j.j. from comment #14)

Is this fixed now per bug 1586216 comment 23 ?

Dave would know... When I try the steps in comment #0 I now see:

[Exception... "Component returned failure code: 0x80520012 (NS_ERROR_FILE_NOT_FOUND) [nsIChannel.open]"  nsresult: "0x80520012 (NS_ERROR_FILE_NOT_FOUND)"  location: "JS frame :: resource://gre/modules/L10nRegistry.jsm :: this.L10nRegistry.loadSync :: line 772"  data: no]

Flagging mossop with the needinfo on this instead of me.

Comment 17

[:Gijs (he/him)]

FWIW, looks like that error is just a result of the file not existing in resource://app/localization, when it does exist in resource://gre/localization . We should probably suppress the "file not found" error, and only error if we fail to find the file anywhere.

Comment 18

[Dave Townsend [:mossop]]

(In reply to :Gijs (he/him) from comment #17)

FWIW, looks like that error is just a result of the file not existing in resource://app/localization, when it does exist in resource://gre/localization . We should probably suppress the "file not found" error, and only error if we fail to find the file anywhere.

This makes sense to me. We cache the failure which is why you don't get it when you reload.

Comment 19

[Zibi Braniecki [:zbraniecki][:gandalf]]

I can't reproduce using currently Nightly. I suspect bug 1621024 might have fix it.