B2G l10n nightly bustage from "too much recursion" error (make[6]: *** [webapp-optimize] Error 3 and other errors)

RESOLVED FIXED

Status

Firefox OS
Gaia::L10n
--
blocker
RESOLVED FIXED
3 years ago
3 years ago

People

(Reporter: RyanVM, Assigned: gandalf)

Tracking

({intermittent-failure, regression})

Firefox Tracking Flags

(Not tracked)

Details

Attachments

(2 attachments)

(Reporter)

Description

3 years ago
This either needs immediate fixing or bug 914414 needs to be backed out.

https://tbpl.mozilla.org/php/getParsedLog.php?id=37533183&tree=Mozilla-Central

b2g_mozilla-central_linux32_gecko_localizer nightly on 2014-04-09 16:09:26 PDT for push 690c810c8e3e
slave: bld-linux64-spot-120

Exception: InternalError: too much recursion

make[6]: *** [webapp-optimize] Error 3
make[5]: *** [libs] Error 2
make[6]: Leaving directory `/builds/slave/m-cen-linux32_g_lz-ntly-000000/build/gaia'
make[5]: Leaving directory `/builds/slave/m-cen-linux32_g_lz-ntly-000000/build/obj-firefox/b2g/gaia'
make[4]: *** [b2g/gaia/libs] Error 2
make[4]: Leaving directory `/builds/slave/m-cen-linux32_g_lz-ntly-000000/build/obj-firefox'
make[3]: *** [libs] Error 2
make[3]: Leaving directory `/builds/slave/m-cen-linux32_g_lz-ntly-000000/build/obj-firefox'
make[2]: *** [default] Error 2
make[2]: Leaving directory `/builds/slave/m-cen-linux32_g_lz-ntly-000000/build/obj-firefox'
make[1]: *** [realbuild] Error 2
make[1]: Leaving directory `/builds/slave/m-cen-linux32_g_lz-ntly-000000/build'
make: *** [build] Error 2
Flags: needinfo?(stas)
It looks like all of the locale-processing code is synchronously generating notifications and the base logic in webapp-optimize.js is just looping through all of the defined locales it knows about:
https://github.com/mozilla-b2g/gaia/blob/master/build/webapp-optimize.js#L573

Per my grep, it has taken care of 41 locales before in 423 frames before it blows the 512k stack limit it has on (non-debug) 32-bit linux.

A setTimeout/do_execute_soon would probably go a long way in that code.  (Specifically, don't touch the setter on https://github.com/mozilla-b2g/gaia/blob/master/build/webapp-optimize.js#L585 in the same loop cycle that dispatchEvent is being called.)
(Assignee)

Comment 2

3 years ago
looking into it
(Assignee)

Comment 3

3 years ago
can't reproduce it on my local machine with 62 locales on macos. Is it possible that we have different stack limit per platform?

The logic around locale cycling didn't change, but the stack have got deeper.
Yes, you have a 7 megabyte stack on OS X.  You will be unable to reproduce.
http://dxr.mozilla.org/mozilla-central/source/js/xpconnect/src/XPCJSRuntime.cpp#3051
(Assignee)

Comment 5

3 years ago
One way to get it out of a single loop would be to just launch each app separately in https://github.com/mozilla-b2g/gaia/blob/master/build/webapp-optimize.js#L706

But there's no setTimeout in that code. Any idea what else can I use?
(Assignee)

Comment 6

3 years ago
Created attachment 8404483 [details] [diff] [review]
patch

This code does exactly the same logic, but skips the pseudo-event dispatching.

It capitalizes on the fact that resource loading in buildtime mode is synchronous.
Assignee: stas → gandalf
Status: NEW → ASSIGNED
Attachment #8404483 - Flags: review?(stas)
(Assignee)

Comment 7

3 years ago
We should overall clean webapp-optimize use of l10n.js. I filed bug 994520.
I can confirm based on inspection that this should address the problem as I understood it from the backtraces.

Specifically, the problem used to be caused by a write to:
  mozL10n.language.code = l10nLocales[processedLocales];
which would then do:
  navigator.mozL10n.ctx.requestLocales(lang);

The revised patch directly calls:
  lang = l10nLocales[processedLocales];
  mozL10n.ctx.requestLocales(lang);

And the control flow has been refactored so that the same effective linear code-path is the same, but without the recursion.

So r=asuth for what it's worth assuming travis is green.
I'm taking over while Gandalf is asleep on the west coast.  I'll create a pull request soon.  Thanks everyone for investigating this.
Flags: needinfo?(stas)
Comment on attachment 8404483 [details] [diff] [review]
patch

Review of attachment 8404483 [details] [diff] [review]:
-----------------------------------------------------------------

r=me.  I'll create a pull request for you and ask Andrew and Vivien for a review.

::: build/webapp-optimize.js
@@ +570,4 @@
>      };
>    };
>  
> +  win.dispatchEvent = function() {};

I'd move that to the beginning of the file where win is defined.

@@ +596,5 @@
> +          processedLocales + '/' + l10nLocales.length);
> +
> +        lang = l10nLocales[processedLocales];
> +
> +        mozL10n.ctx.requestLocales(lang);

I think this can simply read:

mozL10n.ctx.requestLocales(l10nLocales[processedLocales]);
Attachment #8404483 - Flags: review?(stas) → review+
Created attachment 8404638 [details] [review]
Pull request
Attachment #8404638 - Flags: superreview?(21)
Attachment #8404638 - Flags: review?(bugmail)
Comment on attachment 8404638 [details] [review]
Pull request

Noting that I don't really have review authority for this code, (although for a nightly bustage fix I think the bar is a lot lower), this looks substantially the same as the other attachment except for the movement of dispatchEvent, so I'll give it my prestigious r=asuth too.
Attachment #8404638 - Flags: review?(bugmail) → review+
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Thanks, Andrew :)

The Travis build is green: https://travis-ci.org/mozilla-b2g/gaia/builds/22685615
Comment on attachment 8404638 [details] [review]
Pull request

Alexandre, can you take a look?  Thanks!
Attachment #8404638 - Flags: superreview?(21) → review?(poirot.alex)
Comment hidden (Treeherder Robot)
Comment on attachment 8404638 [details] [review]
Pull request

Looks good, just see my one comment in the pull request.
Attachment #8404638 - Flags: review?(poirot.alex) → review+
Thanks!

Commit: https://github.com/mozilla-b2g/gaia/commit/5f7af7526682712d414abb64c4234c3a10eb8f9a
Marged: https://github.com/mozilla-b2g/gaia/commit/43255a9fac46da49937eb145aeeee0389d62ef07
Status: ASSIGNED → RESOLVED
Last Resolved: 3 years ago
Resolution: --- → FIXED
Comment hidden (Treeherder Robot)
You need to log in before you can comment on or make changes to this bug.