1543458 - Convert over various wholly-internal, non-web-exposed users of UTF-8 compilation to non-inflating compilation

Status: RESOLVED FIXED

(Core :: JavaScript Engine, task, P1)

Description

[Jeff Walden [:Waldo]]

Non-inflating compilation is faster and more efficient, but an instantaneous switchover is potentially risky. Let's switch over internal users where if we made a mistake, we'd only be stabbing ourselves, first. :-) Anything web-visible will be separate bugs.

Comment 1

[Jeff Walden [:Waldo]]

Attachment: Bug 1543458 - Perform non-inflating syntactic UTF-8 compilation in jsapi-tests. r=arai

Depends on D26953

Comment 2

[Jeff Walden [:Waldo]]

Attachment: Bug 1543458 - Perform non-inflating nonsyntactic UTF-8 compilation in jsapi-tests. r=arai

Depends on D26954

Comment 3

[Jeff Walden [:Waldo]]

Attachment: Bug 1543458 - Compile lines of UTF-8 interactive input to xpcshell and jsshell without inflating to UTF-16. r=arai

Depends on D26955

Comment 4

[Jeff Walden [:Waldo]]

Attachment: Bug 1543458 - Compile UTF-8 files in jsapi-tests without inflating to UTF-16. r=arai

Depends on D26956

Comment 5

[Jeff Walden [:Waldo]]

Attachment: Bug 1543458 - Make the JS component loader compile UTF-8 directly without inflating it. r=arai

Depends on D26957

Comment 6

[Jeff Walden [:Waldo]]

Attachment: Bug 1543458 - Make the JS subscript loader compile UTF-8 directly without inflating it. r=arai

Depends on D26958

Comment 7

[Jeff Walden [:Waldo]]

jsapi-tests are internal and do arbitrary things, we can do whatever in them.

Interactive input we can tolerate hypothetical bugs in, because it's not something we're using to test stuff, where hypothetically diverging from web behavior would be not so great.

The JS component and subscript loaders only consume our internal components and scripts, as known from past work done to make them always load as UTF-8.

Comment 8

[Jeff Walden [:Waldo]]

The component and/or subscript loader parts of this bug have to advance in lockstep with sourceHook understanding UTF-8. But it appears the rest of this is landable, and getting that in the tree at least helps with keeping my tree from looking totally disjoint from Searchfox, so will do that shortly.

Comment 9

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/mozilla-inbound/rev/ed24501174af
Perform non-inflating syntactic UTF-8 compilation in jsapi-tests.  r=arai
https://hg.mozilla.org/integration/mozilla-inbound/rev/40edaa2016cc
Perform non-inflating nonsyntactic UTF-8 compilation in jsapi-tests.  r=arai

Comment 10

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/mozilla-inbound/rev/e0c75d5168be
Compile lines of UTF-8 interactive input to xpcshell and jsshell without inflating to UTF-16.  r=arai
https://hg.mozilla.org/integration/mozilla-inbound/rev/6ceeb37b708d
Compile UTF-8 files in jsapi-tests without inflating to UTF-16.  r=arai

Comment 11

[Cosmin Sabou [:CosminS]]

https://hg.mozilla.org/mozilla-central/rev/ed24501174af
https://hg.mozilla.org/mozilla-central/rev/40edaa2016cc
https://hg.mozilla.org/mozilla-central/rev/e0c75d5168be
https://hg.mozilla.org/mozilla-central/rev/6ceeb37b708d

Comment 12

[Jeff Walden [:Waldo]]

Subscript loader/component loader patches haven't landed yet, because the source hook mechanism is purely UTF-16 now and thus all our source coordinates for lazy compilation of functions, etc. go awry if there's any non-ASCII in the script. I have patches that'll fix that, in a bug to be filed anon (but not anonymously).

Comment 13

[Jeff Walden [:Waldo]]

Source hook stuff landed earlier today, but now there's another blocker issue to making this change -- bug 1545015/bug 1545034/bug 1546313 concerning perf regressions in column number computations. In principle those (or at least the first) at this point "ought" have been resolved by a backout of the last patch for bug 1504947, but I have so far (perhaps because I was PTO for a week) received a silent reprieve from backout.

Anyway -- if a backout happens before I can fix things, it'll break any column numbering performed for JS subscripts/component that contain non-ASCII text, so I am rather loathe to land these final patches before I deal with that. I'll mark a dependency on whichever of those bugs ends up receiving the algorithmic amelioration, once I figure out which makes the most sense to work in.

Comment 14

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/mozilla-inbound/rev/f20b97afe818
Make the JS component loader compile UTF-8 directly without inflating it.  r=arai
https://hg.mozilla.org/integration/mozilla-inbound/rev/32efcd22ed9c
Make the JS subscript loader compile UTF-8 directly without inflating it.  r=arai

Comment 15

[Daniel Varga [:dvarga]]

https://hg.mozilla.org/mozilla-central/rev/f20b97afe818
https://hg.mozilla.org/mozilla-central/rev/32efcd22ed9c

Comment 16

[Florin Strugariu [:Bebe]]

== Change summary for alert #21074 (as of Fri, 24 May 2019 10:40:58 GMT) ==

Improvements:

3% Base Content JS windows7-32-shippable opt 3,241,658.67 -> 3,147,533.33
2% Base Content JS macosx1010-64-shippable opt 4,095,770.67 -> 3,999,668.00
2% Base Content JS linux64-shippable-qr opt 4,094,809.33 -> 3,998,654.67
2% Base Content JS linux64-shippable opt 4,094,737.33 -> 3,998,634.67
2% Base Content JS windows10-64-shippable-qr opt 4,148,622.67 -> 4,054,770.67
2% Base Content JS windows10-64-shippable opt 4,148,581.33 -> 4,054,752.00

For up to date results, see: https://treeherder.mozilla.org/perf.html#/alerts?id=21074

Comment 17

[Jeff Walden [:Waldo]]

Nice! I was expecting some kind of improvement somewhere but had not taken the time to identify some specific statistic that would change, to measure a difference.

I would expect to see some improvement in memory from these changes. (But maybe a possibly hard to quantify amount, given how many different subsystems -- and sequences of operations in JS performed -- interact to produce the overall memory footprint.) Are these numbers a measure of memory consumption, probably?

Comment 18

[Jan de Mooij [:jandem]]

These are number of bytes (from JS memory reporters) in a mostly-empty content process, as far as I know.

Comment 19

[Florin Strugariu [:Bebe]]

:Waldo I think you can find here the information you need:
https://wiki.mozilla.org/AWSY/Tests#Base_Content_JS

Base Content JS

    An updated test focused on supporting fission. This measures the base overhead of an empty content process. It tracks resident unique, heap unclassified, JS, and explicit memory metrics as well as storing full memory reports as artifacts. The median value for each metric is used from across all content processes. It has much lower thresholds for alerting and is recorded in Perfherder.