1498320 - Make BytecodeCompiler able to handle UTF-8 source text

Status: RESOLVED FIXED

(Core :: JavaScript Engine, enhancement, P2)

Description

[Jeff Walden [:Waldo]]

Mostly this just involves a bunch of refactoring of BytecodeCompiler.* to separate out the stuff that deals with SourceBufferHolder from the rest of it, to reduce demand for doubling up of stuff.  Patches mostly done, still a little to go before ready to review.

Comment 1

[Jeff Walden [:Waldo]]

Attachment: Remove obsolete "|options.utf8| must be set" comments from UTF-8 compilation function docs

Dumping here just because.

Comment 2

[Jeff Walden [:Waldo]]

Attachment: When compiling a module script, create the ModuleObject after creating the script for it

If there's a reason for this, and it's hard to imagine one, it's not documented, so undo this.  (Also jstests/jit-tests passed with this in place, so...)

Comment 3

[Jeff Walden [:Waldo]]

Attachment: Compute script start position after creating the script proper, in BC::compileScript

In hindsight I'm not sure this does anything, but I think it might mean the earlier two calls are unified at some point in this patch stack.  If not, it makes clear the lack of ordering dependency.

Comment 4

[Jeff Walden [:Waldo]]

Attachment: Move module-compilation's createSourceAndParser call into a separate prepareModuleParse function, anticipating when only that function will have to deal with multiple kinds of SourceBufferHolder

More hindsight, I'm not entirely sure this survives to the end of the queue either, but it's not worth the trouble to undo it given it's so small.

Comment 5

[Jeff Walden [:Waldo]]

Attachment: Move script-compilation's createSourceAndParser call into a separate prepareScriptParse function, anticipating when only that function will have to deal with multiple kinds of SourceBufferHolder

Ditto uncertainty about utility here, but also too small to care.

Comment 6

[Jeff Walden [:Waldo]]

Attachment: Move standalone function-compilation's createSourceAndParser call into a separate prepareStandaloneFunctionParse function, anticipating when that function will have to deal with multiple kinds of SourceBufferHolder

Again I'm not sure this survives to end, but again not worth it to undo.

Comment 7

[Jeff Walden [:Waldo]]

Attachment: Move script creation into prepareScriptParse

This cleanup does simplify some stuff tho.

Comment 8

[Jeff Walden [:Waldo]]

Attachment: Move script creation into prepareModuleParse

Comment 9

[Jeff Walden [:Waldo]]

Attachment: Rename BytecodeCompiler::createScript() to BytecodeCompiler::createCompleteScript()

It's tidier/more readable not to rely on overload resolution.

Comment 10

[Jeff Walden [:Waldo]]

Attachment: Split BytecodeCompiler::compileStandaloneFunction into BytecodeCompiler::{parse,compile}StandaloneFunction

I think I might end up undoing this later as well, but it's again too much trouble to undo.  (I keep saying this, and it really is -- problem is I have umpteen patches to BC.cpp atop this, and one tweak cascades through the entire thing because I touch so much of the file.)

Comment 11

[Jeff Walden [:Waldo]]

Attachment: Move BytecodeCompiler's members to the top of the class definition

At the end of all these maybe it's worth adding the usual trailing underscores -- but no sooner, given I keep touching everything.

Comment 12

[Jeff Walden [:Waldo]]

Attachment: Define BytecodeCompiler::sourceObjectPtr() inline, closer to the top of BytecodeCompiler's definition

Not sure why it would/should be any other way.

Comment 13

[Jeff Walden [:Waldo]]

Attachment: Remove BytecodeCompiler::enclosingScope and just make users provide it themselves when they need it

BC has some mode-specific stuff in it that really belongs in subclasses, or passed in by callers.  This is the first bit of that.

Comment 14

[Jeff Walden [:Waldo]]

Attachment: Create a BytecodeCompiler subclass to perform global script compilation

Eventually these subclasses disappear -- we gotta keep parser/sourceBuffer in a Unit-parametrized place -- but for now these are small simple transformation that move us the right direction.

Comment 15

[Jeff Walden [:Waldo]]

Attachment: Create a BytecodeCompiler subclass to perform eval script compilation

Comment 16

[Jeff Walden [:Waldo]]

Attachment: Create a BytecodeCompiler subclass to perform module compilation

Ignore the FooBytecodeCompiler names -- I rename them to FooInfo by the end of these patches.  (Well, except BytecodeCompiler itself because it's too much pain til all this lands.)

Interesting how enclosingScope is always the same thing, so this gets rid of an extra frob that's always set the same.

Comment 17

[Jeff Walden [:Waldo]]

Attachment: Create a BytecodeCompiler subclass for standalone function compilation

Comment 18

[Jeff Walden [:Waldo]]

Attachment: Adjust some access controls now that BytecodeCompiler is used only as base class, not on its own

Comment 19

[Jeff Walden [:Waldo]]

Attachment: Separate source-unit-aware functionality out of BytecodeCompiler into unrelated subclasses so that unit-agnostic functionality can be shared

Sorry this one ends up being a bit of a mess.  :-(  I spent much of the weekend trying to clean it up some, and I did succeed some, but not some enough for my tastes.

The important thing here is, the *Info classes need/want to be usable regardless of source-text type.  We want to be able to fill them out once, for two arms of compilation, in some places.  So the Compiler classes hold all Unit-ful stuff, and they're distinct and not inheritance-related to the Info classes.

Comment 20

[Jeff Walden [:Waldo]]

Attachment: Templatize everything that's source-aware in bytecode compilation

This is separate from the previous step for mildly better readability.

Post hoc note -- parser/syntaxParser should be trailing-underscored, and they would if this were new code, but the existing code does use parser everywhere, so I didn't want to rename and make life painful for me in the patch-stack.  Like renaming BC, renaming these should wait til everything's landed.

Comment 21

[Jeff Walden [:Waldo]]

Attachment: Implement CompileGlobalScript in terms of a template CreateGlobalScript function that handles UTF-8 and UTF-16 both

Exposing only functions that take a GlobalScriptInfo& passed in/created byt eh caller is at the end of this road.  But one step at a time, and this doesn't have to touch callers to work.

Comment 22

[Jeff Walden [:Waldo]]

Attachment: Implement ScriptSource::setSourceCopy to work for both UTF-8 and UTF-16 source text

setSourceCopy is called outside the file it's defined in, if memory serves, so it requires explicit instantiation.

Comment 23

[Jeff Walden [:Waldo]]

Attachment: Create syntax and full parsers in bytecode compilation passing |SourceText::units()|, so that Utf8Unit appearing here will continue to require individualized treatment

Comment 24

[Jeff Walden [:Waldo]]

Attachment: Implement ScriptSource::appendSubstring for UTF-8 source text, using a newly-implemented StringBuffer::append(const Utf8Unit* units, size_t len)

I haven't explicitly tested this all yet, but I think it should get the job done.  It *does* allow some tests (with later patches in place) to pass under UTF-8 parsing, so there's that.

We could debate about exactly what tradeoffs to make inside append(const Utf8Unit*, size_t).  I picked a way to make it work to start that's reasonably simple and stupid and Gets The Job Done For Now.  We can revisit exact approach later if necessary.  No need for omphaloskepsis now.

Comment 25

[Jeff Walden [:Waldo]]

Attachment: Add a UTF-8 parser to the Variant inside EitherParser

Through the Magic of Templates, this Just Works!  "what have we done"

Comment 26

[Jeff Walden [:Waldo]]

Attachment: Introduce a new BytecodeCompilation.h header for fresh bytecode compilation signatures coming soon

This exposes BC to users, so they can fill in a bunch of stuff on the stack in cases where *either* UTF-8 or UTF-16 will happen later, and you want to share filling-in beforehand.  It does a proper file-copy and all.  It also is the super-easiest thing to break with changes made to prior patches.  :-|  Forebearance in making requested changes to prior patches, maybe after this one if it's easier, would be appreciated.

End goal is BC, ScriptInfo, EvalScriptInfo, GlobalScriptInfo, ModuleInfo, StandaloneFunctionInfo will be in the header -- and so will Compile* functions that take (*Info&, SourceText<Unit>&, ...) and do the unit-specific work.  I could have kept building in BC.h, but a separate header seems cleaner.  End of day, probably all of BC.* can be moved elsewhere or whatever.  But currently BC.* is a bit of a mess, and smaller more focused stuff is better IMO.

Comment 27

[Jeff Walden [:Waldo]]

Attachment: Remove the original CompileGlobalScript overload, and rewrite all users to use the new (GlobalScriptInfo&, SourceText<char16_t>&, ScriptSourceObject**) overload

The first new-style overload.  Removal of old-style and callers happens separately.

Comment 28

[Jeff Walden [:Waldo]]

Attachment: Remove the original CompileEvalScript overload, and rewrite all users to use a new (EvalScriptInfo&, SourceText<char16_t>&) overload that calls a UTF-8/16-ready template function

...or maybe the previous patch did rewrite all callers.  Never mind.

Note that I'm really not caring about whether both Unit parametrizations actually *happen*, just making it possible to have both happen whenever Compile* overloads for both unit types are needed.  It may well be that EvalScriptInfo never needs a UTF-8 variant -- eval text coming from strings that are either Latin-1 or UTF-16, and the former I think is always inflated to the latter right now, and maybe even should be.  But, if we never instantiate for UTF-8 now, we never get UTF-8 now, so it doesn't hurt generalizing stuff.

Comment 29

[Jeff Walden [:Waldo]]

Attachment: Convert the two CompileModule functions to template functions, then call them inside non-template overloads

Comment 30

[Jeff Walden [:Waldo]]

Try run of all the patches I bombed you with last night:

https://treeherder.mozilla.org/#/jobs?repo=try&revision=d63c67476c9778520d623a1fb374c3c52bf97494

Comment 31

[Jeff Walden [:Waldo]]

Okay, most of the orange is from missing one |- 1| in the initial patch for bug 1493441.  Bizarrely, jstests/jit-tests do not exercise *at all* compressed source.  Not sure why, and I timed out trying to figure it out earlier today.

More try-running with that fixed is significantly greener, yet turning up some issues that I *may* have fixt in this push:

https://treeherder.mozilla.org/#/jobs?repo=try&revision=6ece5a083389e01a5b6b041d72b5775f99bf4745

but we'll see.  (And notably there's some Rust change that I haven't been able to get to work locally, that might still need tweaks -- but it's also tier2, so there's that.)

Comment 32

[Jeff Walden [:Waldo]]

Okay -- I think everything the try-pushes flagged should be fixed now.  New try-push of this bug and a mess of others:

https://treeherder.mozilla.org/#/jobs?repo=try&revision=736aa614eacd94a11f563f8f0a72464721a77f54

with all the relevant log of changes in this push:

https://hg.mozilla.org/try/pushloghtml?changeset=736aa614eacd94a11f563f8f0a72464721a77f54

(obviously starting where my changes start -- ignore the clang-plugin change, it's not relevant to any of this).

Mostly every patch here did have to be adjusted for the removal of the silly |LifoAlloc& alloc| argument to BytecodeCompiler, but the changes -- aside from being a very very very slow rebase -- were essentially mechanical and didn't fundamentally break anything in these patches.  If you want to see what I have post-changing, look at the try-push, but it doesn't seem necessary to bugspam every patch again here for not-interesting changes.

Comment 33

[Ted Campbell [:tcampbell]]

Comment on attachment 9020992 [details] [diff] [review]
When compiling a module script, create the ModuleObject after creating the script for it

Review of attachment 9020992 [details] [diff] [review]:
-----------------------------------------------------------------

Order does not appear to matter.

Comment 34

[Ted Campbell [:tcampbell]]

Comment on attachment 9020997 [details] [diff] [review]
Move script creation into prepareScriptParse

Review of attachment 9020997 [details] [diff] [review]:
-----------------------------------------------------------------

::: js/src/frontend/BytecodeCompiler.cpp
@@ +52,4 @@
>  
>      // Call this before calling compile{Global,Eval}Script.
>      MOZ_MUST_USE bool prepareScriptParse() {
> +        return createSourceAndParser(ParseGoal::Script) && createScript();

Ehh.. I guess this is what our style rules say but gives a bit of a whiff of hiding side-effects in conditionals. Oh well, it's fine.

Comment 35

[Ted Campbell [:tcampbell]]

Comment on attachment 9021000 [details] [diff] [review]
Split BytecodeCompiler::compileStandaloneFunction into BytecodeCompiler::{parse,compile}StandaloneFunction

Review of attachment 9021000 [details] [diff] [review]:
-----------------------------------------------------------------

::: js/src/frontend/BytecodeCompiler.cpp
@@ +70,5 @@
>          return createSourceAndParser(ParseGoal::Script, parameterListEnd);
>      }
>  
> +    // Call this before calling compileStandaloneFunction.
> +    CodeNode* parseStandaloneFunction(MutableHandleFunction fun, GeneratorKind generatorKind,

Should be |HandleFunction fun|

@@ +74,5 @@
> +    CodeNode* parseStandaloneFunction(MutableHandleFunction fun, GeneratorKind generatorKind,
> +                                      FunctionAsyncKind asyncKind,
> +                                      const Maybe<uint32_t>& parameterListEnd);
> +
> +    bool compileStandaloneFunction(CodeNode* parsedFunction, MutableHandleFunction fun);

I'd generally prefer the MutableHandleFunction to be the first argument since it is the partially initialized subject. In this current form it looks like an out param in our style rather than an in-out. That being said, this isn't worth rebasing over.

Comment 36

[Ted Campbell [:tcampbell]]

Comment on attachment 9021002 [details] [diff] [review]
Define BytecodeCompiler::sourceObjectPtr() inline, closer to the top of BytecodeCompiler's definition

Review of attachment 9021002 [details] [diff] [review]:
-----------------------------------------------------------------

uhh.. drop-down menus are hard, okay?

Comment 37

[Ted Campbell [:tcampbell]]

Comment on attachment 9021006 [details] [diff] [review]
Create a BytecodeCompiler subclass to perform module compilation

Review of attachment 9021006 [details] [diff] [review]:
-----------------------------------------------------------------

::: js/src/frontend/BytecodeCompiler.cpp
@@ +462,2 @@
>  {
> +    if (!createSourceAndParser(ParseGoal::Module) || !createCompleteScript()) {

It would be nice to common this up with prepareScriptParse for consistency. Even at the end of the patch stack there seems to be inconsistency in which parse entry points use prepare/parse/compile.

Comment 38

[Ted Campbell [:tcampbell]]

Comment on attachment 9021007 [details] [diff] [review]
Create a BytecodeCompiler subclass for standalone function compilation

Review of attachment 9021007 [details] [diff] [review]:
-----------------------------------------------------------------

::: js/src/frontend/BytecodeCompiler.cpp
@@ +189,5 @@
> +      : BytecodeCompiler(cx, alloc, options, sourceBuffer)
> +    {}
> +
> +    MOZ_MUST_USE bool prepare(const Maybe<uint32_t>& parameterListEnd) {
> +        return createSourceAndParser(ParseGoal::Script, parameterListEnd);

Might be worth a having a comment to say that createScript is called in SFBC::compile.

In follow-up work it would be nice if we could make StandaloneFunction and GlobalScript/EvalScript agree on script handling for the asm.js case. This one seems to avoid allocating the JSScript, but the EvalScript case always allocates the JSScript.

Comment 39

[Ted Campbell [:tcampbell]]

Comment on attachment 9021009 [details] [diff] [review]
Separate source-unit-aware functionality out of BytecodeCompiler into unrelated subclasses so that unit-agnostic functionality can be shared

Review of attachment 9021009 [details] [diff] [review]:
-----------------------------------------------------------------

Okay, put the jigsaw puzzle together and didn't end up with any missing pieces.

The rebased version on https://hg.mozilla.org/try/rev/fc2825851782 also is r+.

::: js/src/frontend/BytecodeCompiler.cpp
@@ +154,5 @@
>      // This assumes the created script's offsets in the source used to parse it
>      // are the same as are used to compute its Function.prototype.toString()
>      // value.
> +    MOZ_MUST_USE bool createCompleteScript(BytecodeCompiler& info) {
> +        uint32_t len = sourceBuffer_.length();

s/len/toStringEnd/ to make the comment above super clear.

@@ +155,5 @@
>      // are the same as are used to compute its Function.prototype.toString()
>      // value.
> +    MOZ_MUST_USE bool createCompleteScript(BytecodeCompiler& info) {
> +        uint32_t len = sourceBuffer_.length();
> +        return info.internalCreateScript(0, len, len);

/* toStringStart = */ 0

@@ -255,3 @@
>  
>  BytecodeCompiler::BytecodeCompiler(JSContext* cx,
>                                     LifoAlloc& alloc,

I removed this in m-c to cleanup some mess, but now that SourceAwareCompiler exists it might be nice to put the alloc ref there (so that it matches Maybe<Parser> lifetime). Then we could properly have all the FooCompiler instances get passed a LifoAllocScope&. (The LifoAllocScope would be in code instead of wrapped in FooCompiler because I've found embedding LifoAllocScope in structs is asking for trouble).

Anyways, any of this would be after this whole series sticks first.

Comment 40

[Ted Campbell [:tcampbell]]

Comment on attachment 9021010 [details] [diff] [review]
Templatize everything that's source-aware in bytecode compilation

Review of attachment 9021010 [details] [diff] [review]:
-----------------------------------------------------------------

Thanks for splitting off this patch.

Comment 41

[Jeff Walden [:Waldo]]

Attachment: Implement ScriptSource::appendSubstring for UTF-8 source text, using a newly-implemented StringBuffer::append(const Utf8Unit* units, size_t len)

Previous patch had a simple stupid bug pointed out in it when I could actually exercise this.

Comment 42

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/mozilla-inbound/rev/958a348c9ee0
Remove obsolete "|options.utf8| must be set" comments from UTF-8 compilation function docs.  r=tcampbell
https://hg.mozilla.org/integration/mozilla-inbound/rev/15d7f3fb96d5
When compiling a module script, create the ModuleObject after creating the script for it.  r=tcampbell
https://hg.mozilla.org/integration/mozilla-inbound/rev/e6b6326c593d
Compute script start position after creating the script proper, in BC::compileScript.  r=tcampbell
https://hg.mozilla.org/integration/mozilla-inbound/rev/2052c58dfe98
Move module-compilation's createSourceAndParser call into a separate prepareModuleParse function, anticipating when only that function will have to deal with multiple kinds of SourceBufferHolder.  r=tcampbell
https://hg.mozilla.org/integration/mozilla-inbound/rev/d9e71df3d1d7
Move script-compilation's createSourceAndParser call into a separate prepareScriptParse function, anticipating when only that function will have to deal with multiple kinds of SourceBufferHolder.  r=tcampbell
https://hg.mozilla.org/integration/mozilla-inbound/rev/ac6aea751ee9
Move standalone function-compilation's createSourceAndParser call into a separate prepareStandaloneFunctionParse function, anticipating when that function will have to deal with multiple kinds of SourceBufferHolder.  r=tcampbell
https://hg.mozilla.org/integration/mozilla-inbound/rev/37b4694f5fcf
Move script creation into prepareScriptParse.  r=tcampbell
https://hg.mozilla.org/integration/mozilla-inbound/rev/b0c005ef9cee
Move script creation into prepareModuleParse.  r=tcampbell
https://hg.mozilla.org/integration/mozilla-inbound/rev/2792e47984ee
Rename BytecodeCompiler::createScript() to BytecodeCompiler::createCompleteScript().  r=tcampbell
https://hg.mozilla.org/integration/mozilla-inbound/rev/1737079f61dd
Split BytecodeCompiler::compileStandaloneFunction into BytecodeCompiler::{parse,compile}StandaloneFunction.  r=tcampbell
https://hg.mozilla.org/integration/mozilla-inbound/rev/49b72ccc9682
Move BytecodeCompiler's members to the top of the class definition.  r=tcampbell
https://hg.mozilla.org/integration/mozilla-inbound/rev/0db3c534950c
Define BytecodeCompiler::sourceObjectPtr() inline, closer to the top of BytecodeCompiler's definition.  r=tcampbell
https://hg.mozilla.org/integration/mozilla-inbound/rev/17243a560006
Remove BytecodeCompiler::enclosingScope and just make users provide it themselves when they need it.  r=tcampbell

Comment 43

[Jeff Walden [:Waldo]]

A number of the patches here can be disentangled from dependency on bug 1503086 -- landed those.  Patches after that point start not applying cleanly upon |hg qpu --move|; I think I could fix some of that, but I run out of runway at some point, and I keep hoping that patch will get reviewed soon enough to not be worth the hassle.

So, land some now, leave-open for the rest (including the ones still outstanding).

Comment 44

[Cosmin Sabou [:CosminS]]

https://hg.mozilla.org/mozilla-central/rev/958a348c9ee0
https://hg.mozilla.org/mozilla-central/rev/15d7f3fb96d5
https://hg.mozilla.org/mozilla-central/rev/e6b6326c593d
https://hg.mozilla.org/mozilla-central/rev/2052c58dfe98
https://hg.mozilla.org/mozilla-central/rev/d9e71df3d1d7
https://hg.mozilla.org/mozilla-central/rev/ac6aea751ee9
https://hg.mozilla.org/mozilla-central/rev/37b4694f5fcf
https://hg.mozilla.org/mozilla-central/rev/b0c005ef9cee
https://hg.mozilla.org/mozilla-central/rev/2792e47984ee
https://hg.mozilla.org/mozilla-central/rev/1737079f61dd
https://hg.mozilla.org/mozilla-central/rev/49b72ccc9682
https://hg.mozilla.org/mozilla-central/rev/0db3c534950c
https://hg.mozilla.org/mozilla-central/rev/17243a560006

Comment 45

[Ted Campbell [:tcampbell]]

Comment on attachment 9021013 [details] [diff] [review]
Implement CompileGlobalScript in terms of a template CreateGlobalScript function that handles UTF-8 and UTF-16 both

Review of attachment 9021013 [details] [diff] [review]:
-----------------------------------------------------------------

::: js/src/frontend/BytecodeCompiler.cpp
@@ +266,5 @@
> +    explicit ScriptCompiler(SourceText<Unit>& srcBuf)
> +      : Base(srcBuf)
> +    {}
> +
> +    MOZ_MUST_USE bool prepareScriptParse(BytecodeCompiler& compiler) {

(Optional) This is probably a unnecessary redefinition.

Comment 46

[Ted Campbell [:tcampbell]]

Comment on attachment 9021031 [details] [diff] [review]
Introduce a new BytecodeCompilation.h header for fresh bytecode compilation signatures coming soon

Review of attachment 9021031 [details] [diff] [review]:
-----------------------------------------------------------------

Sum of result files seems to correspond to original file and spot-checks all looked good.

Comment 47

[Ted Campbell [:tcampbell]]

Comment on attachment 9021032 [details] [diff] [review]
Remove the original CompileGlobalScript overload, and rewrite all users to use the new (GlobalScriptInfo&, SourceText<char16_t>&, ScriptSourceObject**) overload

Review of attachment 9021032 [details] [diff] [review]:
-----------------------------------------------------------------

\o/

Comment 48

[Ted Campbell [:tcampbell]]

Comment on attachment 9024142 [details] [diff] [review]
Implement ScriptSource::appendSubstring for UTF-8 source text, using a newly-implemented StringBuffer::append(const Utf8Unit* units, size_t len)

Review of attachment 9024142 [details] [diff] [review]:
-----------------------------------------------------------------

Looks reasonable. I'm assuming any weirdness about start/end would have been addressed in the column number fixes bug so I did not stare too hard at the pre-existing code for that.

::: js/src/vm/CharacterEncoding.cpp
@@ +631,5 @@
> +
> +            ++units;
> +            --len;
> +        }
> +        if (len == 0) {

Need a comment after this.

// Encountered non-ASCII text so inflate this StringBuffer and continue converting remaining characters.

Comment 49

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/mozilla-inbound/rev/43b8810bb3ed
Create a BytecodeCompiler subclass to perform global script compilation.  r=tcampbell
https://hg.mozilla.org/integration/mozilla-inbound/rev/76feab5328ed
Create a BytecodeCompiler subclass to perform eval script compilation.  r=tcampbell
https://hg.mozilla.org/integration/mozilla-inbound/rev/3ed7e8a37562
Create a BytecodeCompiler subclass to perform module compilation.  r=tcampbell
https://hg.mozilla.org/integration/mozilla-inbound/rev/9dc874d79c1f
Create a BytecodeCompiler subclass for standalone function compilation.  r=tcampbell
https://hg.mozilla.org/integration/mozilla-inbound/rev/630a16450066
Adjust some access controls now that BytecodeCompiler is used only as base class, not on its own.  r=tcampbell
https://hg.mozilla.org/integration/mozilla-inbound/rev/532b05c76fa0
Separate source-unit-aware functionality out of BytecodeCompiler into unrelated subclasses so that unit-agnostic functionality can be shared.  r=tcampbell
https://hg.mozilla.org/integration/mozilla-inbound/rev/422924c19ce5
Templatize everything that's source-aware in bytecode compilation.  r=tcampbell
https://hg.mozilla.org/integration/mozilla-inbound/rev/9d7d25507cac
Implement CompileGlobalScript in terms of a template CreateGlobalScript function that handles UTF-8 and UTF-16 both.  r=tcampbell
https://hg.mozilla.org/integration/mozilla-inbound/rev/ecf813f9f9ea
Implement ScriptSource::setSourceCopy to work for both UTF-8 and UTF-16 source text.  r=tcampbell
https://hg.mozilla.org/integration/mozilla-inbound/rev/ba4bdabbdd52
Create syntax and full parsers in bytecode compilation passing |SourceText::units()|, so that Utf8Unit appearing here will continue to require individualized treatment.  r=tcampbell
https://hg.mozilla.org/integration/mozilla-inbound/rev/774a0684f724
Implement ScriptSource::appendSubstring for UTF-8 source text, using a newly-implemented StringBuffer::append(const Utf8Unit* units, size_t len).  r=tcampbell
https://hg.mozilla.org/integration/mozilla-inbound/rev/ba6ba95b3cd2
Add a UTF-8 parser to the Variant inside EitherParser.  r=tcampbell
https://hg.mozilla.org/integration/mozilla-inbound/rev/1dd5267839b3
Introduce a new BytecodeCompilation.h header for fresh bytecode compilation signatures coming soon.  r=tcampbell
https://hg.mozilla.org/integration/mozilla-inbound/rev/24670a08843e
Remove the original CompileGlobalScript overload, and rewrite all users to use the new (GlobalScriptInfo&, SourceText<char16_t>&, ScriptSourceObject**) overload.  r=tcampbell
https://hg.mozilla.org/integration/mozilla-inbound/rev/9d9b2d6342f7
Remove the original CompileEvalScript overload, and rewrite all users to use a new (EvalScriptInfo&, SourceText<char16_t>&) overload that calls a UTF-8/16-ready template function.  r=tcampbell
https://hg.mozilla.org/integration/mozilla-inbound/rev/a005d87f9e92
Convert the two CompileModule functions to template functions, then call them inside non-template overloads.  r=tcampbell
https://hg.mozilla.org/integration/mozilla-inbound/rev/280587e10c97
Make StandaloneFunctionCompiler::parse's |MutableHandleFunction fun| argument merely a |HandleFunction|, and make the compile function take its |MutableHandleFunction| argument first, not last.  r=tcampbell as part of prior review comments

Comment 50

[Arthur Iakab [arthur_iakab]]

https://hg.mozilla.org/mozilla-central/rev/43b8810bb3ed
https://hg.mozilla.org/mozilla-central/rev/76feab5328ed
https://hg.mozilla.org/mozilla-central/rev/3ed7e8a37562
https://hg.mozilla.org/mozilla-central/rev/9dc874d79c1f
https://hg.mozilla.org/mozilla-central/rev/630a16450066
https://hg.mozilla.org/mozilla-central/rev/532b05c76fa0
https://hg.mozilla.org/mozilla-central/rev/422924c19ce5
https://hg.mozilla.org/mozilla-central/rev/9d7d25507cac
https://hg.mozilla.org/mozilla-central/rev/ecf813f9f9ea
https://hg.mozilla.org/mozilla-central/rev/ba4bdabbdd52
https://hg.mozilla.org/mozilla-central/rev/774a0684f724
https://hg.mozilla.org/mozilla-central/rev/ba6ba95b3cd2
https://hg.mozilla.org/mozilla-central/rev/1dd5267839b3
https://hg.mozilla.org/mozilla-central/rev/24670a08843e
https://hg.mozilla.org/mozilla-central/rev/9d9b2d6342f7
https://hg.mozilla.org/mozilla-central/rev/a005d87f9e92
https://hg.mozilla.org/mozilla-central/rev/280587e10c97