1177735 - (six-speed) Investigate six-speed performance

Status: NEW

Description

[Jan de Mooij [:jandem]]

I just saw this on Twitter: http://kpdecker.github.io/six-speed/

It compares performance of ES6 features to ES5.

Comment 1

[Guilherme Lima]

The author filled 2 bugs: bug 1177319 and bug 1177518

Comment 2

[Jan de Mooij [:jandem]]

(In reply to Guilherme Lima from comment #1)
> The author filled 2 bugs: bug 1177319 and bug 1177518

Thanks! I didn't realize these are related.

The arrow-args test is silly; our |arguments| inside arrows is not lexically bound so we do a ton of extra work...

Comment 3

[Jan de Mooij [:jandem]]

(In reply to Jan de Mooij [:jandem] from comment #2)
> The arrow-args test is silly; our |arguments| inside arrows is not lexically
> bound so we do a ton of extra work...

(Also because the arguments[0] is out of bounds with our bogus arguments binding, so we're not using lazy arguments there.)

Comment 4

[Jan de Mooij [:jandem]]

Clearing the NI. I'm working on some of these bugs.

Comment 5

[André Bargull [:anba]]

The following patch to the arewefastyet version of six-speed improves our score in multiple sub-benchmarks remarkably:

------------------------------------------------------------------------
diff --git a/benchmarks/six-speed/test.js b/benchmarks/six-speed/test.js
index e50af51..7578b2f 100644
--- a/benchmarks/six-speed/test.js
+++ b/benchmarks/six-speed/test.js
@@ -1,6 +1,6 @@
 
 function assertEqual() {}
-function test(fn) {
+var testTemplate = function test(fn) {
     var its = iterations;
     var start = Date.now();
     for (var i = 0; i < its; i++) {
@@ -73,12 +73,14 @@ for (var z = 0; z < tests.length; z++) {
 
     // ES5
     var content = read("tests/"+testname+".es5")
+    content = testTemplate.toString() + content;
     var func = new Function(content);
     func();
     print(testname+"-es5:", timing);
 
     // ES6
     var content = read("tests/"+testname+".es6")
+    content = testTemplate.toString() + content;
     var func = new Function(content);
     func();
     print(testname+"-es6:", timing);
------------------------------------------------------------------------

This benchmark is so broken. :-/

Comment 6

[Boris Zbarsky [:bzbarsky, bz on IRC]]

Do we end up failing to optimize the very-polymorphic test() function well and end up with some sub-benchmarks dominated by harness code, not test code?

Comment 7

[André Bargull [:anba]]

(In reply to Boris Zbarsky [:bz] (no decent commit message means r-) from comment #6)
> Do we end up failing to optimize the very-polymorphic test() function well
> and end up with some sub-benchmarks dominated by harness code, not test code?

Well, yes. With the changes from comment #5 some sub-benchmarks are even entirely optimized away.

                          baseline      unique test()
                         es5   es6       es5   es6
---------------------------------------------------
template_string:        [168,  948]     [168,  171]
defaults:               [293,  307]     [ 85,   85]
map-set-lookup:         [339,   26]     [338,   23]
spread:                 [146,  208]     [135,  201]
object-assign:          [262,  351]     [215,  342]
spread-literal:         [125,  187]     [126,  175]
map-set:                [ 68,  686]     [ 72,  681]
destructuring-simple:   [188,  191]     [ 19,   18]
super:                  [115, 2678]     [ 49, 2601]
for-of-object:          [ 91,  335]     [ 81,  314]
rest:                   [358,   30]     [349,   29]
regex-u:                [ 54,  144]     [ 53,  143]
arrow:                  [216,  214]     [ 18,   18]
bindings-compound:      [171,  171]     [ 18,   18]
classes:                [169,  202]     [ 81,   84]
template_string_tag:    [201,  185]     [  5,    4]
map-string:             [364, 1656]     [ 26, 1370]
arrow-declare:          [290,  291]     [ 26,   27]
spread-generator:       [218,  877]     [206,  836]
object-literal-ext:     [125,  144]     [ 95,  100]
generator:              [162, 1705]     [128, 1594]
arrow-args:             [357,  359]     [190,  192]
for-of-array:           [497,  648]     [426,  631]
bindings:               [181,  202]     [ 19,   18]
destructuring:          [214,  909]     [ 17,  682]
map-set-object:         [171,  675]     [168,  678]