If you think a bug might affect users in the 57 release, please set the correct tracking and status flags for Release Management.
Bug 1304984 (ares-6)

Optimize WebKit's ARES-6 benchmark

NEW
Unassigned

Status

()

Core
JavaScript Engine
a year ago
2 months ago

People

(Reporter: tetsuharu, Unassigned)

Tracking

(Depends on: 5 bugs, Blocks: 1 bug, {perf})

Trunk
Points:
---
Dependency tree / graph

Firefox Tracking Flags

(firefox52 wontfix)

Details

Attachments

(1 attachment)

The result of /PerformanceTests/ES6SampleBench/index.html in WebKit's tree is very slower (~5x) than the result of WebKit.

## Benchmark codes

- https://github.com/WebKit/webkit/tree/d2679d89a5417abd1f4b89501f88dba11bf08d25/PerformanceTests/ES6SampleBench

## Measurement envionment

  - MacBook Pro (Retina, 15-inch, Mid 2014), 2.8 GHz Intel Core i7, 16 GB 1600 MHz DDR3
  - OSX 10.11.6
  - x64

## Results

### Firefox Nightly 52.0a1, 20160920030429


Geometric Mean Result: 579.33 ms +- 10.15 ms

| Benchmark |First Iteration       | Worst 2%              | Steady State             |
|-----------|----------------------|---------------------- |--------------------------|
| Air       | 195.29 ms +- 1.93 ms | 123.89 ms +- 14.82 ms | 11416.11 ms +- 237.49 ms |
| Basic     | 105.67 ms +- 3.53 ms | 98.28 ms +- 9.08 ms   | 13452.75 ms +- 520.96 ms |




### Chrome Canary 55.0.2866.0 canary (64-bit)


Geometric Mean Result: 398.87 ms +- 9.03 ms

| Benchmark |First Iteration     | Worst 2%          | Steady State           |
|-----------|--------------------|-------------------|------------------------|
|Air        |152.76 ms +- 5.49 ms|59.35 ms +- 3.36 ms|6956.44 ms +- 33.39 ms  |
|Basic      |78.34 ms +- 1.95 ms |71.23 ms +- 5.66 ms|11525.40 ms +- 169.84 ms|


### WebKit Nightly r206163


Geometric Mean Result: 113.36 ms +- 3.36 ms

| Benchmark |First Iteration    | Worst 2%          | Steady State         |
|-----------|-------------------|-------------------|----------------------|
|Air        |53.88 ms +- 6.67 ms|28.98 ms +- 2.53 ms|2364.88 ms +- 61.77 ms|
|Basic      |26.78 ms +- 0.85 ms|14.56 ms +- 0.79 ms|1500.21 ms +- 21.12 ms|
Created attachment 8794340 [details]
ES6SampleBench.zip

Thanks for pointing this out to us, I am always interested in ES6 benchmarks

I tried modifying cli.js to work with our shell, but I think using newGlobal() + evaluate() probably causes us to unnecessarily recompile stuff, which would skew our results.
Tracelogger suggest most time is spend in Baseline, but again that might be because of evaluate.
IONFLAGS=aborts shows various aborts for spreadcallarray, pushlexicalenv and generators.
Flags: needinfo?(jdemooij)
Depends on: 1273858
Blocks: 1307395
I think the newest version of this https://github.com/WebKit/webkit/tree/master/PerformanceTests/ARES-6. Almost looks like something they want to release publicly.

Updated

7 months ago
Depends on: 1338920
Keywords: perf
Summary: Slower 5x than WebKit in WebKit's /PerformanceTests/ES6SampleBench/index.html → Optimize WebKit's ARES-6 benchmark

Updated

7 months ago
Depends on: 1341087

Updated

7 months ago
Depends on: 1341263

Updated

7 months ago
Alias: ares-6

Updated

7 months ago
Depends on: 1341265
Depends on: 1341937
Clearing NI. Tom and Ted have been working on this.
Flags: needinfo?(jdemooij)

Updated

6 months ago
Depends on: 1317690
Depends on: 1353358
Too late for firefox 52, mass-wontfix.
status-firefox52: affected → wontfix
Depends on: 1169746
This is now a public benchmark: http://browserbench.org/ARES-6/

And there's a blog post where Firefox compares very unfavorably against Chrome and Safari:
https://webkit.org/blog/7536/jsc-loves-es6/
Depends on: 1378189

Updated

2 months ago
Depends on: 1380281
You need to log in before you can comment on or make changes to this bug.