Bug 1339557 (Speedometer_V2)

[Meta] Quantum Release Criteria: Figure out why we are so slow on Speedometer v2

NEW
Unassigned

Status

()

defect
P2
normal
2 years ago
2 months ago

People

(Reporter: Ehsan, Unassigned)

Tracking

(Depends on 19 bugs, Blocks 1 bug, {meta, perf})

Firefox Tracking Flags

(Not tracked)

Details

(Whiteboard: [qf:meta], )

Reporter

Description

2 years ago
See <https://twitter.com/bz_moz/status/831585613258379265>

We have looked at this before a bit, see bug 1245279 and bug 1022988.  We need to profile again and find out why we are so slow.  This is important since this test executes a bunch of popular frameworks.

Andrew, can we find an owner for this please?
Flags: needinfo?(overholt)
Let me work on it.
Reporter

Updated

2 years ago
Depends on: 1339593
Reporter

Comment 2

2 years ago
I spent a bit of time looking at some random profiles from a long Speedometer run and all I can see is JS.  Perhaps someone on the JS team would be a better candidate.
Component: DOM → JavaScript Engine
Flags: needinfo?(nihsanullah)
Depends on: 1245279

Comment 3

2 years ago
Marked improvement overnight after fix via bug: 1339535

My fiber today is shocking for some reason, but still seeing a ~35% speed improvement. Testing in another location in around an hour.

Arch Linux rolling release. Nightly 2017-02-28 vs 2017-02-27. Dell Latitude E4300 16Gb RAM.
Flags: needinfo?(nihsanullah)
Whiteboard: [qf:investigate][qf:p1]
Flags: needinfo?(overholt)
Depends on: 1346723
Depends on: 1347525

Comment 4

2 years ago
Despite improvements to Nightly, which I've experienced on my laptop since Feb 27, testing is now shockingly slow on a Galaxy Tab A (9.7) running Fx55. Speedometer test been running for 45 mins and not yet half way :(

Updated

2 years ago
Whiteboard: [qf:investigate][qf:p1] → [qf:investigate][qf:meta]
Alias: Speedometer_V2
Keywords: meta
Summary: Figure out why we are so slow on Speedometer → [Meta] Figure out why we are so slow on Speedometer v2
Depends on: 1356315
Depends on: 1364854
Depends on: 1346546
Depends on: 1364908
Depends on: 1349924
Depends on: 1022714
Depends on: 1022988
Depends on: 1245294
Depends on: 1355472
Depends on: 1363438
Can we put a link to the v2 speedometer in this bug?  Not sure where it lives.
I added it to the URL field of this bug.
Depends on: 1363829

Updated

2 years ago
No longer blocks: TimeToFirstPaint_FB
Reporter

Updated

2 years ago
Depends on: 1366803
Depends on: 903519
Reporter

Updated

2 years ago
Depends on: 1367458
Reporter

Updated

2 years ago
Depends on: 1349255
Reporter

Updated

2 years ago
Depends on: 1338802

Updated

2 years ago
Alias: Speedometer_V2 → QRC_Speedometer_V2
Summary: [Meta] Figure out why we are so slow on Speedometer v2 → [Meta] Quantum Release Criteria: Figure out why we are so slow on Speedometer v2
Depends on: 1351769
Depends on: 1365361
Depends on: 1366263
Depends on: 1366375
Depends on: 1366377

Updated

2 years ago
Alias: QRC_Speedometer_V2 → Speedometer_V2
Reporter

Updated

2 years ago
Depends on: 1364805
No longer blocks: QuantumFlow

Comment 7

2 years ago
According to bug 1347525 slow InnerHTML is the main reason for Firefox's slowness in Speedometer. This was resolved, but Speedometer is still at best only half as fast as in Chrome for me. 

With an i7 6700k I get 199 runs/minute in Chrome, 98 runs/minute in Firefox. This scales down with CPU power, on an i5 6200U I get 42 runs/minute with Firefox, 88 with Chrome.

How relevant is Speedometer for Gecko, anyway? V8 describes it as a great tool for measuring real world performance, however in bug 1347525 comment #3 seems to suggest otherwise.
(In reply to TMart from comment #7)
> According to bug 1347525 slow InnerHTML is the main reason for Firefox's
> slowness in Speedometer.

I never said that. In that bug I was talking about one of the subtests, but there are a lot of these and we knew fixing that bug would not magically fix everything.

> With an i7 6700k I get 199 runs/minute in Chrome, 98 runs/minute in Firefox.
> This scales down with CPU power, on an i5 6200U I get 42 runs/minute with
> Firefox, 88 with Chrome.

This might be better with Speedometer v2.

> How relevant is Speedometer for Gecko, anyway? V8 describes it as a great
> tool for measuring real world performance, however in bug 1347525 comment #3
> seems to suggest otherwise.

There are a bunch of silly things for sure. Overall it's not an unreasonable benchmark though and so far almost all performance issues we found/fixed will help real-world websites too.
Duplicate of this bug: 1022239
Depends on: 1369140
No longer depends on: 1363829
Depends on: 1364816
Reporter

Updated

2 years ago
Depends on: 1369587
Reporter

Updated

2 years ago
No longer depends on: 1369587

Updated

2 years ago
Blocks: QRC_FX57

Updated

2 years ago
No longer blocks: Quantum
Reporter

Updated

2 years ago
Depends on: 1371097
Reporter

Updated

2 years ago
Depends on: 1372794
Reporter

Updated

2 years ago
Depends on: 1372974
Reporter

Updated

2 years ago
Depends on: 1373063
Reporter

Updated

2 years ago
Depends on: 1373672
Depends on: 1373723
Reporter

Updated

2 years ago
Depends on: 1374033
Reporter

Updated

2 years ago
Depends on: 1374416
Depends on: 1373079
Reporter

Updated

2 years ago
Depends on: 1374892
Reporter

Updated

2 years ago
Depends on: 1375568
The machine that runs speedometer got this morning a change that will make the score of speedometer improve (bug 1373396).

I don't know how it will affect the score of Chrome but we will know later today. AWFY is currently running:
https://arewefastyet.com/#machine=36&view=breakdown&suite=speedometer-misc

Also note that we currently run non-PGO builds for speedometer which could mean that our score could even be a bit higher.
Reporter

Updated

2 years ago
Depends on: 1375573
Reporter

Updated

2 years ago
Depends on: 1375599
Reporter

Updated

2 years ago
Depends on: 1375619
Reporter

Updated

2 years ago
Depends on: 1375331
Reporter

Updated

2 years ago
Depends on: 1375701
Reporter

Updated

2 years ago
Depends on: 1375741
Depends on: 1373396
Reporter

Updated

2 years ago
Depends on: 1376323
Reporter

Updated

2 years ago
Depends on: 1365970
Reporter

Updated

2 years ago
Depends on: 1376544
Reporter

Updated

2 years ago
Depends on: 1376580
Reporter

Updated

2 years ago
Depends on: 1376594
Depends on: 1376603
Reporter

Updated

2 years ago
Depends on: 1343005
Reporter

Updated

2 years ago
Depends on: 1376695
Depends on: 1376698
Reporter

Updated

2 years ago
Depends on: 1376936
Depends on: 1377264
Reporter

Updated

2 years ago
Depends on: 1377818
Reporter

Updated

2 years ago
Depends on: 1377999
I see PollWrapper() in profiles, has anyone tried to understand why? Like is it the test designed to short break between runs, or is it firefox waiting on something?
Reporter

Comment 12

2 years ago
(In reply to Ting-Yu Chou [:ting] from comment #11)
> I see PollWrapper() in profiles, has anyone tried to understand why? Like is
> it the test designed to short break between runs, or is it firefox waiting
> on something?

The test schedules a lot of asychronous work, such as loading web pages from the network, but it doesn't measure the entire time that it runs.  See this function for what the benchmark really measures: https://github.com/WebKit/webkit/blob/057552ca85b3efd9c97d13a0f044abd0daba7526/PerformanceTests/Speedometer/resources/benchmark-runner.js#L121
Reporter

Updated

2 years ago
Depends on: 1382914
Reporter

Updated

2 years ago
Depends on: 1384222
Depends on: 1383358
Reporter

Updated

2 years ago
Depends on: 1384588
Reporter

Updated

2 years ago
Depends on: 1384717
Reporter

Updated

2 years ago
Depends on: 1384798
Depends on: 1384840
Depends on: 1385165
Depends on: 1385181
Depends on: 1373476
The previous Speedtest v2 URL showed a "Not Found" page. I have been told that this is the correct URL to use (despite a page title of "Speedtest 1.0", which I'm told is incorrect).
Reporter

Updated

2 years ago
No longer depends on: 1384798
Reporter

Updated

2 years ago
Depends on: 714455
Reporter

Updated

2 years ago
Depends on: 1390626
Reporter

Updated

2 years ago
Depends on: 1390928
Reporter

Updated

2 years ago
No longer depends on: 714455
Reporter

Updated

2 years ago
Depends on: 1391298
Reporter

Updated

2 years ago
Depends on: 651120
Depends on: 1356652
Reporter

Updated

2 years ago
Depends on: 1392123
Reporter

Updated

2 years ago
Depends on: 1392888
Reporter

Updated

2 years ago
Depends on: 1392891
Reporter

Updated

2 years ago
Depends on: 1377560
Reporter

Updated

2 years ago
Depends on: 1392892
Depends on: 1392928
Reporter

Updated

2 years ago
Depends on: 1395727
Reporter

Updated

2 years ago
Depends on: 1392841
Reporter

Updated

2 years ago
Depends on: 1400095
Keywords: perf
Reporter

Updated

2 years ago
Depends on: 1356654
Priority: -- → P2
Depends on: 1445332

Updated

5 months ago
Whiteboard: [qf:investigate][qf:meta] → [qf:meta]
You need to log in before you can comment on or make changes to this bug.