2.99 - 10.36% raptor-tp6 (linux64, windows10-64) regression on push 3cddc7cd4da5fcd26b93becb55a028621ab68f64 (Fri Mar 8 2019)

REOPENED
Assigned to

Status

()

defect
P2
normal
REOPENED
2 months ago
a month ago

People

(Reporter: igoldan, Assigned: baku)

Tracking

(Blocks 1 bug, {perf, regression})

Trunk
mozilla68
Unspecified
All
Points:
---
Dependency tree / graph

Firefox Tracking Flags

(firefox67 wontfix, firefox68 fixed)

Details

(Whiteboard: [domsecurity-active])

Attachments

(2 attachments)

Raptor has detected a Firefox performance regression from push:

https://hg.mozilla.org/integration/autoland/pushloghtml?changeset=3cddc7cd4da5fcd26b93becb55a028621ab68f64

As author of one of the patches included in that push, we need your help to address this regression.

Regressions:

10% raptor-tp6-microsoft-firefox fcp windows10-64 pgo 482.17 -> 532.12
10% raptor-tp6-wikia-firefox linux64 pgo 164.93 -> 181.44
5% raptor-tp6-wikia-firefox loadtime linux64 pgo 183.94 -> 192.29
5% raptor-tp6-imdb-firefox loadtime linux64 pgo 522.06 -> 545.62
4% raptor-tp6-imgur-firefox fcp linux64 pgo 271.75 -> 281.92
3% raptor-tp6-amazon-firefox fcp windows10-64 pgo 255.77 -> 264.54
3% raptor-tp6-imgur-firefox linux64 pgo 294.21 -> 303.02

Improvements:

6% raptor-assorted-dom-firefox linux64 pgo 26.69 -> 25.10

You can find links to graphs and comparison views for each of the above tests at: https://treeherder.mozilla.org/perf.html#/alerts?id=19829

On the page above you can see an alert for each affected platform as well as a link to a graph showing the history of scores for this test. There is also a link to a Treeherder page showing the Raptor jobs in a pushlog format.

To learn more about the regressing test(s) or reproducing them, please see: https://wiki.mozilla.org/Performance_sheriffing/Raptor

*** Please let us know your plans within 3 business days, or the offending patch(es) will be backed out! ***

Our wiki page outlines the common responses and expectations: https://wiki.mozilla.org/Performance_sheriffing/Talos/RegressionBugsHandling

Component: General → DOM: Security
Product: Testing → Core
Flags: needinfo?(amarchesini)
(Assignee)

Comment 2

2 months ago

I'm debugging this regression. I'll tell you more about the time range for the fix for today, max tomorrow.

(Assignee)

Updated

2 months ago
Assignee: nobody → amarchesini
Flags: needinfo?(amarchesini)
Status: NEW → ASSIGNED
Priority: -- → P2
Whiteboard: [domsecurity-active]

Comment 4

2 months ago
Pushed by amarchesini@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/33d53c6ee136
Avoid sending CookieSettings from parent to child if not needed, r=Ehsan

Comment 5

2 months ago
bugherder
Status: ASSIGNED → RESOLVED
Last Resolved: 2 months ago
Resolution: --- → FIXED
Target Milestone: --- → mozilla67

I'm afraid push from comment 4 didn't fix the regressions. I only saw a partial fix on raptor-tp6-amazon-firefox fcp windows10-64-pgo-qr, but the other baselines remained unchanged.

Status: RESOLVED → REOPENED
Resolution: FIXED → ---
(Assignee)

Updated

2 months ago
Flags: needinfo?(amarchesini)

I also noticed a fix for raptor-tp6-microsoft-firefox fcp linux64 pgo:

== Change summary for alert #19948 (as of Mon, 18 Mar 2019 06:58:24 GMT) ==

Improvements:

4% raptor-tp6-microsoft-firefox fcp linux64 pgo 511.06 -> 488.92

For up to date results, see: https://treeherder.mozilla.org/perf.html#/alerts?id=19948

This is the Compare view for the new patch Andrea is working on. Keeping it here for easy reference.

(Assignee)

Updated

2 months ago
Flags: needinfo?(amarchesini)

Comment 10

2 months ago
Pushed by amarchesini@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/66501cf7b6a4
Principal serialization should not serialize base domain if not needed, r=Ehsan

Comment 11

2 months ago
bugherder
Status: REOPENED → RESOLVED
Last Resolved: 2 months ago2 months ago
Resolution: --- → FIXED
Target Milestone: mozilla67 → mozilla68

Push from comment 10 still didn't fix the perf regressions. All of them remained the same.
There's lots of noise, especially after March 29, which makes the changes even more difficult to identify.

Status: RESOLVED → REOPENED
Resolution: FIXED → ---
(Assignee)

Comment 13

2 months ago

Can you tell me the % of regression after comment 10?

Flags: needinfo?(igoldan)

(In reply to Andrea Marchesini [:baku] from comment #13)

Can you tell me the % of regression after comment 10?

Yes, but you'll have to wait a bit so I can do some retriggers to obtain that percentage.

I've done the retriggers. In 3 ours I should have enough data to reply back.

I looked over the retriggers and and it's difficult to precisely state in percentages how much we've fixed.
The noise & big harness update around March 29 (which should mix into the equation) hide the small gains.

Flags: needinfo?(igoldan)

Marking as wontfix for 67 as we are not sure that the performance regressions would be addressed by the patches in this bug and we are in the middle of the nightly cycle already.

You need to log in before you can comment on or make changes to this bug.