Closed Bug 1571624 Opened 4 months ago Closed 4 months ago

5.07 - 21.57% Images (linux64-shippable-qr, windows10-64-shippable-qr) regression on push 37229cef2cc79d44470afc9e04016bac8ddd0ae8 (Fri August 2 2019)

Categories

(Firefox :: New Tab Page, defect, P2)

defect

Tracking

()

RESOLVED FIXED
Firefox 70
Tracking Status
firefox-esr60 --- unaffected
firefox-esr68 --- unaffected
firefox68 --- unaffected
firefox69 --- wontfix
firefox70 --- fixed

People

(Reporter: alexandrui, Assigned: Mardak)

References

(Blocks 1 open bug, Regression)

Details

(Keywords: perf, regression)

Attachments

(1 file)

We have detected an awsy regression from push:

https://hg.mozilla.org/integration/autoland/pushloghtml?changeset=37229cef2cc79d44470afc9e04016bac8ddd0ae8

As author of one of the patches included in that push, we need your help to address this regression.

Regressions:

22% Images linux64-shippable-qr opt stylo tp6 8,529,808.50 -> 10,369,912.98
19% Images windows10-64-shippable-qr opt 7,496,049.95 -> 8,951,452.91
5% Images linux64-shippable-qr opt 5,927,519.55 -> 6,227,785.36

You can find links to graphs and comparison views for each of the above tests at: https://treeherder.mozilla.org/perf.html#/alerts?id=22270

On the page above you can see an alert for each affected platform as well as a link to a graph showing the history of scores for this test. There is also a link to a treeherder page showing the jobs in a pushlog format.

To learn more about the regressing test(s), please see: https://wiki.mozilla.org/AWSY/Tests

Flags: needinfo?(khudson)
Component: General → New Tab Page
Product: Testing → Firefox
Version: Version 3 → unspecified

Can we please get the profile information from those tests?

Flags: needinfo?(alexandru.ionescu)
Flags: needinfo?(khudson)

(In reply to Andrei Oprea [:andreio] from comment #1)

Can we please get the profile information from those tests?

Unfortunately, we don't provide Gecko profiles for AWSY perf tests.

(In reply to Andrei Oprea [:andreio] from comment #1)

Can we please get the profile information from those tests?

If it helps in any way, Bug 1570745 was backed out once here

Flags: needinfo?(alexandru.ionescu)

The runs do clearly show it is related to the export bug 1570745:

https://treeherder.mozilla.org/#/jobs?repo=autoland&tochange=0928c6245907b29a632be53031c0f4ea358151ee&searchStr=%28sy%2C10-64%2Cqr&fromchange=ee3b704f5009d1234dba2bb36914d11230724926

7.6 before initial landing
8.9 after initial landing
7.5 after backout
9.2 after relanding

Similar to bug 1572682, this is because we are now showing triplets images on about:home/newtab for new users. Ideally we have a new reference "with triplets" and "without triplets" where the former would be the new baseline and latter should match pre-regression levels.

The "without triplets" can be done now by running talos with trailhead.firstrun.branches=join pref to match the common case behavior of users not seeing triplets. But it would still be good to know how performance changes for new users who will be seeing triplets.

See Also: → 1572682
Pushed by elee@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/6ddb703d8e53
Switch awsy tests to measure common case non-triplets new tab page r=erahm
Status: NEW → RESOLVED
Closed: 4 months ago
Resolution: --- → FIXED
Target Milestone: --- → Firefox 70
Assignee: nobody → edilee

== Change summary for alert #22445 (as of Wed, 14 Aug 2019 11:01:51 GMT) ==

Improvements:

20% Images linux64-shippable-qr opt stylo tp6 10,303,742.84 -> 8,202,356.50
12% Images windows10-64-shippable-qr opt 8,384,749.27 -> 7,413,176.62
6% Images linux64-shippable opt 4,978,494.14 -> 4,668,896.24
5% Images macosx1014-64-shippable opt 4,806,900.02 -> 4,576,701.82
5% Images windows10-64-shippable opt 5,768,974.36 -> 5,505,433.90

For up to date results, see: https://treeherder.mozilla.org/perf.html#/alerts?id=22445

Duplicate of this bug: 1576098
Duplicate of this bug: 1576102

Probably not worthwhile uplifting the "fix" to beta/rc as it doesn't change the user facing behavior.

You need to log in before you can comment on or make changes to this bug.