1554968 - 4.37 - 4.38% raptor-speedometer-firefox (linux64-shippable, linux64-shippable-qr) regression on push 37fb14fc763aa87699c0c00157e00546333186d8 (Mon May 27 2019)

Status: RESOLVED WONTFIX

(Firefox Build System :: General, defect)

Description

[Florin Strugariu [:Bebe]]

Raptor has detected a Firefox performance regression from push:

https://hg.mozilla.org/integration/autoland/pushloghtml?changeset=37fb14fc763aa87699c0c00157e00546333186d8

As author of one of the patches included in that push, we need your help to address this regression.

Regressions:

4% raptor-speedometer-firefox linux64-shippable-qr opt 91.62 -> 87.60
4% raptor-speedometer-firefox linux64-shippable opt 92.62 -> 88.58

You can find links to graphs and comparison views for each of the above tests at: https://treeherder.mozilla.org/perf.html#/alerts?id=21131

On the page above you can see an alert for each affected platform as well as a link to a graph showing the history of scores for this test. There is also a link to a Treeherder page showing the Raptor jobs in a pushlog format.

To learn more about the regressing test(s) or reproducing them, please see: https://wiki.mozilla.org/Performance_sheriffing/Raptor

*** Please let us know your plans within 3 business days, or the offending patch(es) will be backed out! ***

Our wiki page outlines the common responses and expectations: https://wiki.mozilla.org/Performance_sheriffing/Talos/RegressionBugsHandling

Comment 1

[:Gijs (he/him)]

This is delightful, especially given that I pushed to try specifically to make sure this wasn't gonna happen. Anyway, let's back out and we'll sort things out after (I've asked sheriffs to deal with the backout).

Comment 2

[:Gijs (he/him)]

Also please please please get whoever works on the software that generates your comments to prioritize fixing the links and/or allowing you to omit them from the comment. Comments that go "hi this push regressed a thing" that then point somewhere completely different waste my time.

Comment 3

[:Gijs (he/him)]

Can this be closed now that the regressing cset has been backed out? ( https://treeherder.mozilla.org/perf.html#/alerts?id=21137 )

Comment 4

[Florin Strugariu [:Bebe]]

== Change summary for alert #21137 (as of Tue, 28 May 2019 20:12:07 GMT) ==

Improvements:

5% raptor-speedometer-firefox linux64-shippable opt 88.49 -> 92.74
4% raptor-speedometer-firefox linux64-shippable-qr opt 87.43 -> 90.83

For up to date results, see: https://treeherder.mozilla.org/perf.html#/alerts?id=21137

Comment 5

[Florin Strugariu [:Bebe]]

(In reply to :Gijs (he/him) from comment #2)

Also please please please get whoever works on the software that generates your comments to prioritize fixing the links and/or allowing you to omit them from the comment. Comments that go "hi this push regressed a thing" that then point somewhere completely different waste my time.

Perfherder is going through a migration process now... All of these issues will be fixed hopefully.

Comment 6

[Dave Hunt [:davehunt] [he/him] ⌚BST]

(In reply to :Gijs (he/him) from comment #2)

Also please please please get whoever works on the software that generates your comments to prioritize fixing the links and/or allowing you to omit them from the comment. Comments that go "hi this push regressed a thing" that then point somewhere completely different waste my time.

Thanks for the feedback. Which link caused the issue?

Comment 7

[:Gijs (he/him)]

(In reply to Dave Hunt [:davehunt] [he/him] ⌚️UTC from comment #6)

(In reply to :Gijs (he/him) from comment #2)

Also please please please get whoever works on the software that generates your comments to prioritize fixing the links and/or allowing you to omit them from the comment. Comments that go "hi this push regressed a thing" that then point somewhere completely different waste my time.

Thanks for the feedback. Which link caused the issue?

(In reply to Florin Strugariu [:Bebe] from comment #0)

Raptor has detected a Firefox performance regression from push:

https://hg.mozilla.org/integration/autoland/pushloghtml?changeset=37fb14fc763aa87699c0c00157e00546333186d8

This pushlog doesn't correspond to the regression and the bug that caused this regression. In a previous bug ( https://bugzilla.mozilla.org/show_bug.cgi?id=1552425#c2 ) I was told this was a perfherder bug.

Comment 8

[Ionuț Goldan [:igoldan]]

Seems like latest push from bug 1196094 caused these build time increases on Linux 32bit.

== Change summary for alert #21246 (as of Mon, 03 Jun 2019 09:23:59 GMT) ==

Regressions:

10% build times linux32-shippable opt nightly taskcluster-c4.4xlarge 6,256.58 -> 6,860.88
9% build times linux32-shippable opt nightly taskcluster-c5d.4xlarge 5,411.84 -> 5,891.77

For up to date results, see: https://treeherder.mozilla.org/perf.html#/alerts?id=21246

Comment 9

[:Gijs (he/him)]

(In reply to Ionuț Goldan [:igoldan], Performance Sheriff from comment #8)

Seems like latest push from bug 1196094 caused these build time increases on Linux 32bit.

== Change summary for alert #21246 (as of Mon, 03 Jun 2019 09:23:59 GMT) ==

Regressions:

10% build times linux32-shippable opt nightly taskcluster-c4.4xlarge 6,256.58 -> 6,860.88
9% build times linux32-shippable opt nightly taskcluster-c5d.4xlarge 5,411.84 -> 5,891.77

For up to date results, see: https://treeherder.mozilla.org/perf.html#/alerts?id=21246

Rather than clearing all the tracking for the extant issue, this would have been better as a new bug, as it's not the same issue at all...

Anyway, just like in bug 1555298, I don't think we should worry about this. I'd prefer to mark this bug fixed again but I'm aware we didn't do anything about the build times so perhaps we should mark it wontfix? It'll skew your tracking of perf regressions and how we resolve them either way - let me know what you prefer.

Comment 11

[Ionuț Goldan [:igoldan]]

Forgot to remove ni? from myself