1379578 - 8 - 28% Strings PerfStripCRLF/PerfStripWhitespace (windows7/linux) regressions on push a5c0a74876ae (Tue June 27 2017)

Status: RESOLVED FIXED

(Core :: XPCOM, defect)

Description

[Ionuț Goldan [:igoldan]]

Talos has detected a Firefox performance regression from push:

https://hg.mozilla.org/integration/mozilla-inbound/pushloghtml?fromchange=6419ffd3458f1addf65bf0c11842009cf219a7c4&tochange=a5c0a74876aeef09e624059dd189b7a72e6b1387

As author of one of the patches included in that push, we need your help to address this regression.

Regressions:

 28%  Strings PerfStripCRLF windows7-32-vm opt      106,213.25 -> 136,344.92
 28%  Strings PerfStripWhitespace windows7-32-vm opt 90,907.58 -> 116,530.17
 22%  Strings PerfStripCRLF linux32 opt             122,132.17 -> 149,373.83
 19%  Strings PerfStripWhitespace linux32 opt       118,321.33 -> 140,965.92
  8%  Strings PerfStripCRLF linux64 pgo             113,587.58 -> 122,663.58
  8%  Strings PerfStripWhitespace linux64 pgo       98,952.08 -> 106,787.83

Improvements:

 35%  Strings PerfStripWhitespace osx-10-10 opt      139,193.33 -> 91,165.92
 23%  Strings PerfStripWhitespace windows8-64 opt    126,939.17 -> 97,246.75
 23%  Strings PerfStripCRLF windows8-64 opt          133,609.08 -> 103,004.00
 14%  Strings PerfStripCRLF linux64 asan             260,616.67 -> 225,175.17
 14%  Strings PerfStripWhitespace linux64 asan       222,966.58 -> 192,698.58

You can find links to graphs and comparison views for each of the above tests at: https://treeherder.mozilla.org/perf.html#/alerts?id=7555

On the page above you can see an alert for each affected platform as well as a link to a graph showing the history of scores for this test. There is also a link to a treeherder page showing the Talos jobs in a pushlog format.

To learn more about the regressing test(s), please see: https://wiki.mozilla.org/Buildbot/Talos/Tests

For information on reproducing and debugging the regression, either on try or locally, see: https://wiki.mozilla.org/Buildbot/Talos/Running

*** Please let us know your plans within 3 business days, or the offending patch(es) will be backed out! ***

Our wiki page outlines the common responses and expectations: https://wiki.mozilla.org/Buildbot/Talos/RegressionBugsHandling

Comment 1

[Ionuț Goldan [:igoldan]]

:ehsan Could you look over these regressions and then estimate a fix time for them?

Comment 2

[Eric Rahm [:erahm]]

Ehsan is out for a week, Nathan who reviewed the patches is out for 2 weeks. I think we should just go ahead and backout for now and let them sort it out when they get back.

Interestingly it looks like this did help perf on 64-bit (except linux pgo, I don't see windows pgo listed) but regressed a fair amount on 32-bit.

Comment 3

[Ionuț Goldan [:igoldan]]

:erahm I would need to backout 2 bugs: bug 1376323 and bug 1376472 also, which wrote over the regressive code. Are you fine with that?

Comment 4

[Ionuț Goldan [:igoldan]]

This is the backout of the 2 bugs on Try:
https://treeherder.mozilla.org/#/jobs?repo=try&revision=4177c6d2bec8807dd3d18494d53089a0853c33c0

Comment 5

[Eric Rahm [:erahm]]

(In reply to Ionuț Goldan [:igoldan], Performance Sheriffing from comment #3)
> :erahm I would need to backout 2 bugs: bug 1376323 and bug 1376472 also,
> which wrote over the regressive code. Are you fine with that?

That's fine.

Comment 6

[Joel Maher ( :jmaher ) (UTC -8)]

it looks like the next steps are to backout the 2 bugs, :igoldan, can you confirm that fixes this and then get the sheriffs to help out with that?

Comment 7

[Ionuț Goldan [:igoldan]]

The two bugs mentioned in comment 3 have been backed out on July 13th:
https://bugzilla.mozilla.org/show_bug.cgi?id=1376323#c6
https://bugzilla.mozilla.org/show_bug.cgi?id=1376472#c7


I confirm the backout fixed the regressions. This partial alert summary, landed after the backout, shows it:

== Change summary for alert #7937 (as of July 13 2017 14:47 UTC) ==

Regressions:

 16%  Strings PerfStripCRLF linux64 asan      224,383.92 -> 260,907.08
 16%  Strings PerfStripWhitespace linux64 asan 192,487.25 -> 222,969.67

Improvements:

 18%  Strings PerfStripCRLF linux32 opt      149,070.75 -> 122,240.75
 15%  Strings PerfStripWhitespace linux32 opt 140,636.75 -> 119,262.17
  8%  Strings PerfStripWhitespace linux64 pgo 107,173.75 -> 98,220.08
  8%  Strings PerfStripCRLF linux64 pgo      122,855.33 -> 112,862.33

For up to date results, see: https://treeherder.mozilla.org/perf.html#/alerts?id=7937

Comment 8

[Ryan VanderMeulen [:RyanVM]]

Not sure why this was left open since it was fixed by backout.