1626620 - 3.65 - 3.85% displaylist_mutate (windows7-32-shippable) regression on push efa017cf034f4485591fd9afe0ba2960ffafadc0 (Fri March 27 2020)

Status: RESOLVED FIXED

(Firefox Build System :: Toolchains, defect, P3)

Description

[Alexandru Ionescu (needinfo me) [:alexandrui]]

Talos has detected a Firefox performance regression from push:

https://hg.mozilla.org/integration/autoland/pushloghtml?changeset=efa017cf034f4485591fd9afe0ba2960ffafadc0

As author of one of the patches included in that push, we need your help to address this regression.

Regressions:

4% displaylist_mutate windows7-32-shippable opt e10s stylo 1,377.74 -> 1,430.84
4% displaylist_mutate windows7-32-shippable opt e10s stylo 1,379.23 -> 1,429.60

You can find links to graphs and comparison views for each of the above tests at: https://treeherder.mozilla.org/perf.html#/alerts?id=25524

On the page above you can see an alert for each affected platform as well as a link to a graph showing the history of scores for this test. There is also a link to a treeherder page showing the Talos jobs in a pushlog format.

To learn more about the regressing test(s), please see: https://wiki.mozilla.org/TestEngineering/Performance/Talos

For information on reproducing and debugging the regression, either on try or locally, see: https://wiki.mozilla.org/TestEngineering/Performance/Talos/Running

*** Please let us know your plans within 3 business days, or the offending patch(es) will be backed out! ***

Our wiki page outlines the common responses and expectations: https://wiki.mozilla.org/TestEngineering/Performance/Talos/RegressionBugsHandling

Comment 1

[Paul Adenot (:padenot)]

David, this is probably of importance. We'll have to check what's up when we reduce the runs (bug 1624987).

Comment 2

[(Away)]

efa017cf034f dramatically increased the number of webaudio samples at the top of the profile, which drowns out other functions like displaylist code.

before

Top 4 functions with the largest internal block counts:
  /builds/worker/checkouts/gecko/media/libspeex_resampler/src/resample.c:resampler_basic_direct_single, max count = 438176095
  moz_speex_inner_product_single, max count = 435523349
  moz_speex_have_single_simd, max count = 434780045
  moz_speex_resampler_process_float, max count = 413381638

after

Top 4 functions with the largest internal block counts:
  /builds/worker/checkouts/gecko/media/libspeex_resampler/src/resample.c:resampler_basic_direct_single, max count = 1322775636
  moz_speex_inner_product_single, max count = 1313265531
  moz_speex_have_single_simd, max count = 1312522227
  moz_speex_resampler_process_float, max count = 1230258604

We should monitor how this changes by the landing of bug 1625912 and then further keep an eye on the numbers as we reduce the duration.

Comment 3

[(Away)]

speex functions have dropped out of the top 10 after e70a66b181f8 of bug 1625912. If the perf numbers come in favorably, we might not even need to make any further changes to duration.

Comment 4

[(Away)]

It looks good: https://treeherder.mozilla.org/perf.html#/graphs?highlightAlerts=1&series=autoland,1925793,1,1&series=autoland,1925248,1,1&timerange=172800&zoom=1585763503348,1585764130118,1298.6207689616524,1447.006052498722

I'll let the sheriffs resolve this once we have the full set of alerts in.

Comment 5

[Ethan Tseng [:ethan]]

Hi Eric, while Nathan is away, could you find someone to help triage this bug?

Comment 6

[Julien Cristau [:jcristau]]

Alexandru where is this at now?

Comment 7

[Alexandru Ionescu (needinfo me) [:alexandrui]]

Julien, looks like Bug 1625912 laded a fix for this. Pushlog | Graph

Comment 8

[Ryan VanderMeulen [:RyanVM]]

Is there something we should be uplifting to Beta to resolve this for 76 still?

Comment 9

[Ryan VanderMeulen [:RyanVM]]

The graph shows a clear win on Beta as well after uplifting bug 1625912 there.

Comment 10

[Paul Adenot (:padenot)]

David, how do we fare on 76, do we need to fix something still?

Comment 11

[(Away)]

Unfortunately, I just noticed a problem, bug 1630661.