1516966 - 4.64 - 29.51% Base*, Resident Unique Memory (all platforms) regression on push a791cbab57a3908cf66afbfddf9bbc563353e700 (Thu Dec 27 2018)

Status: RESOLVED WONTFIX

(Testing :: AWSY, defect)

Description

[Joel Maher ( :jmaher ) (UTC -8)]

We have detected an awsy regression from push:

https://hg.mozilla.org/integration/autoland/pushloghtml?changeset=a791cbab57a3908cf66afbfddf9bbc563353e700

As author of one of the patches included in that push, we need your help to address this regression.

Regressions:

 30%  Base Content JS osx-10-10 opt stylo                          4,498,524.00 -> 5,825,829.33
 29%  Base Content JS linux64-qr opt stylo                         4,497,432.00 -> 5,822,063.33
 29%  Base Content JS linux64 opt stylo                            4,497,401.33 -> 5,822,018.00
 26%  Base Content JS windows10-64-qr opt stylo                    4,656,008.67 -> 5,881,337.33
 25%  Base Content Explicit linux64-qr opt stylo                   11,832,320.00 -> 14,735,701.33
 24%  Base Content JS windows10-64 pgo stylo                       4,761,548.00 -> 5,901,636.00
 24%  Base Content JS windows10-64 opt stylo                       4,760,830.67 -> 5,900,270.67
 23%  Base Content Explicit linux64 opt stylo                      11,820,714.67 -> 14,513,152.00
 21%  Base Content Resident Unique Memory linux64-qr opt stylo     17,655,381.33 -> 21,438,464.00
 21%  Base Content Explicit osx-10-10 opt stylo                    10,302,634.67 -> 12,469,248.00
 21%  Base Content Resident Unique Memory linux64 opt stylo        17,634,474.67 -> 21,328,896.00
 21%  Base Content JS windows7-32 pgo stylo                        3,875,762.67 -> 4,683,646.67
 21%  Base Content JS windows7-32 opt stylo                        3,875,760.00 -> 4,683,657.33
 17%  Base Content Heap Unclassified linux64-qr opt stylo          4,634,948.67 -> 5,440,898.00
 17%  Base Content Heap Unclassified linux64 opt stylo             4,636,964.33 -> 5,442,836.67
 15%  Base Content Explicit windows10-64-qr opt stylo              10,336,341.33 -> 11,922,261.33
 12%  Base Content Explicit windows7-32 pgo stylo                  8,100,010.67 -> 9,055,914.67
 10%  Base Content Explicit windows7-32 opt stylo                  8,078,677.33 -> 8,918,357.33
 10%  Base Content Heap Unclassified windows10-64-qr opt stylo     1,460,606.67 -> 1,611,958.00
 10%  Base Content Explicit windows10-64 pgo stylo                 11,101,696.00 -> 12,194,133.33
 10%  Base Content Explicit windows10-64 opt stylo                 11,094,357.33 -> 12,172,288.00
  5%  Base Content Heap Unclassified windows10-64 opt stylo        1,529,958.67 -> 1,606,980.00
  5%  Base Content Heap Unclassified windows10-64 pgo stylo        1,557,501.00 -> 1,629,837.33

Improvements:

  2%  Base Content Resident Unique Memory windows10-64 pgo stylo     15,300,437.33 -> 14,985,216.00


You can find links to graphs and comparison views for each of the above tests at: https://treeherder.mozilla.org/perf.html#/alerts?id=18511

On the page above you can see an alert for each affected platform as well as a link to a graph showing the history of scores for this test. There is also a link to a treeherder page showing the jobs in a pushlog format.

To learn more about the regressing test(s), please see: https://wiki.mozilla.org/AWSY/Tests

Comment 1

[Joel Maher ( :jmaher ) (UTC -8)]

:heycam, I assume this is expected, but can you confirm this looks correct both in scope and across all configs?

Comment 2

[Cameron McCormack (:heycam)]

Hi Joel, yes I expected regressions in all Base Content values across all platforms from the bug 1516493 change.

Although when I look at the graphs for Base Content Resident Unique Memory, all of the Windows platforms have an improvement, while other platforms have a regression.  I don't have a good explanation for why that is.  The regressions make sense to me, since dropping the pre-allocated process (as bug 1516493 did) might mean that we now have pages in the content process that are not shared with any other process.  It's possible these Resident Unique Memory values will change again anyway when bug 1515201 lands and there are multiple content processes running.

For the rest of the numbers I don't have a great sense of how expected the regression magnitude is, but they all seem plausible to me.

Comment 3

[Joel Maher ( :jmaher ) (UTC -8)]

I vote for closing this as wontfix