1602765 - 2.14 - 2.51% JS (windows7-32, windows7-32-shippable) regression on push 3fcb7b24eef862475ebea731a8930692589b9d6d (Sat November 30 2019)

Status: RESOLVED WONTFIX

(Firefox :: Security, defect, P2)

Description

[Alexandru Ionescu (needinfo me) [:alexandrui]]

We have detected an awsy regression from push:

https://hg.mozilla.org/integration/autoland/pushloghtml?changeset=3fcb7b24eef862475ebea731a8930692589b9d6d

As author of one of the patches included in that push, we need your help to address this regression.

Regressions:

3% JS windows7-32 opt 81,905,971.47 -> 83,958,869.67
2% JS windows7-32-shippable opt 82,029,362.41 -> 83,989,287.06
2% JS windows7-32-shippable opt 82,146,900.42 -> 83,904,969.79

You can find links to graphs and comparison views for each of the above tests at: https://treeherder.mozilla.org/perf.html#/alerts?id=24386

On the page above you can see an alert for each affected platform as well as a link to a graph showing the history of scores for this test. There is also a link to a treeherder page showing the jobs in a pushlog format.

To learn more about the regressing test(s), please see: https://wiki.mozilla.org/AWSY/Tests

Comment 1

[Julien Cristau [:jcristau]]

A couple of questions..

1. This landed in nightly 72, why is 72 marked as unaffected?
2. Why did it take 2 weeks from the push to this report?

Comment 2

[Alexandru Ionescu (needinfo me) [:alexandrui]]

(In reply to Julien Cristau [:jcristau] from comment #1)

A couple of questions..

1. This landed in nightly 72, why is 72 marked as unaffected?
2. Why did it take 2 weeks from the push to this report?

1. Probably Firefox 73 metabug was already open at that time and this regression was linjked to it.
2. Because of the superseded jobs

Comment 3

[Ryan VanderMeulen [:RyanVM]]

I'm assuming this is overhead from loading another extension on startup. Not sure there's much we can do about this in practice?

Comment 4

[(inactive) Nihanth Subramanya [:nhnt11]]

Ugh, I was going to respond here but it slipped into my backlog.

This is a memory issue and is likely due to the way we're loading modules in the extension APIs in this add-on.

rhelmer suggested that we can disable the add-on and only enable it if the pref is turned on, but I want to avoid this because it will introduce more scope for errors when remotely flipping this on from Normandy and we don't have a lot of QA resources to keep it well tested.

I think we should try converting some of the module imports to lazy getters and see if that helps. If not, this will be converted to a JSM in the next quarter anyway, so I'd suggest keeping this open and verifying and closing it once that's done.

Comment 5

[:Gijs (out for now; he/him)]

If you look at the detailed (per-test) views, you can see at what point (ie which subtest/measuring-point) exactly memory use regressed, and you can download dumps for the before/after state from the files list on treeherder for the performance jobs that flagged this. You can then feed 2 of those files into about:memory's diff tool to see exactly what has changed in terms of memory usage.

If this was really about the extension, I'd have expected the regression to be in extension processes, not the main process. I'm also surprised it's 2mb - that seems like a lot for "just" some JSMs.

Comment 6

[Liz Henry (:lizzard) (relman/hg->git project)]

Nihanth is this something you intend to work on for 74 ? And can you set a priority ? Thanks!

Comment 7

[Ethan Tseng [:ethan]]

Needinfo myself to follow up this bug.

Comment 8

[Ethan Tseng [:ethan]]

Discussed with Nihanth. We are targeting Firefox 76 to fix this issue.

Comment 9

[(inactive) Nihanth Subramanya [:nhnt11]]

Discussed with Nihanth. We are targeting Firefox 76 to fix this issue.

This regression is unfortunate and significant, but with the resourcing we have, I don't think that spending time on this directly is the right strategy. I already started working on rewriting the heuristics add-on as a JSM in bug 1603779, which I am pretty sure will solve this regression, and if not, we can address it then.

Comment 10

[Eric Rahm [:erahm]]

(In reply to Nihanth Subramanya [:nhnt11] from comment #9)

Discussed with Nihanth. We are targeting Firefox 76 to fix this issue.

This regression is unfortunate and significant, but with the resourcing we have, I don't think that spending time on this directly is the right strategy. I already started working on rewriting the heuristics add-on as a JSM in bug 1603779, which I am pretty sure will solve this regression, and if not, we can address it then.

That sounds reasonable. Please let us know if you need help verifying memory improvements, we have some docs on the fission memory wiki page that give example commands to run the tests locally and in automation.

Comment 11

[(inactive) Nihanth Subramanya [:nhnt11]]

Eric, the rewriting work I mentioned in comment 9 is now complete. Could you help me figure out how to check whether there has been any impact on this regression?

Comment 12

[Eric Rahm [:erahm]]

Looking at the awsy win7 js graph It kind of looks like we've improved. I'd expect an alert on bug 1603779 in the next week or so to confirm things. If you want to verify on try you can use something like:

hg up base_revision
./mach try fuzzy  -q "awsy win7 'opt-awsy-e10" --rebuild 5
hg up new_revision
./mach try fuzzy  -q "awsy win7 'opt-awsy-e10" --rebuild 5

And then via treeherder you can select the sy task and click the "Compare against another revision" link at the bottom of the performance panel

Comment 13

[Gregory Pappas [:gregp]]

Closing old win7 regressions