Closed Bug 1701294 Opened 3 years ago Closed 3 years ago

Negative heap-unclassified on Windows on AWSY pageset

Categories

(Core :: Graphics: WebRender, defect, P3)

Product:
Core
Component:
Graphics: WebRender
Type:
defect

Tracking

()

Status:
RESOLVED WORKSFORME

People

(Reporter: mccr8, Unassigned, NeedInfo)

References

(Blocks 1 open bug)

Details

Dave Hunt is trying to update the page set used by AWSY in Bug 1700955, and on Windows with WebRender enabled he's hitting negative heap-unclassified. My guess would be that it is font-related, because there is quite a lot of font data, and I think we've had some memory reporting issues with fonts in the past.

The best route to investigate this would be to reproduce the issue, narrow down which of the 3 pages is causing the negative heap-unclassified, then get a DMD report, which might give you some idea of what is being overreported.

(In reply to Andrew McCreight [:mccr8] from comment #0)

The best route to investigate this would be to reproduce the issue, narrow down which of the 3 pages is causing the negative heap-unclassified, then get a DMD report, which might give you some idea of what is being overreported.

The three pages which were involved in Bug 1700955 are:

  1. https://docs.google.com/spreadsheets/d/1jT9qfZFAeqNoOK97gruc34Zb7y_Q-O_drZ8kSXT-4D4/edit#gid=1115838130
  2. https://www.apple.com/macbook-pro/
  3. https://www.paypal.com/myaccount/summary/

Moving to gfx-triage to find an owner.

Blocks: gfx-triage
Severity: -- → S3
Priority: -- → P3

Can we get the DMD report for this?

Flags: needinfo?(dave.hunt)

You can find a DMD report by going into this try push that was linked in Phabricator for the patch in bug 1700955:
https://treeherder.mozilla.org/jobs?repo=try&revision=8119709443972d12167d75d12df1273e7da58c64&searchStr=sy-tp6

Then select one of the "Windows 10 x64 WebRender Shippable" builds, then click on the "Artifacts" tab. When I checked, I found negative heap-unclassified in memory-report-TabsOpen-0.json.gz

Flags: needinfo?(dave.hunt)

Wait, sorry, I misread and didn't notice that you were looking for a DMD report...

Flags: needinfo?(dave.hunt)

What's a DMD report, and how can I generate one?

Flags: needinfo?(dave.hunt) → needinfo?(jmathies)

It looks like that we do not have AWSY tp6 jobs that produce DMD artifacts. It's only available for tp5(?) as sy-dmd. Would those jobs also work? Otherwise it might make sense to also add DMD jobs for tp6?

Flags: needinfo?(jmathies) → needinfo?(dave.hunt)

I doubt anything related to tp5 will help here as that page set hasn't been updated in many years. I'm unable to provide much help here as I'm unfamiliar with DMD reports and how to generate them.

Flags: needinfo?(dave.hunt) → needinfo?(jmathies)

I'm not saying that it needs an update of the tp5 pageset, but the jobs we actually run for awsy in CI that have DMD enabled. See:
https://searchfox.org/mozilla-central/source/taskcluster/ci/test/awsy.yml#75-81

But after checking bug 1700955 comment 2 it's not clear to me where Andrew got the negative heap unclassified memory from. So maybe he run the attached patch locally? mach awsy-test actually supports a --dmd argument that will cause the tool to create all the necessary DMD reports. Maybe that's already enough here?

I didn't run anything locally. If you look at the AWSY job under artifacts on the try job, you can download the memory reports.

I'm not really sure how to enable DMD on try. Maybe you could set some environment variables and get DMD to save its files to the upload dir thing?

But those don't contain the DMD reports. So to get DMD reports on try you could simply add --dmd as extra argument for mozharness here:
https://searchfox.org/mozilla-central/rev/62135a96327f42fd1ccf8a04feb62be04b102195/taskcluster/ci/test/awsy.yml#73

Flags: needinfo?(jmathies)

dmd.py failed to write the dumps due to:

[task 2021-04-12T17:33:07.487Z] 17:33:07     INFO - TEST-UNEXPECTED-ERROR | awsy/test_memory_usage.py TestMemoryUsage.test_open_tabs | OSError: [Errno 9] write() on read-only GzipFile object
[task 2021-04-12T17:33:07.487Z] 17:33:07     INFO - Traceback (most recent call last):
[task 2021-04-12T17:33:07.487Z] 17:33:07     INFO -   File "/opt/worker/tasks/task_161824651024069/build/venv/lib/python3.7/site-packages/marionette_harness/marionette_test/testcases.py", line 247, in run
[task 2021-04-12T17:33:07.487Z] 17:33:07     INFO -     self.tearDown()
[task 2021-04-12T17:33:07.487Z] 17:33:07     INFO -   File "/opt/worker/tasks/task_161824651024069/build/tests/awsy/awsy/test_memory_usage.py", line 161, in tearDown
[task 2021-04-12T17:33:07.487Z] 17:33:07     INFO -     AwsyTestCase.tearDown(self)
[task 2021-04-12T17:33:07.487Z] 17:33:07     INFO -   File "/opt/worker/tasks/task_161824651024069/build/venv/lib/python3.7/site-packages/awsy/awsy_test_case.py", line 125, in tearDown
[task 2021-04-12T17:33:07.487Z] 17:33:07     INFO -     self.cleanup_dmd()
[task 2021-04-12T17:33:07.488Z] 17:33:07     INFO -   File "/opt/worker/tasks/task_161824651024069/build/venv/lib/python3.7/site-packages/awsy/awsy_test_case.py", line 150, in cleanup_dmd
[task 2021-04-12T17:33:07.488Z] 17:33:07     INFO -     fixStackTraces(f, isZipped, gzip.open)
[task 2021-04-12T17:33:07.488Z] 17:33:07     INFO -   File "/opt/worker/tasks/task_161824651024069/build/application/Firefox Nightly.app/Contents/Resources/dmd.py", line 287, in fixStackTraces
[task 2021-04-12T17:33:07.488Z] 17:33:07     INFO -     tmpFile.write(fix(line))
Blocks: sw-wr-testing
No longer blocks: gfx-triage

This happens on the awsy tests which provides a reliable STR.

Depends on: 1690956

(In reply to Andrew Osmond [:aosmond] from comment #14)

This happens the awsy tests which provides a reliable STR.

Sorry :aosmond, I don't follow this comment. Could you perhaps rephrase or elaborate?

Flags: needinfo?(aosmond)

We've been successful in landing bug 1700955 with an updated page set and the dependency bug 1690956 is resolved, so I'm going to close this one.

Status: NEW → RESOLVED
Closed: 3 years ago
Resolution: --- → WORKSFORME
Flags: needinfo?(jmathies)
You need to log in before you can comment on or make changes to this bug.