1644834 - about:memory tests fail on my machines (macOS Catalina, Ubuntu 18.04)

Status: RESOLVED FIXED

(Toolkit :: about:memory, defect)

Description

[David Teller [:Yoric] - still alive but not very active]

The tests of toolkit/components/aboutmemory pass on try. However, attempting to launch the tests on my machine, I get:

toolkit/components/aboutmemory/tests/test_aboutmemory5.xhtml
  FAIL three child processes present in loaded data
SimpleTest.ok@chrome://mochikit/content/tests/SimpleTest/SimpleTest.js:299:16
copyPasteAndCheck@chrome://mochitests/content/chrome/toolkit/components/aboutmemory/tests/test_aboutmemory5.xhtml:131:13
setTimeout handler*SimpleTest_setTimeoutShim@chrome://mochikit/content/tests/SimpleTest/SimpleTest.js:788:41
copyPasteAndCheck@chrome://mochitests/content/chrome/toolkit/components/aboutmemory/tests/test_aboutmemory5.xhtml:153:23
loadAndCheck@chrome://mochitests/content/chrome/toolkit/components/aboutmemory/tests/test_aboutmemory5.xhtml:100:7
toolkit/components/aboutmemory/tests/test_memoryReporters.xhtml
  FAIL heap-allocated has 2 report
SimpleTest.ok@chrome://mochikit/content/tests/SimpleTest/SimpleTest.js:299:16
checkSpecialReport@chrome://mochitests/content/chrome/toolkit/components/aboutmemory/tests/test_memoryReporters.xhtml:228:7
checkResults@chrome://mochitests/content/chrome/toolkit/components/aboutmemory/tests/test_memoryReporters.xhtml:242:25
setTimeout handler*SimpleTest_setTimeoutShim@chrome://mochikit/content/tests/SimpleTest/SimpleTest.js:788:41
runNext@chrome://mochitests/content/chrome/toolkit/components/aboutmemory/tests/test_memoryReporters.xhtml:202:15
  FAIL vsize has 2 report
SimpleTest.ok@chrome://mochikit/content/tests/SimpleTest/SimpleTest.js:299:16
checkSpecialReport@chrome://mochitests/content/chrome/toolkit/components/aboutmemory/tests/test_memoryReporters.xhtml:228:7
checkResults@chrome://mochitests/content/chrome/toolkit/components/aboutmemory/tests/test_memoryReporters.xhtml:247:23
setTimeout handler*SimpleTest_setTimeoutShim@chrome://mochikit/content/tests/SimpleTest/SimpleTest.js:788:41
runNext@chrome://mochitests/content/chrome/toolkit/components/aboutmemory/tests/test_memoryReporters.xhtml:202:15
  FAIL resident has 2 report
SimpleTest.ok@chrome://mochikit/content/tests/SimpleTest/SimpleTest.js:299:16
checkSpecialReport@chrome://mochitests/content/chrome/toolkit/components/aboutmemory/tests/test_memoryReporters.xhtml:228:7
checkResults@chrome://mochitests/content/chrome/toolkit/components/aboutmemory/tests/test_memoryReporters.xhtml:248:23
setTimeout handler*SimpleTest_setTimeoutShim@chrome://mochikit/content/tests/SimpleTest/SimpleTest.js:788:41
runNext@chrome://mochitests/content/chrome/toolkit/components/aboutmemory/tests/test_memoryReporters.xhtml:202:15
toolkit/components/aboutmemory/tests/test_memoryReporters2.xhtml
  FAIL correct resident count
SimpleTest.ok@chrome://mochikit/content/tests/SimpleTest/SimpleTest.js:299:16
processReports@chrome://mochitests/content/chrome/toolkit/components/aboutmemory/tests/test_memoryReporters2.xhtml:79:9
  FAIL correct non-empty process name prefix: Socket (pid 9177)
SimpleTest.ok@chrome://mochikit/content/tests/SimpleTest/SimpleTest.js:299:16
processReports@chrome://mochitests/content/chrome/toolkit/components/aboutmemory/tests/test_memoryReporters2.xhtml:86:13
  FAIL correct non-empty process name count
SimpleTest.ok@chrome://mochikit/content/tests/SimpleTest/SimpleTest.js:299:16
processReports@chrome://mochitests/content/chrome/toolkit/components/aboutmemory/tests/test_memoryReporters2.xhtml:92:9

I'm running macOS 10.15.5 (19F101) (Catalina).

Comment 1

[David Teller [:Yoric] - still alive but not very active]

Similar problem on Ubuntu 18.04: test_memoryReporters.xhtml fails

njn, this is kinda blocking bug 1597562. Do you have an idea what could be going on?

Comment 2

[David Teller [:Yoric] - still alive but not very active]

Checked with other people, the tests fail on their machines, too (Linux and macOS).

It's both surprising and worrying.

Comment 3

[Andrew McCreight [:mccr8]]

I'm unfamiliar with these tests. I looked at test_aboutmemory5.xhtml, which is also failing for me locally. It looks like this is checking to make sure that there are only 4 processes, of which one is going to be the parent and 3 are child processes, but there are actually 5. The fifth process is the socket process. I don't know why we end up with a socket process when running locally, but not in automation. You could modify the tests to allow for the possibility of a socket process in the count.

Comment 4

[Nicholas Nethercote [inactive]]

On my Ubuntu 19.10 box I get one failure in test_memoryReporters.xhtml.

On my MacBook Pro I get one failure in test_aboutmemory5.xhtml, three failures in test_memoryReporters.xhtml, and three failures in test_memoryReporters2.xhtml.

Prior to bug 1634987 I was seeing failures in all three of the abovementioned tests on my Ubuntu box. After that bug landed I was seeing zero failures, but now I'm seeing one. That bug reverted changes made in bug 1515390 to account for the socket process.

As Andrew says, there are inconsistencies between CI and automation around the presence of the socket process, and based on my results above, there may also be inconsistencies across platforms.

Valentin, do you know about the socket process, and why it might be showing up in some places and not others?

If it is expected that the socket process is absent in some cases and present in others, then I guess we'll have to adjust the tests to be more forgiving in what they accept. Though if there are significant differences like this between CI and local runs, keeping the local tests passing will always be something of a losing battle :(

Comment 5

[Valentin Gosu [:valentin] (he/him)]

I suspect there might be differences in how fast the socket process gets up and running?

Comment 6

[Kershaw Chang [:kershaw]]

The reason why socket process is absent on try is because network.process.enabled is disabled here. We only enable socket process on try for some media tests and xpcshell tests for now.

I think we should adjust the tests as njn said in comment #4.

Comment 7

[Kershaw Chang [:kershaw]]

Attachment: Bug 1644834 - Fix about:memory tests for socket process, r=njn

Comment 8

[Kershaw Chang [:kershaw]]

https://treeherder.mozilla.org/#/jobs?repo=try&revision=1f340aba6bfa1ae823ed9edeff7656eaa9849fb0

Comment 9

[Nicholas Nethercote [inactive]]

Kershaw: the patch makes the tests pass on my Mac but not on my Linux box. The problem is that the Socket process does not have a vsize measurement on Linux. For example, here is the "Other measurements" section when I open about:memory in Nightly on Linux:

Other Measurements

1.87 MB (100.0%) -- heap-committed
├──1.04 MB (55.77%) ── allocated
└──0.83 MB (44.23%) ── overhead

12 (100.0%) -- ipc-channels
├──10 (83.33%) ── PSocketProcessBridgeParent
├───1 (08.33%) ── PProfilerChild
└───1 (08.33%) ── PSocketProcessChild

12 (100.0%) -- ipc-channels-peak
├──10 (83.33%) ── PSocketProcessBridgeParent
├───1 (08.33%) ── PProfilerChild
└───1 (08.33%) ── PSocketProcessChild

41 (100.0%) -- observer-service
└──41 (100.0%) -- referent
   ├──27 (65.85%) -- weak
   │  ├──27 (65.85%) ── alive
   │  └───0 (00.00%) ── dead
   └──14 (34.15%) ── strong

1 (100.0%) -- preference-service
└──1 (100.0%) -- referent
   ├──1 (100.0%) ── strong
   └──0 (00.00%) ++ weak

 1.04 MB ── heap-allocated
 1.00 MB ── heap-chunksize
 3.50 MB ── heap-mapped
       0 ── page-faults-hard
   2,674 ── page-faults-soft
52.53 MB ── resident-peak
 0.00 MB ── system-heap-allocated
       0 ── unresolved-ipc-responses

This is really strange, I've never seen a missing vsize before. On Linux, vsize is read from the first field in /proc/<pid>/statm, and that file looks perfectly normal on my Linux box for the Socket process, so I'm not sure what's going on.

resident is also missing, and it's also read from /proc/<pid>/statm. vsize and resident are also missing from the RDD process's measurements. Other processes have them, though, including the GPU process.

Comment 10

[Nicholas Nethercote [inactive]]

Aha! The read of /proc/self/statm is failing on the fopen call, and glandium suggested that the sandbox might be rejecting it. But the fopen call works in content processes and GPU processes. So maybe the RDD and Socket processes have slightly different sandbox settings.

jld, does that sound right? Could this call be made to work on the RDD and Socket processes?

Comment 11

[Nicholas Nethercote [inactive]]

I filed bug 1647957 for the Linux sandboxing issue.

Comment 12

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/dce1d6a72699
Fix about:memory tests for socket process, r=njn

Comment 13

[Cristian Brindusan [:cbrindusan]]

https://hg.mozilla.org/mozilla-central/rev/dce1d6a72699

Comment 14

[David Teller [:Yoric] - still alive but not very active]

Unfortunately, this isn't fixed for me: test_memoryReporters.xhtml, test_memoryReporters2.xhtml and test_aboutmemory5.xhtml still fail.

Running Ubuntu 19.10.

Comment 15

[Kershaw Chang [:kershaw]]

(In reply to David Teller [:Yoric] (please use "needinfo") from comment #14)

Unfortunately, this isn't fixed for me: test_memoryReporters.xhtml, test_memoryReporters2.xhtml and test_aboutmemory5.xhtml still fail.

Running Ubuntu 19.10.

This is tracked in bug 1647957.

Comment 16

[David Teller [:Yoric] - still alive but not very active]

Thanks.