Open Bug 938691 (MochiMem) Opened 6 years ago Updated 2 years ago

Investigate Mochitest memory usage

Categories

(Core :: General, defect)

defect
Not set

Tracking

()

People

(Reporter: mccr8, Unassigned)

References

(Depends on 9 open bugs, Blocks 1 open bug)

Details

(Whiteboard: [MemShrink:meta])

Recent tree closures (bug 932781 and bug 937997) have indicated we don't really know what is going on with memory usage in Mochitests.  There are a variety of problems, from absurdly memory intensive tests (bug 933072 and some other one in bc) to actual leaks either in the test harness or in the actual code (such as bug 932867 and bug 937997).  None of these things have been particularly difficult to fix, we just haven't measured them.

In bug 937997, shu and froydnj did some cool stuff to check mochitest-bc memory usage at various points, via about:memory and DMD.  I think we should expand that across all mochitests.  I bet we could find more leakiness, places more memory reporters would be useful, more tests that use immense amounts of memory, generally improving our stability in both the browser and TBPL.

I'd like to land code so somebody can easily collect about:memory or DMD reports at many checkpoints, though it would be off by default.  We also need some way to visualize or analyze a set of measurements taken over time.  If it is useful enough, we could have some kind of arewemochitestslimyet.com-ish thing to track regressions.  areweslimyet is great for the stuff it tests, but Mochitests run a much larger percentage of our code.
Depends on: 938289
Depends on: 938786
An about:memory dump a third of the way through mochitest-bc contains this:

4,009 (100.0%) -- observer-service
└──4,009 (100.0%) -- referent
   ├──2,428 (60.56%) -- weak
   │  ├──1,763 (43.98%) ── dead
   │  └────665 (16.59%) ── alive
   └──1,581 (39.44%) ── strong

1,839 (100.0%) -- observer-service-suspect
├────799 (43.45%) ── referent(topic=formsubmit)
├────486 (26.43%) ── referent(topic=xpcom-shutdown)
├────289 (15.72%) ── referent(topic=dom-window-destroyed)
└────265 (14.41%) ── referent(topic=memory-pressure)

A similar dump from ~150 tests earlier has:

1,888 (100.0%) -- observer-service
└──1,888 (100.0%) -- referent
   ├────953 (50.48%) ── strong
   └────935 (49.52%) -- weak
        ├──497 (26.32%) ── dead
        └──438 (23.20%) ── alive

1,094 (100.0%) -- observer-service-suspect
├────363 (33.18%) ── referent(topic=xpcom-shutdown)
├────337 (30.80%) ── referent(topic=formsubmit)
├────225 (20.57%) ── referent(topic=memory-pressure)
└────169 (15.45%) ── referent(topic=dom-window-destroyed)

Pretty sure those numbers shouldn't be changing that much over the course of the test.  Working on hunting those down.
Depends on: 938989
Depends on: 939369
Depends on: 939036
Depends on: 939606
Depends on: 938945
Depends on: 939414
Depends on: 938411
Depends on: 819839
Alias: MochiMem
Depends on: 939914
Depends on: 940426
Depends on: 941837
Depends on: 949700
Depends on: 961106
Depends on: 961793
Depends on: 962662
Depends on: 963259
Depends on: 964448
Nathan - big thanks for your efforts digging into this - especially since it seems like at least one of the issues identified has resulted in fixes to real-world use cases \o/ (Bug 961793 fixed bug 964386)
Depends on: 937181
Depends on: 977921
Depends on: 984843
Depends on: 984930
Depends on: 983948
Depends on: 991311
Depends on: 435915
Depends on: LifetimeTesting
Blocks: 996504
Depends on: 949607
In bug 990355, we're OOMing during worker chrome tests because test_fileSubWorker.xul spins up 8 workers at the same time, and thanks to GGC, each worker consumes 25MB of contiguous space for a nursery.
Depends on: 990355
Excuse me, 16MB per worker.
Assignee: continuation → nobody
You need to log in before you can comment on or make changes to this bug.