1309309 - Report memory used by ANGLE in about:memory as its own block?

Status: NEW

(Core :: Graphics: CanvasWebGL, enhancement, P4)

Description

[Jukka Jylänki]

As part of our attempts to slim down asm.js/wasm application memory usage to be closer to par with native, we are running traces of applications to find where we are using memory, and trying to identify mechanisms that we could give to games developers to do the same kind of identification themselves.

Current asm.js pages tend to show quite a bit of their memory used in the heap-unclassified in about:memory (varying in the 5%-30% of total memory consumption of the whole page)

Looking through these heap-unclassified allocations in DMD build of Firefox (https://developer.mozilla.org/en-US/docs/Mozilla/Performance/DMD), a considerable number of them come from ANGLE. Some of these entries have been reported before as

https://bugzilla.mozilla.org/show_bug.cgi?id=1241105
https://bugzilla.mozilla.org/show_bug.cgi?id=1241102
https://bugzilla.mozilla.org/show_bug.cgi?id=1241095

Looking at the callstacks of those, it seems that all ANGLE allocations already do get hooked into by Mozilla build system, so that ANGLE allocates memory via calling the moz_xmalloc() function here: https://dxr.mozilla.org/mozilla-central/source/memory/mozalloc/mozalloc.cpp?q=moz_xmalloc&redirect_type=direct#81

Presumably the moz_xmalloc function pointer is explicitly passed to ANGLE somewhere so that it knows how to call that function specifically for its memory allocations?

If so, would it be possible to create a specialization in the form of moz_angle_xmalloc() and moz_angle_xfree(), which would keep a special tab of the amount of memory that that is allocated via them, and this would then be shown in about:memory in its own tree item "allocated by ANGLE", that could live under the webgl node?

The rationale is that this would allow estimating how much ANGLE actually gulps, and how much of the heap-unclassified remaining after that would be due to something else, and if there's something we're not seeing because of the large number of unclassified allocations coming from ANGLE. Also we've done a lot of developer education to tell games developers to use about:memory to get a first snapshot. Letting them see the ANGLE-related allocations might give them at least some initial clues on how different things they do in their code will affect the amount that ANGLE ends taking up (not to mention that in bug 1241095 we are pondering if ANGLE might have a memory leak, so this might help triage any such leaks).

Comment 1

[Jukka Jylänki]

For example, profiling a code sample coming from a partner, it shows

1,074.35 MB (100.0%) -- explicit
├────663.55 MB (61.76%) -- window-objects/top(http://localhost:8000/, id=2147483649)
│    ├──660.49 MB (61.48%) -- active
│    │  ├──658.44 MB (61.29%) -- window(http://localhost:8000/)
│    │  │  ├──657.88 MB (61.24%) -- js-compartment(http://localhost:8000/)
│    │  │  │  ├──656.74 MB (61.13%) -- classes
│    │  │  │  │  ├──589.70 MB (54.89%) -- class(ArrayBuffer)/objects
│    │  │  │  │  │  ├──432.00 MB (40.21%) ── non-heap/elements/wasm
│    │  │  │  │  │  ├──157.70 MB (14.68%) ── malloc-heap/elements/normal
│    │  │  │  │  │  └────0.00 MB (00.00%) ── gc-heap
│    │  │  │  │  ├───65.90 MB (06.13%) -- class(WebAssembly.Instance)/objects
│    │  │  │  │  │   ├──46.88 MB (04.36%) ── non-heap/code/wasm
│    │  │  │  │  │   ├──19.02 MB (01.77%) -- malloc-heap
│    │  │  │  │  │   │  ├──19.02 MB (01.77%) ── misc
│    │  │  │  │  │   │  └───0.00 MB (00.00%) ── slots
│    │  │  │  │  │   └───0.00 MB (00.00%) ── gc-heap
│    │  │  │  │  └────1.14 MB (00.11%) ++ (8 tiny)
│    │  │  │  └────1.14 MB (00.11%) ++ (9 tiny)
│    │  │  └────0.55 MB (00.05%) ++ (4 tiny)
│    │  └────2.05 MB (00.19%) ++ (4 tiny)
│    └────3.06 MB (00.28%) ++ js-zone(0x7f421147b000)
├────169.82 MB (15.81%) ── heap-unclassified
├────125.51 MB (11.68%) -- webaudio
│    ├──125.50 MB (11.68%) ── audiobuffer
│    └────0.01 MB (00.00%) ++ (2 tiny)
├─────43.64 MB (04.06%) -- dom
│     ├──43.61 MB (04.06%) -- memory-file-data/large
│     │  ├──43.59 MB (04.06%) ── file(length=45704402, sha1=be6dc186ab93a2faf6dc452e8f86b35c9f8c1ae8)
│     │  └───0.02 MB (00.00%) ── file(length=18201, sha1=eb7025626c1a00f82b70317492fb0a60e3008481)
│     └───0.03 MB (00.00%) ++ (2 tiny)
├─────31.36 MB (02.92%) -- js-non-window
│     ├──21.06 MB (01.96%) -- runtime
│     │  ├──15.94 MB (01.48%) ── shared-immutable-strings-cache
│     │  └───5.12 MB (00.48%) ++ (11 tiny)
│     └──10.30 MB (00.96%) ++ (2 tiny)
├─────27.84 MB (02.59%) -- dmd
│     ├──24.00 MB (02.23%) ── live-block-table
│     └───3.84 MB (00.36%) ++ (2 tiny)
└─────12.63 MB (01.18%) ++ (17 tiny)

and in here, estimating that 100MB+ of heap-unclassified is ANGLE, so quite a considerable portion.

Comment 2

[Jukka Jylänki]

Running a new profile today of DMD on the UE4 Zen Garden demo, with STR

1. Do a DMD build of Firefox from https://developer.mozilla.org/en-US/docs/Mozilla/Performance/DMD
2. Visit http://mzl.la/webassemblydemo
3. Run a snapshot and analyze the content process

there is

├──160.25 MB (16.24%) ── heap-unclassified

of memory that is untracked, which is for the most part coming from ANGLE in the following two call sites:

Unreported {
  14,134 blocks in heap block record 1 of 2,715
  115,785,728 bytes (115,785,728 requested / 0 slop)
  Individual block sizes: 8,192 x 14,134
  19.75% of the heap (19.75% cumulative)
  67.33% of unreported (67.33% cumulative)
  Allocated at {
    #01: moz_xmalloc (e:\gecko-dev\memory\mozalloc\mozalloc.cpp:84)
    #02: TPoolAllocator::allocate (e:\gecko-dev\gfx\angle\src\compiler\translator\poolalloc.cpp:305)
  }
}

Unreported {
  146 blocks in heap block record 4 of 2,715
  8,436,544 bytes (8,348,336 requested / 88,208 slop)
  Individual block sizes: 3,203,072; 1,048,576 x 4; 196,608; 102,400; 65,536 x 4; 45,056; 32,768; 24,576 x 2; 20,480 x 2; 16,384 x 2; 12,288; 8,192 x 12; 4,096 x 23; 2,048 x 14; 1,024 x 34; 512 x 2; 480 x 2; 464; 448; 432 x 2; 384 x 2; 320 x 2; 272; 240 x 2; 224; 192 x 7; 176 x 3; 144; 112 x 2; 96; 80 x 2; 64; 48 x 3; 32 x 4; 16 x 3
  1.44% of the heap (26.58% cumulative)
  4.91% of unreported (90.58% cumulative)
  Allocated at {
    #01: rx::MemoryBuffer::resize (e:\gecko-dev\gfx\angle\src\common\memorybuffer.cpp:45)
    #02: rx::Buffer11::SystemMemoryStorage::resize (e:\gecko-dev\gfx\angle\src\libangle\renderer\d3d\d3d11\buffer11.cpp:1522)
  }
}

Given that this is such a high portion of all memory allocations on that page (12.3% of all memory consumption), it would be good to track these explicitly in about:memory.