Track the amount of malloc memory precisely
Categories
(Core :: JavaScript: GC, enhancement, P3)
Tracking
()
People
(Reporter: jonco, Assigned: jonco)
References
(Blocks 1 open bug)
Details
Attachments
(44 files, 5 obsolete files)
|
47 bytes,
text/x-phabricator-request
|
Details | Review | |
|
47 bytes,
text/x-phabricator-request
|
Details | Review | |
|
47 bytes,
text/x-phabricator-request
|
Details | Review | |
|
47 bytes,
text/x-phabricator-request
|
Details | Review | |
|
47 bytes,
text/x-phabricator-request
|
Details | Review | |
|
47 bytes,
text/x-phabricator-request
|
Details | Review | |
|
47 bytes,
text/x-phabricator-request
|
Details | Review | |
|
47 bytes,
text/x-phabricator-request
|
Details | Review | |
|
47 bytes,
text/x-phabricator-request
|
Details | Review | |
|
47 bytes,
text/x-phabricator-request
|
Details | Review | |
|
47 bytes,
text/x-phabricator-request
|
Details | Review | |
|
47 bytes,
text/x-phabricator-request
|
Details | Review | |
|
47 bytes,
text/x-phabricator-request
|
Details | Review | |
|
47 bytes,
text/x-phabricator-request
|
Details | Review | |
|
47 bytes,
text/x-phabricator-request
|
Details | Review | |
|
47 bytes,
text/x-phabricator-request
|
Details | Review | |
|
47 bytes,
text/x-phabricator-request
|
Details | Review | |
|
47 bytes,
text/x-phabricator-request
|
Details | Review | |
|
47 bytes,
text/x-phabricator-request
|
Details | Review | |
|
47 bytes,
text/x-phabricator-request
|
Details | Review | |
|
47 bytes,
text/x-phabricator-request
|
Details | Review | |
|
47 bytes,
text/x-phabricator-request
|
Details | Review | |
|
47 bytes,
text/x-phabricator-request
|
Details | Review | |
|
47 bytes,
text/x-phabricator-request
|
Details | Review | |
|
47 bytes,
text/x-phabricator-request
|
Details | Review | |
|
47 bytes,
text/x-phabricator-request
|
Details | Review | |
|
47 bytes,
text/x-phabricator-request
|
Details | Review | |
|
47 bytes,
text/x-phabricator-request
|
Details | Review | |
|
47 bytes,
text/x-phabricator-request
|
Details | Review | |
|
47 bytes,
text/x-phabricator-request
|
Details | Review | |
|
47 bytes,
text/x-phabricator-request
|
Details | Review | |
|
47 bytes,
text/x-phabricator-request
|
Details | Review | |
|
47 bytes,
text/x-phabricator-request
|
Details | Review | |
|
47 bytes,
text/x-phabricator-request
|
Details | Review | |
|
47 bytes,
text/x-phabricator-request
|
Details | Review | |
|
47 bytes,
text/x-phabricator-request
|
Details | Review | |
|
47 bytes,
text/x-phabricator-request
|
Details | Review | |
|
47 bytes,
text/x-phabricator-request
|
Details | Review | |
|
47 bytes,
text/x-phabricator-request
|
Details | Review | |
|
47 bytes,
text/x-phabricator-request
|
Details | Review | |
|
47 bytes,
text/x-phabricator-request
|
Details | Review | |
|
47 bytes,
text/x-phabricator-request
|
Details | Review | |
|
47 bytes,
text/x-phabricator-request
|
Details | Review | |
|
47 bytes,
text/x-phabricator-request
|
Details | Review |
At the moment we have counters for malloced memory that count down to zero from a set threshold when memory is allocated. When they hit zero we trigger a GC. We have one for the runtime and one per zone. There are several problems with this: - it doesn't reflect freed memory so we may trigger a GC when there is nothing to collect - the threshold doesn't depend on the size of the heap so we may trigger GCs more frequently than necessary for large heaps - we lose information when we merge zones (bug 1341093) - they work differently from the GC memory counters which is confusing We should make these counters track allocated size exactly, the same as we do for GC allocated memory. This is a large project and not all of the details are clear. In particular it's not clear whether we should make all our free methods take a size in bytes or rely on the malloc implementation (e.g. jemalloc) to get this information.
|
||
Updated•7 years ago
|
|
Assignee | |
Updated•5 years ago
|
|
|
Assignee | |
Updated•5 years ago
|
|
|
Assignee | |
Comment 1•5 years ago
•
|
||
I'm going to start posting patches and hope to land them incrementally.
The general strategy is to add APIs to tell the GC about malloc memory associated with a cell:
void AddCellMemory(gc::Cell* cell, size_t nbytes, MemoryUse use);
void RemoveCellMemory(gc::Cell* cell, size_t nbytes, MemoryUse use)
Calls to these must be balanced and this is checked in debug builds.
FreeOp is updated with methods to call RemoveCellMemory and free. The long term aim is to replace all use of the current FreeOp::free_/delete_ with these methods to ensure all memory is tracked.
To start with I'm going to add tracking for memory which can contribute significantly to the total, e.g. is very frequently used or has a size which is under user control.
|
|
Assignee | |
Comment 2•5 years ago
|
||
|
|
Assignee | |
Comment 3•5 years ago
|
||
Depends on D30514
|
|
Assignee | |
Comment 4•5 years ago
|
||
Depends on D30515
|
|
Assignee | |
Comment 5•5 years ago
|
||
This adds memory tracking for string contents while leaving the current scheme in place for the time being.
Depends on D30516
Pushed by jcoppeard@mozilla.com: https://hg.mozilla.org/integration/mozilla-inbound/rev/772b3ec0102d Add APIs to track internal memory assocated with GC things r=sfink https://hg.mozilla.org/integration/mozilla-inbound/rev/95399bf8d949 Add FreeOp methods free memory and update memory accounting r=sfink https://hg.mozilla.org/integration/mozilla-inbound/rev/54227b612212 Track malloc memory associated with array buffers r=sfink https://hg.mozilla.org/integration/mozilla-inbound/rev/01140845cccc Track malloc memory associated with strings r=jandem
|
||
Comment 7•5 years ago
|
||
| bugherder | ||
|
|
Assignee | |
Comment 8•5 years ago
|
||
Since we now have precise memory accounting for externally allocated memory associated with GC things we should be able to remove use of the existing malloc counter here. This should help with cases where we trigger too many GCs because we think there is more memory associated than there really is.
Pushed by jcoppeard@mozilla.com: https://hg.mozilla.org/integration/autoland/rev/441a4c70ac22 Remove existing malloc accounting for externally allocated memory r=sfink?
|
||
Comment 10•5 years ago
|
||
== Change summary for alert #20930 (as of Mon, 13 May 2019 11:46:58 GMT) ==
Improvements:
4% raptor-tp6-twitter-firefox loadtime windows7-32-shippable opt 257.33 -> 246.29
For up to date results, see: https://treeherder.mozilla.org/perf.html#/alerts?id=20930
|
||
Comment 11•5 years ago
|
||
| bugherder | ||
|
|
Assignee | |
Comment 12•5 years ago
|
||
This code is debug-only, but it seems a shame to waste have this key structure take up two words when it will fix into one. This also moves the MemoryUse enum definition to gc/GCEnum.h.
|
|
Assignee | |
Comment 13•5 years ago
|
||
This adds memory tracking for object elements. I had to swap the association in JSObject::swap. I also moved NativeObject::shrinkCapacityToInitializedLength out of line so as not to have to pull Zone.h into NativeObject.h. I don't know how performance sensitive this is - if it is I could look at this again.
Depends on D32170
|
|
Assignee | |
Comment 14•5 years ago
|
||
Depends on D32171
|
||
Comment 15•5 years ago
|
||
Pushed by jcoppeard@mozilla.com: https://hg.mozilla.org/integration/mozilla-inbound/rev/740009023b1b Pack MemoryTracker hashtable keys into a single word on 64bit platforms r=sfink
|
||
Updated•5 years ago
|
|
|
Assignee | |
Comment 16•5 years ago
|
||
Depends on D32172
|
||
Comment 17•5 years ago
|
||
| bugherder | ||
|
|
||
Comment 18•5 years ago
|
||
Pushed by jcoppeard@mozilla.com: https://hg.mozilla.org/integration/mozilla-inbound/rev/94dadeeee608 Track malloc memory associated with JSObject elements r=jandem https://hg.mozilla.org/integration/mozilla-inbound/rev/92fc3159f152 Track malloc memory associated with JSObject slots r=jandem https://hg.mozilla.org/integration/mozilla-inbound/rev/5585ac78378a Track malloc memory associated with JSScripts r=tcampbell
|
||
Comment 19•5 years ago
|
||
| bugherder | ||
|
|
Assignee | |
Comment 20•5 years ago
|
||
This changes ZoneAllocPolicy to use the new precise memory tracking rather than the old malloc counter. This works because the mozilla standard containers (HashMap, Vector, etc) already call the AllocyPolicy free_ method with the correct size. This patch tracks individual instances of ZoneAllocPolicy to check that the numbers balance.
Depends on D33000
|
|
||
Comment 21•5 years ago
|
||
Pushed by jcoppeard@mozilla.com: https://hg.mozilla.org/integration/mozilla-inbound/rev/b37b6c015168 Add an alloc policy that tracks malloc memory associated with GC things r=sfink
|
|
Assignee | |
Comment 22•5 years ago
|
||
This refactors the malloc allocation tracking so that the MemoryTracker object is only present in debug builds and the count of malloc bytes is kept separately in the Zone. This makes things a little clearer I think and removes one level of inlining.
|
|
Assignee | |
Comment 23•5 years ago
|
||
This splits out the allocation functions and allocation tracking state into a new base class, ZoneAllocator, which lives in its own header file. We can include this for the common case of allocating malloc memory for GC things without dragging in the full complexity of Zone.h.
Depends on D33179
|
|
||
Comment 24•5 years ago
|
||
Pushed by jcoppeard@mozilla.com: https://hg.mozilla.org/integration/mozilla-inbound/rev/50769742e295 Refactor malloc allocation tracking r=sfink https://hg.mozilla.org/integration/mozilla-inbound/rev/a4daa44cdb9c Split out zone memory allocation framework into separate base class r=sfink
|
|
Assignee | |
Comment 25•5 years ago
|
||
Depends on D33313
|
||
Comment 26•5 years ago
|
||
| bugherder | ||
|
|
||
Comment 27•5 years ago
|
||
Pushed by jcoppeard@mozilla.com: https://hg.mozilla.org/integration/autoland/rev/351d19c8bdd3 Add APIs to handle memory accounting while initializing object pointers to malloc memory r=sfink?
|
||
Comment 28•5 years ago
|
||
Backed out 3 changesets (bug 1395509, bug 1555935) for causing build busateges CLOSED TREE
Backout revision https://hg.mozilla.org/integration/autoland/rev/176ac3f06690a9e5071028ec601a2e743545d045
Log failure https://treeherder.mozilla.org/logviewer.html#?job_id=249693971&repo=autoland
Jon can you please take a look?
|
|
||
Comment 29•5 years ago
|
||
Pushed by jcoppeard@mozilla.com: https://hg.mozilla.org/integration/autoland/rev/ab12b34ff823 Add APIs to handle memory accounting while initializing object pointers to malloc memory r=sfink?
|
||
Comment 30•5 years ago
|
||
| bugherder | ||
|
|
Assignee | |
Comment 31•5 years ago
|
||
The change to split out ZoneAllocator messed up the MemoryTracker code that prints out what failed to be removed by making it run after the Zone destructor which will already assert in the case (but without printing useful information first).
|
|
Assignee | |
Comment 32•5 years ago
|
||
Depends on D33974
|
|
Assignee | |
Comment 33•5 years ago
|
||
Previously I rolled the malloc byte count into a total byte count for each zone but this may adversely affect GC scheduling (e.g. by triggering more non-incremental GCs because allocation volumes appear higher with this change). So that we can land this machinery without disturbing benchmarks too much, this patch splits out the new malloc memory accounting into a separate counter and uses the maxMallocBytes setting as the threshold (default value is 128MB vs 30MB for the GC heap threshold). This should make the behaviour closer to the original behaviour for now. We can go back and adjust the parameters later to obtain the desired behaviour.
Depends on D34180
|
|
Assignee | |
Updated•5 years ago
|
|
|
Assignee | |
Comment 34•5 years ago
|
||
Use memory tracking APIs to track malloc memory associated with Map and Set objects.
Depends on D34181
|
|
Assignee | |
Comment 35•5 years ago
|
||
Use memory tracking APIs to track malloc memory associated with BigInts.
Depends on D34371
|
|
Assignee | |
Comment 36•5 years ago
|
||
Use memory tracking APIs to track malloc memory associated with Scopes.
Depends on D34373
|
|
Assignee | |
Comment 37•5 years ago
|
||
Use memory tracking APIs to track malloc memory associated with the different ctypes objects.
I ended up creating new public APIs because ctypes currently mostly uses our public APIs, but I actaully don't know why. I don't think it can be built standalone. Maybe this should use the internal APIs instead.
Depends on D34374
|
|
Assignee | |
Comment 38•5 years ago
|
||
Use memory tracking APIs to track malloc memory associated with weak maps.
Depends on D34375
|
|
Assignee | |
Comment 39•5 years ago
|
||
Use memory tracking APIs to track malloc memory associated with Shapes. I had to thread FreeOp and BaseShape through in various places so there's enough context.
Depends on D34376
|
|
||
Comment 40•5 years ago
|
||
Pushed by jcoppeard@mozilla.com: https://hg.mozilla.org/integration/mozilla-inbound/rev/f3e11ae5fb19 Fix assertions that all tracked memory is removed when a zone is collected r=sfink https://hg.mozilla.org/integration/mozilla-inbound/rev/06ef8a540487 Move HeapSize class to gc/Scheduling.h where it belongs r=sfink https://hg.mozilla.org/integration/mozilla-inbound/rev/1b3207f97971 Add a separate byte count for malloc allocations r=sfink
|
|
Assignee | |
Comment 41•5 years ago
|
||
Use memory tracking APIs to track malloc memory associated with module IndirectBindingMap. These are attached to ModuleObjects and ModuleNamespaceObjects.
Depends on D34377
|
|
Assignee | |
Comment 42•5 years ago
|
||
Use memory tracking APIs to track malloc memory associated with IonScripts and BaselineScripts. This does the memory accounting when they are attached/detached from the JSScript rather than on initialisation/finalization as normally. I had to record the allocation size in the objects themselves. Hopefully this isOK.
Depends on D34551
|
|
Assignee | |
Comment 43•5 years ago
|
||
Use memory tracking APIs to track malloc memory associated with ArgumentsObjects. This one is slightly more complicated because we can allocate these in the nursery and malloc memory is not tracked for nursery objects, so we have to do this if they get tenured.
Depends on D34552
|
|
Assignee | |
Comment 44•5 years ago
|
||
Use memory tracking APIs to track malloc memory associated with NativeIterator objects. I had to store the initial propery count as propertiesEnd can change during the lifetime of this object.
Depends on D34553
|
|
Assignee | |
Comment 45•5 years ago
|
||
Depends on D34554
|
|
Assignee | |
Comment 46•5 years ago
|
||
I wanted to make VectorMatchPairs use ZoneAllocPolicy but this is also used in a bunch of places where it's not attached to a GC thing, so I left this as a todo.
Depends on D34555
|
||
Comment 47•5 years ago
|
||
| bugherder | ||
|
|
||
Comment 48•5 years ago
|
||
Pushed by jcoppeard@mozilla.com: https://hg.mozilla.org/integration/mozilla-inbound/rev/059c1897eb8f Track malloc memory used by JS Map and Set objects r=sfink https://hg.mozilla.org/integration/mozilla-inbound/rev/db2d48f8abf1 Track malloc memory associated with BigInts r=pbone https://hg.mozilla.org/integration/mozilla-inbound/rev/e4f684303335 Track malloc memory associated with Scopes r=jandem https://hg.mozilla.org/integration/mozilla-inbound/rev/0943e5feaa8d Track malloc memory associated with ctypes objects r=jandem https://hg.mozilla.org/integration/mozilla-inbound/rev/1a2985fdef25 Track malloc memory associated with WeakMap objects r=pbone https://hg.mozilla.org/integration/mozilla-inbound/rev/82c5c40a8881 Track malloc memory used by shapes r=jandem https://hg.mozilla.org/integration/mozilla-inbound/rev/c9c24c9e108c Track malloc memory used by IndirectBindingMaps associated with modules r=pbone
|
|
Assignee | |
Comment 49•5 years ago
|
||
Some kinds of memory use (e.g. reg exp byte code) require multiple memory associations for a single cell. This patch adds machinery to let that happen for specific uses.
Depends on D34556
|
|
Assignee | |
Comment 50•5 years ago
|
||
I had to store the length as the first word of the bytecode. Hopefully this is not too onerous. I also shuffled RegExpShared fields around to make the class a little smaller.
Depends on D34723
|
|
Assignee | |
Comment 51•5 years ago
|
||
Note that we only track the for typed arrays that are not backed by an ArrayBuffer as the memory is tracked there from then on.
Depends on D34726
|
|
Assignee | |
Comment 52•5 years ago
|
||
I'm not sure what these are, but track them anyway.
Depends on D34729
|
|
Assignee | |
Comment 53•5 years ago
|
||
This changes the format of the trace list from using -1 as a delimter to storing the list lengths up front so that we have length information.
Depends on D34730
|
||
Comment 54•5 years ago
|
||
| bugherder | ||
https://hg.mozilla.org/mozilla-central/rev/059c1897eb8f
https://hg.mozilla.org/mozilla-central/rev/db2d48f8abf1
https://hg.mozilla.org/mozilla-central/rev/e4f684303335
https://hg.mozilla.org/mozilla-central/rev/0943e5feaa8d
https://hg.mozilla.org/mozilla-central/rev/1a2985fdef25
https://hg.mozilla.org/mozilla-central/rev/82c5c40a8881
https://hg.mozilla.org/mozilla-central/rev/c9c24c9e108c
|
|
||
Comment 55•5 years ago
|
||
Pushed by jcoppeard@mozilla.com: https://hg.mozilla.org/integration/mozilla-inbound/rev/fa841547378a Track malloc memory used by JIT scrips r=jandem https://hg.mozilla.org/integration/mozilla-inbound/rev/10cde525f0c8 Track malloc memory used by Arguments objects r=sfink
|
|
||
Comment 56•5 years ago
|
||
Pushed by jcoppeard@mozilla.com: https://hg.mozilla.org/integration/mozilla-inbound/rev/1fca3e8b8e5e Track malloc memory used by FileObject objects r=sfink https://hg.mozilla.org/integration/mozilla-inbound/rev/37f4b005af23 Track malloc memory used by RegExpStatics objects r=pbone https://hg.mozilla.org/integration/mozilla-inbound/rev/d37bdcb35091 Allow multiple associations for some memory uses r=sfink https://hg.mozilla.org/integration/mozilla-inbound/rev/b6850f7bb6b6 Track malloc memory used by RegExp bytecode r=jandem https://hg.mozilla.org/integration/mozilla-inbound/rev/5d720afe50ea Track malloc memory used by non-inline TypedArray elements r=sfink https://hg.mozilla.org/integration/mozilla-inbound/rev/d7492a7c1b7d Track malloc memory used by PerfMesaurement objects r=sfink https://hg.mozilla.org/integration/mozilla-inbound/rev/9de6fe9f087e Track malloc memory used by TypedObject trace lists r=sfink
|
|
||
Comment 57•5 years ago
|
||
Backout by rgurzau@mozilla.com: https://hg.mozilla.org/integration/mozilla-inbound/rev/d8a83a5f0fb9 Backed out 7 changesets for SM build failures on a CLOSED TREE.
|
|
||
Comment 58•5 years ago
|
||
Backed out 7 changesets (bug 1395509) for SM build failures on a CLOSED TREE.
Backout link: https://hg.mozilla.org/integration/mozilla-inbound/rev/d8a83a5f0fb9165f97d8000976bb2241d44cda34
Push with failures: https://treeherder.mozilla.org/#/jobs?repo=mozilla-inbound&resultStatus=testfailed%2Cbusted%2Cexception&revision=9de6fe9f087eb386f829300206084da2a0861ea3&selectedJob=251889998
Log snippet:
[task 2019-06-14T13:48:17.817Z] File "/usr/lib/python2.7/subprocess.py", line 540, in check_call
[task 2019-06-14T13:48:17.817Z] raise CalledProcessError(retcode, cmd)
[task 2019-06-14T13:48:17.817Z] subprocess.CalledProcessError: Command '['/builds/worker/workspace/obj-haz-shell/dist/bin/js', '/builds/worker/checkouts/gecko/js/src/devtools/rootAnalysis/computeGCTypes.js', 'gcTypes.tmp', 'typeInfo.tmp']' returned non-zero exit status -11
[task 2019-06-14T13:48:17.861Z] Loaded /builds/worker/checkouts/gecko/js/src/devtools/rootAnalysis/t/out/suppression/defaults.py
[task 2019-06-14T13:48:17.861Z] Running gcTypes to generate ('gcTypes.txt', 'typeInfo.txt')
[task 2019-06-14T13:48:17.861Z] PATH="/builds/worker/workspace/sixgill/usr/bin:${PATH}" SOURCE='/builds/worker/checkouts/gecko/js/src/devtools/rootAnalysis/t/suppression' ANALYZED_OBJDIR='/builds/worker/checkouts/gecko/js/src/devtools/rootAnalysis/t/out/suppression' LD_LIBRARY_PATH="${LD_LIBRARY_PATH}:/builds/worker/workspace/obj-haz-shell/dist/bin" XDB='/builds/worker/workspace/sixgill/usr/bin/xdb.so' /builds/worker/workspace/obj-haz-shell/dist/bin/js /builds/worker/checkouts/gecko/js/src/devtools/rootAnalysis/computeGCTypes.js gcTypes.txt typeInfo.txt
[task 2019-06-14T13:48:17.887Z] Traceback (most recent call last):
[task 2019-06-14T13:48:17.887Z] File "/builds/worker/checkouts/gecko/js/src/devtools/rootAnalysis/analyze.py", line 339, in <module>
[task 2019-06-14T13:48:17.887Z] run_job(step, data)
[task 2019-06-14T13:48:17.887Z] File "/builds/worker/checkouts/gecko/js/src/devtools/rootAnalysis/analyze.py", line 209, in run_job
[task 2019-06-14T13:48:17.887Z] subprocess.check_call(command, env=env(config))
[task 2019-06-14T13:48:17.887Z] File "/usr/lib/python2.7/subprocess.py", line 540, in check_call
[task 2019-06-14T13:48:17.887Z] raise CalledProcessError(retcode, cmd)
|
|
Assignee | |
Comment 59•5 years ago
|
||
(In reply to Raul Gurzau (:RaulGurzau) from comment #58)
Steve do you know what happened here? It seems my patches broke the hazard analysis task somehow. I wasn't able to see anything useful in the logs. I'm currently trying to bisect this on try.
|
|
||
Comment 60•5 years ago
|
||
Pushed by jcoppeard@mozilla.com: https://hg.mozilla.org/integration/mozilla-inbound/rev/d3436ffbb75c Track malloc memory used by RegExpStatics objects r=pbone https://hg.mozilla.org/integration/mozilla-inbound/rev/bc1b32cdf4ca Allow multiple associations for some memory uses r=sfink https://hg.mozilla.org/integration/mozilla-inbound/rev/54817eb3c564 Track malloc memory used by RegExp bytecode r=jandem https://hg.mozilla.org/integration/mozilla-inbound/rev/848bf60827e4 Track malloc memory used by non-inline TypedArray elements r=sfink https://hg.mozilla.org/integration/mozilla-inbound/rev/219909ea3e4f Track malloc memory used by PerfMesaurement objects r=sfink https://hg.mozilla.org/integration/mozilla-inbound/rev/19f38c9cdd67 Track malloc memory used by TypedObject trace lists r=sfink
|
||
Comment 61•5 years ago
|
||
| bugherder | ||
https://hg.mozilla.org/mozilla-central/rev/d3436ffbb75c
https://hg.mozilla.org/mozilla-central/rev/bc1b32cdf4ca
https://hg.mozilla.org/mozilla-central/rev/54817eb3c564
https://hg.mozilla.org/mozilla-central/rev/848bf60827e4
https://hg.mozilla.org/mozilla-central/rev/219909ea3e4f
https://hg.mozilla.org/mozilla-central/rev/19f38c9cdd67
|
|
Assignee | |
Comment 62•5 years ago
|
||
Depends on D34554
|
|
Assignee | |
Comment 63•5 years ago
|
||
Depends on D35345
|
|
Assignee | |
Comment 64•5 years ago
|
||
Depends on D35346
|
|
Assignee | |
Comment 65•5 years ago
|
||
This refactors freeing stubs and adds memory tracking. It also adds a back pointer to the JSObject to ForOfPIC::Chain which is slightly annoying, but I think there's only one per global so this shouldn't be too bad.
Depends on D35347
|
|
||
Updated•5 years ago
|
|
|
||
Comment 66•5 years ago
|
||
Pushed by jcoppeard@mozilla.com: https://hg.mozilla.org/integration/mozilla-inbound/rev/d01efbed93ae Track malloc memory used by native iterator objects r=jandem https://hg.mozilla.org/integration/mozilla-inbound/rev/3c5dbd08506b Track malloc memory used by JitScripts r=jandem https://hg.mozilla.org/integration/mozilla-inbound/rev/340f561967eb Track malloc memory used by ObjectGroups r=tcampbell https://hg.mozilla.org/integration/mozilla-inbound/rev/fb304348f7b1 Track malloc memory used by script breakpoints r=sfink https://hg.mozilla.org/integration/mozilla-inbound/rev/2e0419940dd6 Track malloc memory used by ForOfPIC objects r=jandem
|
|
Assignee | |
Comment 67•5 years ago
|
||
|
|
Assignee | |
Comment 68•5 years ago
|
||
This adds tracking of malloc memory to WasmInstanceObject, WasmGlobalObject, WasmMemoryObject and ResolveResponseClosure (the straightforward cases).
Depends on D35484
|
|
||
Comment 69•5 years ago
|
||
| bugherder | ||
|
|
||
Comment 70•5 years ago
|
||
Pushed by jcoppeard@mozilla.com: https://hg.mozilla.org/integration/mozilla-inbound/rev/442d11bc5876 Add FreeOp methods to deal with releasing RefCounted types r=sfink https://hg.mozilla.org/integration/mozilla-inbound/rev/b396c73d8978 Track malloc memory used by wasm objects, part 1 r=luke
|
||
Comment 71•5 years ago
|
||
| bugherder | ||
|
|
Assignee | |
Comment 72•5 years ago
|
||
Add memory tracking for tables and modules. Tables can grow and so update their memory use when that happens (if associated with a JSObject). Modules have lots of associated data so calculate and cache a size on creation. The size doesn't have to be exact so this reuses the serialization framework to get an approximate value.
|
|
Assignee | |
Comment 73•5 years ago
|
||
The memory tracking needs the GC thing so I made WasmBreakpointSite hold a pointer to the Instance rather than just the DebugState. I added DebugState::getBreakpointSite for cases were we already expect this to exist, like JSScript::getBreakpointSite.
Depends on D35793
|
|
Assignee | |
Comment 74•5 years ago
|
||
Depends on D35795
|
|
Assignee | |
Comment 75•5 years ago
|
||
Depends on D35796
|
|
||
Updated•5 years ago
|
|
|
Assignee | |
Comment 76•5 years ago
|
||
Depends on D34555
|
|
Assignee | |
Comment 77•5 years ago
|
||
I realised I was using the wrong threshold to compare the malloc bytes counter against in ZoneAllocator::maybeMallocTriggerZoneGC. This was using the GC heap bytes threshold. In other places this correctly uses gcMallocThreshold.
|
|
||
Comment 78•5 years ago
|
||
Pushed by jcoppeard@mozilla.com: https://hg.mozilla.org/integration/mozilla-inbound/rev/d7cd95d34fb6 Use correct threshold for new malloc bytes counter r=sfink https://hg.mozilla.org/integration/mozilla-inbound/rev/68eecc44047f Track malloc memory used by wasm objects, part 2 r=luke
|
|
Assignee | |
Comment 79•5 years ago
|
||
Patch to allow ZoneAllocPolicy insstance to be moved after having memory accounted to it. This is useful in the situation where a data structure is populated before being attached to a GC thing by moving it. The code will currently assert (in unregisterPolicy()) if a ZoneAllocPolicy is copied after having memory accounted to it. This could be allowed but it seems better to force users to move in this situation.
Depends on D36175
|
|
Assignee | |
Comment 80•5 years ago
|
||
The patch switches some ctypes data structures to ZoneAllocPolicy so that their memory use is tracked against the zone. The creation of heapHash in StructType::DefineInternal relies on the ZoneAllocPolicy being movable after having memory accounted to it.
Depends on D36177
|
|
||
Comment 81•5 years ago
|
||
| bugherder | ||
|
|
||
Comment 82•5 years ago
|
||
Pushed by jcoppeard@mozilla.com: https://hg.mozilla.org/integration/mozilla-inbound/rev/f7088c17e423 Track malloc memory used by FileObject objects r=sfink
|
|
||
Comment 83•5 years ago
|
||
| bugherder | ||
|
|
Assignee | |
Comment 84•5 years ago
|
||
This replaces SystemAllocPolicy with ZoneAllocPolicy for things related to zones where possible. There are a few places where this isn't possible at the moment due to header include order. I'll try and revisit these later after removing the current malloc memory tracking as this simplifies the situation.
Depends on D36178
|
|
Assignee | |
Comment 85•5 years ago
|
||
This restores the use of GC invocation kind that was inadvertantly removed by the patch in bug 1395509 comment 33 from the call to ZoneAllocator::updateAllGCThresholds in GCRuntime::endSweepingSweepGroup. Also refactors ZoneMallocThreshold::computeZoneTriggerBytes a little.
Depends on D36923
|
|
Assignee | |
Comment 86•5 years ago
|
||
FF 69 is moving to beta. I'm going to close this bug and continue this work in bug 1564072. Outstanding patches will land as part of that bug.
|
|
Assignee | |
Updated•5 years ago
|
|
|
||
Comment 87•5 years ago
|
||
Comment on attachment 9073822 [details]
Bug 1395509 - Track malloc memory used by WasmBreakpoingSites r=luke?
Revision D35795 was moved to bug 1564072. Setting attachment 9073822 [details] to obsolete.
|
|
||
Comment 88•5 years ago
|
||
Comment on attachment 9073823 [details]
Bug 1395509 - Track malloc memory used by Breakpoints r=jimb?
Revision D35796 was moved to bug 1564072. Setting attachment 9073823 [details] to obsolete.
|
|
||
Comment 89•5 years ago
|
||
Comment on attachment 9073825 [details]
Bug 1395509 - Deprecate FreeOp APIs that don't track memory and rename them to make their use obvious r=sfink?
Revision D35798 was moved to bug 1564072. Setting attachment 9073825 [details] to obsolete.
|
|
||
Comment 90•5 years ago
|
||
Comment on attachment 9075979 [details]
Bug 1395509 - Associate more per-zone memory with the zone r=sfink?
Revision D36923 was moved to bug 1564072. Setting attachment 9075979 [details] to obsolete.
|
|
||
Comment 91•5 years ago
|
||
Comment on attachment 9075980 [details]
Bug 1395509 - Fix use of invocation kind that was previously lost and refactor malloc threshold updates r=sfink?
Revision D36924 was moved to bug 1564078. Setting attachment 9075980 [details] to obsolete.
|
||
Updated•5 years ago
|
Description
•