With a buffersize in the UI of 90MB, I see in about:memory that profiler-state for every process (including GPU...) is 144MB (Windows 10, x64).
Note also that using this much memory per-process will be a problem in Fission; it's already a problem on a low-memory machine like the 2017 reference laptop with 4GB - 4 tabs open, almost no browsing (3 are profiles), and we have 9 processes * 144MB just for profiler state.
The 90MB from the UI are calculated as 9 bytes per entry * 10000000 entries. However, when requesting 10000000 entries, the ProfileBuffer constructor rounds it up to the next power of two, which is 16777216. So we request an allocation of 16777216 * 9 = 150994944 bytes, which is 144MiB.
We could fix this by making the UI snap to entry counts which are powers of two.
So this would be a bug for https://github.com/firefox-devtools/Gecko-Profiler-Addon ?
Also I think the UI isn't clear that the memory amount is per process (or even per thread?).
Fixed by bug 1634216 and bug 1634234: about:profiling now shows the exact power of 2 in bytes that the profiler will use.
Also, thanks to bug 1632750 this limit is now for the whole Firefox application, instead of a local limit in each process.