1753192 - Use a common buffer in both Base and Gecko Profilers

Status: RESOLVED FIXED

(Core :: Gecko Profiler, task, P2)

Description

[Gerald Squelart (he/him) (not at Mozilla since 2022-09-15)]

Currently each profiler uses its own buffer, and when the Gecko Profiler (GP) starts while the Base Profiler (BP) is active, GP retrieves the full profile from BP and stops BP. This is quite expensive in memory and work.

Both profilers use the exact same serialization mechanism, so GP can read BP data straight from the buffer.

So the idea is to have only one buffer overall, initially created by BP, and then handed over to GP without any processing, and hopefully with minimal interruption.

This will also allow base markers to continue targeting the mozglue-visible common buffer, so they will be recorded even when BP has stopped (and GP is active).

Comment 1

[Gerald Squelart (he/him) (not at Mozilla since 2022-09-15)]

Attachment: Bug 1753192 - Move InChunkPointer to new header ProfileChunkedBufferDetail.h - r?canaltinova

Depends on D137392

Comment 2

[Gerald Squelart (he/him) (not at Mozilla since 2022-09-15)]

Attachment: Bug 1753192 - Removed unused ProfileChunkedBuffer::ExtractChunkManager() - r?canaltinova

Depends on D137800

Comment 3

[Gerald Squelart (he/him) (not at Mozilla since 2022-09-15)]

Attachment: Bug 1753192 - Move core ProfileChunkedBuffers to static singletons in profiler_get_core_buffer() - r?canaltinova

Note that they are still separate buffers for now.

Depends on D137801

Comment 4

[Gerald Squelart (he/him) (not at Mozilla since 2022-09-15)]

Attachment: Bug 1753192 - Make ActivePS::mProfileBufferChunkManager a UniquePtr, to allow transferring it between profilers - r?canaltinova

Depends on D137802

Comment 5

[Gerald Squelart (he/him) (not at Mozilla since 2022-09-15)]

Attachment: Bug 1753192 - Use combined core buffer, and transfer ownership of chunk manager from base to gecko profiler - r?canaltinova

Depends on D137803

Comment 6

[Gerald Squelart (he/him) (not at Mozilla since 2022-09-15)]

Attachment: Bug 1753192 - Add handover marker - r?canaltinova

Depends on D137804

Comment 7

[Gerald Squelart (he/him) (not at Mozilla since 2022-09-15)]

Attachment: Bug 1753192 - Fix apparent negative leak of RequestedChunkRefCountedHolder - r?canaltinova

Once the buffers are combined, a RequestedChunkRefCountedHolder could be referenced from the Base Profiler, which is invisible to the leak catcher; then handed over to the Gecko Profiler where it will eventually be dereferenced, which is logged with the leak catcher, resulting in an apparent negative leak.
This is fixed by keeping all (de)references secret.

Comment 8

[Gerald Squelart (he/him) (not at Mozilla since 2022-09-15)]

Attachment: Bug 1753192 - ThreadRegistrationInfo reuses the Base Profiler thread registration time if any - r?canaltinova

Once the buffers are combined, some data could be stored in a thread before its registration with the Gecko Profiler.
Without this fix, the front-end would think the thread had not started yet, which could prevent interacting with that early data.

Depends on D138238

Comment 9

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/5bc8bd033453
Move InChunkPointer to new header ProfileChunkedBufferDetail.h - r=canaltinova
https://hg.mozilla.org/integration/autoland/rev/1d5d69d92db9
Removed unused ProfileChunkedBuffer::ExtractChunkManager() - r=canaltinova
https://hg.mozilla.org/integration/autoland/rev/c8a0d3c41d2f
Move core ProfileChunkedBuffers to static singletons in profiler_get_core_buffer() - r=canaltinova
https://hg.mozilla.org/integration/autoland/rev/c2f22d8a8fc8
Make ActivePS::mProfileBufferChunkManager a UniquePtr, to allow transferring it between profilers - r=canaltinova
https://hg.mozilla.org/integration/autoland/rev/f79cd543e537
Fix apparent negative leak of RequestedChunkRefCountedHolder - r=canaltinova
https://hg.mozilla.org/integration/autoland/rev/07a8ffa8d12f
ThreadRegistrationInfo reuses the Base Profiler thread registration time if any - r=canaltinova
https://hg.mozilla.org/integration/autoland/rev/1a71d954b83f
Use combined core buffer, and transfer ownership of chunk manager from base to gecko profiler - r=canaltinova
https://hg.mozilla.org/integration/autoland/rev/2f0c24b1f049
Add handover marker - r=canaltinova

Comment 10

[Marian-Vasile Laza]

Backed out 8 changesets (Bug 1753192) for causing Gtest failures on GeckoProfiler.cpp.
Backout link
Push with failures
Failure Log

Comment 11

[Gerald Squelart (he/him) (not at Mozilla since 2022-09-15)]

I forgot to update a Gecko Profiler gtest to match the new behavior in https://phabricator.services.mozilla.com/D138239 , where a thread may now get its registration time from an earlier registration in the Base Profiler.

Comment 12

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/5f7c37c11c3a
Move InChunkPointer to new header ProfileChunkedBufferDetail.h - r=canaltinova
https://hg.mozilla.org/integration/autoland/rev/88cdd3bb1df1
Removed unused ProfileChunkedBuffer::ExtractChunkManager() - r=canaltinova
https://hg.mozilla.org/integration/autoland/rev/3747cc0c4b8e
Move core ProfileChunkedBuffers to static singletons in profiler_get_core_buffer() - r=canaltinova
https://hg.mozilla.org/integration/autoland/rev/4f575897b67f
Make ActivePS::mProfileBufferChunkManager a UniquePtr, to allow transferring it between profilers - r=canaltinova
https://hg.mozilla.org/integration/autoland/rev/bf1925f95330
Fix apparent negative leak of RequestedChunkRefCountedHolder - r=canaltinova
https://hg.mozilla.org/integration/autoland/rev/506ef4b3a636
ThreadRegistrationInfo reuses the Base Profiler thread registration time if any - r=canaltinova
https://hg.mozilla.org/integration/autoland/rev/b60043138a22
Use combined core buffer, and transfer ownership of chunk manager from base to gecko profiler - r=canaltinova
https://hg.mozilla.org/integration/autoland/rev/c40948535daf
Add handover marker - r=canaltinova

Comment 13

[Marian-Vasile Laza]

https://hg.mozilla.org/mozilla-central/rev/5f7c37c11c3a
https://hg.mozilla.org/mozilla-central/rev/88cdd3bb1df1
https://hg.mozilla.org/mozilla-central/rev/3747cc0c4b8e
https://hg.mozilla.org/mozilla-central/rev/4f575897b67f
https://hg.mozilla.org/mozilla-central/rev/bf1925f95330
https://hg.mozilla.org/mozilla-central/rev/506ef4b3a636
https://hg.mozilla.org/mozilla-central/rev/b60043138a22
https://hg.mozilla.org/mozilla-central/rev/c40948535daf