1626837 - Implement ProfileChunkedBuffer

Status: RESOLVED FIXED

(Core :: Gecko Profiler, task, P2)

Description

[Gerald Squelart (he/him) (not at Mozilla since 2022-09-15)]

ProfileChunkedBuffer simulates a near-infinite buffer over ProfileBufferChunks.
It uses a ProfileBufferChunkManager to get chunks and later release them.
Its use is similar to BlocksRingBuffer:

• It reserves blocks in chunks, adds some structure (just the size of the entry that follows), and lets a user-provided writer write the entry.
• It allows reading past entries.
• It can be in an "out-of-session" state where APIs are still available but do nothing.

It is intended to be a replacement for BlocksRingBuffer.

Comment 1

[Gerald Squelart (he/him) (not at Mozilla since 2022-09-15)]

Attachment: Bug 1626837 - Add BlocksRingBuffer::IsInSession() - r?canaltinova!,gregtatum

ProfileBuffer used to check if a BlocksRingBuffer was in session by looking
at its buffer size.
Now IsInSession() should be used, it is clearer in intent, and will make the
transition to ProfileChunkedBuffer slightly easier.

Comment 2

[Gerald Squelart (he/him) (not at Mozilla since 2022-09-15)]

Attachment: Bug 1626837 - ProfileChunkedBuffer basic data and functions - r?canaltinova!,gregtatum

ProfileChunkedBuffer simulates a near-infinite buffer over ProfileBufferChunks.
It uses a ProfileBufferChunkManager to get chunks and later release them.
Its use is similar to BlocksRingBuffer:

• It reserves blocks in chunks, adds some structure (just the size of the entry
  that follows), and lets a user-provided writer write the entry.
• It allows reading past entries.
• It can be in an "out-of-session" state where APIs are still available but do
  nothing.

It is intended to eventually replace BlocksRingBuffer.

This patch starts with the basic structure, following patches will add all
planned features.

Depends on D69492

Comment 3

[Gerald Squelart (he/him) (not at Mozilla since 2022-09-15)]

Attachment: Bug 1626837 - ProfileChunkedBuffer ChunkManager handling - r?canaltinova!,gregtatum

ProfileChunkedBuffer can handle zero of one ProfileBufferChunkManager at a
time, and can optionally take ownership of the manager.

Depends on D69493

Comment 4

[Gerald Squelart (he/him) (not at Mozilla since 2022-09-15)]

Attachment: Bug 1626837 - ProfileChunkedBuffer chunk request mechanism - r?canaltinova!,gregtatum

To ensure that a spare chunk is ready to handle data that will eventually
overflow the current chunk, ProfileChunkedBuffer uses
ProfileBufferChunk::RequestChunk() to queue a request for a new chunk.

This request should be handled off-thread by the buffer user -- but a response
is not guaranteed, so the buffer does not rely on it and can get a new chunk
on the spot if really needed.

Because the request is asynchronous, and because either the buffer or the user
could be destroyed while a request is in flight, a shared
RequestedChunkRefCountedHolder object is used:

• When the request is handled, the new chunk (or nullptr) is given to the
  holder.
• When the buffer needs a new chunk, it can retrieve the new chunk if the
  request was successfully fulfilled.
  If the requestee is destroyed first, the request won't be fulfilled and the
  buffer will carry on without relying on requests.
  If the requester is destroyed first, the holder (with a potential requested
  chunk) will just get destroyed after the request is fulfilled or the requestee
  is destroyed as well.

Depends on D69494

Comment 5

[Gerald Squelart (he/him) (not at Mozilla since 2022-09-15)]

Attachment: Bug 1626837 - ProfileChunkedBuffer write functions - r?canaltinova!,gregtatum

Depends on D69495

Comment 6

[Gerald Squelart (he/him) (not at Mozilla since 2022-09-15)]

Attachment: Bug 1626837 - ProfileChunkedBuffer detail::InChunkPointer - r?canaltinova!,gregtatum

InChunkPointer is an internal accessor pointing at a position inside a chunk.
It can handle up to two groups of chunks (typically the extant chunks stored in
the chunk manager, and the current chunk.

Depends on D69496

Comment 7

[Gerald Squelart (he/him) (not at Mozilla since 2022-09-15)]

Attachment: Bug 1626837 - ProfileChunkedBuffer read functions - r?canaltinova!,gregtatum

Depends on D69497

Comment 8

[Gerald Squelart (he/him) (not at Mozilla since 2022-09-15)]

Attachment: Bug 1626837 - ProfileChunkedBuffer (de)serialization - r?canaltinova!,gregtatum

This is needed to embed a small buffer (e.g., containing one backtrace attached
to a marker) into a bigger buffer (e.g., the main profiler buffer).

Depends on D69498

Comment 9

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/e207f1d1f211
Add BlocksRingBuffer::IsInSession() - r=canaltinova
https://hg.mozilla.org/integration/autoland/rev/e4f1e52ecd5b
ProfileChunkedBuffer basic data and functions - r=canaltinova
https://hg.mozilla.org/integration/autoland/rev/3df5df7105cd
ProfileChunkedBuffer ChunkManager handling - r=canaltinova
https://hg.mozilla.org/integration/autoland/rev/c8e55e1a9e86
ProfileChunkedBuffer chunk request mechanism - r=canaltinova
https://hg.mozilla.org/integration/autoland/rev/a1edf5289acd
ProfileChunkedBuffer write functions - r=canaltinova
https://hg.mozilla.org/integration/autoland/rev/7ab49e4c1621
ProfileChunkedBuffer detail::InChunkPointer - r=canaltinova
https://hg.mozilla.org/integration/autoland/rev/f6a19e0ba1a9
ProfileChunkedBuffer read functions - r=canaltinova
https://hg.mozilla.org/integration/autoland/rev/8243d7c3eeaf
ProfileChunkedBuffer (de)serialization - r=canaltinova

Comment 10

[Stefan Hindli [:stefan_hindli]]

https://hg.mozilla.org/mozilla-central/rev/e207f1d1f211
https://hg.mozilla.org/mozilla-central/rev/e4f1e52ecd5b
https://hg.mozilla.org/mozilla-central/rev/3df5df7105cd
https://hg.mozilla.org/mozilla-central/rev/c8e55e1a9e86
https://hg.mozilla.org/mozilla-central/rev/a1edf5289acd
https://hg.mozilla.org/mozilla-central/rev/7ab49e4c1621
https://hg.mozilla.org/mozilla-central/rev/f6a19e0ba1a9
https://hg.mozilla.org/mozilla-central/rev/8243d7c3eeaf