1654413 - Add better access to markers from Rust

Status: RESOLVED FIXED

(Core :: Gecko Profiler, task, P3)

Description

[Gerald Squelart (he/him) (not at Mozilla since 2022-09-15)]

There are already some simple hooks in https://searchfox.org/mozilla-central/rev/3b6958c26049c1e27b2790a43154caaba9f6dd4a/gfx/webrender_bindings/src/bindings.rs#847

After meta bug 1640956 lands (or at least bug 1646266), it would be useful to offer more complete access to profiler markers.

But since the new C++ API is templated and variadic, I don't think it's possible to just add bindings. I'm thinking that it may be possible to create a similar generic interface in Rust, and have it serialize the input the same way C++ does (it's really only C PODs and strings in the profile buffer) and pass binary blobs to C++; The later JSONification will need more thoughts!
Newbie disclaimer: I haven't coded anything real in Rust yet, so I may be wrong about many things, don't trust me!

Comment 1

[Nazım Can Altınova [:canova][:canaltinova on phabricator]]

Attachment: Bug 1654413 - Rerun the profiler API build.rs when the included header files are changed r?emilio

For some reason, I thought that parse_callbacks(Box::new(CargoCallbacks))
would also automatically adds the initial header files that are manually added.
But apparently parse_callbacks is only executed when the inner included
header files are found.

Comment 2

[Nazım Can Altınova [:canova][:canaltinova on phabricator]]

Attachment: Bug 1654413 - Add ProfilerTime Rust struct for marker API r?emilio!,gerald!

This will be used to get the time from profiler, so marker API can use it to
understand the start or end time.

Depends on D124017

Comment 3

[Nazım Can Altınova [:canova][:canaltinova on phabricator]]

Attachment: Bug 1654413 - Add a small gecko_profiler_category macro as a sugar for marker categories r?emilio!

This is a syntactical sugar to write the categories more easily. I prefer to
have this macro because this is the same syntax as the categories in the label
frames API.

Depends on D124018

Comment 4

[Nazım Can Altınova [:canova][:canaltinova on phabricator]]

Attachment: Bug 1654413 - Add a MarkerTiming and MarkerOptions Rust structs for the marker API r?emilio!,gerald!

These structs are needed for the marker APIs. We also have the same structs as
the C++ classes. See mozilla::MarkerTiming and mozilla::MarkerOptions.

Depends on D124019

Comment 5

[Nazım Can Altınova [:canova][:canaltinova on phabricator]]

Attachment: Bug 1654413 - Add a MarkerStack Rust enum for the marker API r?emilio!,gerald!

MarkerStack object is a bit more complex in the C++ side. But we don't need a
complex object like that in here. A simple enum is enough to determine what
type of marker stack we are capturing. C++ side will be handling the capturing
of the stack.

Depends on D124020

Comment 6

[Nazım Can Altınova [:canova][:canaltinova on phabricator]]

Attachment: Bug 1654413 - Add add_untyped_marker Rust API r?emilio!,gerald!

This is the first and simplest API for the markers. There will be two more
APIs in the following patches (add_text_marker and add_marker). You can see the
PROFILER_MARKER_UNTYPED macro for the C++ counterpart.

Depends on D124021

Comment 7

[Nazım Can Altınova [:canova][:canaltinova on phabricator]]

Attachment: Bug 1654413 - Add add_text_marker Rust API r?emilio!,gerald!

This is the second API for the markers. This allows one payload as a text for
more information. See the PROFILER_MARKER_TEXT macro for the C++ counterpart.

Depends on D124022

Comment 8

[Nazım Can Altınova [:canova][:canaltinova on phabricator]]

Attachment: Bug 1654413 - Add JSONWriter struct for marker schema r?emilio!,gerald!

JSON writer will be used for the third and last marker API. This is needed
because we need to describe a marker payload struct on how to serialize it.

Depends on D124023

Comment 9

[Nazım Can Altınova [:canova][:canaltinova on phabricator]]

Attachment: Bug 1654413 - Add MarkerSchema struct and its enums for marker API r?emilio!,gerald!

You can see the mozilla::MarkerSchema for the C++ counterpart. This Rust
struct simply wraps the C++ object and keeps the reference of it as RAII. This
heap allocates the inner C++ object but it's fine to do it here, because it's
we only create a MarkerSchema object at the end of a profiling session and it
happens once per marker type. It should be very rare.

Depends on D124024

Comment 10

[Nazım Can Altınova [:canova][:canaltinova on phabricator]]

Attachment: Bug 1654413 - Add add_marker Rust profiler API and serialize the marker to the buffer r?emilio!,gerald!

This only adds the API and then adds the profiler payload to the buffer. The
deserialization and streaming will happen in the next patch.

Depends on D124025

Comment 11

[Nazım Can Altınova [:canova][:canaltinova on phabricator]]

Attachment: Bug 1654413 - Deserialize and stream the markers and marker schemas r?emilio!,gerald!

Now we can deserialize and stream everything to the JSON.

Depends on D124026

Comment 12

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/6974cc158d51
Rerun the profiler API build.rs when the included header files are changed r=emilio
https://hg.mozilla.org/integration/autoland/rev/57fe3b6c0acf
Add ProfilerTime Rust struct for marker API r=emilio,gerald
https://hg.mozilla.org/integration/autoland/rev/99e2adc9f848
Add a small gecko_profiler_category macro as a sugar for marker categories r=emilio
https://hg.mozilla.org/integration/autoland/rev/36660c399584
Add a MarkerTiming and MarkerOptions Rust structs for the marker API r=emilio,gerald
https://hg.mozilla.org/integration/autoland/rev/89debdc6aaac
Add a MarkerStack Rust enum for the marker API r=emilio,gerald
https://hg.mozilla.org/integration/autoland/rev/c82aee76d44d
Add add_untyped_marker Rust API r=emilio,gerald
https://hg.mozilla.org/integration/autoland/rev/fde7a635924e
Add add_text_marker Rust API r=emilio,gerald
https://hg.mozilla.org/integration/autoland/rev/680cd33b1642
Add JSONWriter struct for marker schema r=emilio,gerald
https://hg.mozilla.org/integration/autoland/rev/65e0c7376519
Add MarkerSchema struct and its enums for marker API  r=emilio,gerald
https://hg.mozilla.org/integration/autoland/rev/5ac58c00d75b
Add add_marker Rust profiler API and serialize the marker to the buffer r=emilio,gerald
https://hg.mozilla.org/integration/autoland/rev/8d2ac26f5f6f
Deserialize and stream the markers and marker schemas r=emilio,gerald

Comment 13

[Narcis Beleuzu [:NarcisB]]

https://hg.mozilla.org/mozilla-central/rev/6974cc158d51
https://hg.mozilla.org/mozilla-central/rev/57fe3b6c0acf
https://hg.mozilla.org/mozilla-central/rev/99e2adc9f848
https://hg.mozilla.org/mozilla-central/rev/36660c399584
https://hg.mozilla.org/mozilla-central/rev/89debdc6aaac
https://hg.mozilla.org/mozilla-central/rev/c82aee76d44d
https://hg.mozilla.org/mozilla-central/rev/fde7a635924e
https://hg.mozilla.org/mozilla-central/rev/680cd33b1642
https://hg.mozilla.org/mozilla-central/rev/65e0c7376519
https://hg.mozilla.org/mozilla-central/rev/5ac58c00d75b
https://hg.mozilla.org/mozilla-central/rev/8d2ac26f5f6f