1603246 - Delete the Gecko Profiler WebExtension API

Status: NEW

(DevTools :: Performance Tools (Profiler/Timeline), task, P3)

Description

[Greg Tatum [:gregtatum]]

After removing the add-on, we should remove the Gecko Profiler WebExtension API. I'm not aware of any other consumers of it, but we should double check this before doing it.

Comment 1

[Greg Tatum [:gregtatum]]

Hello Raptor friends. I believe Raptor is built using the Gecko Profiler secret WebExtension API. We no longer use this with the Firefox Profiler, and the API has been somewhat unmaintained since we started work on the new integrated profiler recording mechanism. What are your plans and needs with this API, as it's also somewhat of a burden on us to maintain due to code duplication, and we'd be interested in removing it.

I'm asking this based on the module ownership information for Raptor:
Owner: Dave Hunt
Peer(s): Henrik Skupin, Greg Mierzwinski

Comment 2

[Greg Tatum [:gregtatum]]

Oh, and maybe it's used in other test tools, I had this conversation a few months ago and may be forgetting who exactly is using the API.

Comment 3

[Henrik Skupin [:whimboo][⌚️UTC+2]]

Do you have more information? Some URLs to sources in mozilla-central? I don't know what this bug is about.

Comment 4

[Greg Tatum [:gregtatum]]

Sorry about the terseness of information. Looks like this work was added by Rob Wood, so you all may not have context.

It looks like Raptor is using the "geckoProfiler" webextension API. We built this API to be used to with the Gecko Profiler Addon. A month or so ago we finally removed the addon, and integrated the UI directly into Firefox. At this point the API has been fairly unmaintained, and now it's becoming more divergent with the realities of the profiler. Looking into it a bit more, Raptor is the only remaining consumer of this API. The work with Raptor was done without much conversation with the perf tools team, so we're not really up to date with the requirements or needs. The point of this bug is to track the decision on whether it's feasible to delete this API.

• Here is where Raptor is giving itself permission to use this API
• Here is where Raptor is using the API

The WebExtension API is a bit of a burden to keep up to date as we add and change features, and we would prefer to minimize APIs that can access the profiler, especially as extensions have a decent amount of complexity.

So I guess the questions I have is:

• Is there a way to do this work without the API?
• Is it feasible to delete the WebExtension?
• If not, can we simplify the API to make it easier to maintain.

Perhaps it would be good to have some synchronous conversation on a call or chat to figure this out.

Comment 5

[Henrik Skupin [:whimboo][⌚️UTC+2]]

Ok, I see. So here my thoughts from when I worked on pieces of that like bug 1550702. That's what we need:

• Set various options for running the profiler like entries in the buffer, and interval
• Allow setting of threads and features to include in the profile
• Start / stopping the profiler
• Storing the profile to disk and letting Raptor know where it can find it

I think that those are the features we need. Maybe you can explain which replacements exists for the internal profiler. Are those only the environment variables?

Comment 6

[Greg Tatum [:gregtatum]]

The environment variables should all be very well-supported and reflect the current state of the profiler. They are also really easy to maintain as we change profiler internals, as they are very searchable in the codebase. They also have documentation via:

MOZ_PROFILER_HELP=1 ./mach run

Profiler environment variable usage:

  MOZ_PROFILER_HELP
  If set to any value, prints this message.
  Use MOZ_BASE_PROFILER_HELP for BaseProfiler help.

  MOZ_LOG
  Enables logging. The levels of logging available are
  'prof:3' (least verbose), 'prof:4', 'prof:5' (most verbose).

  MOZ_PROFILER_STARTUP
  If set to any value other than '' or '0'/'N'/'n', starts the
  profiler immediately on start-up.
  Useful if you want profile code that runs very early.

  MOZ_PROFILER_STARTUP_ENTRIES=<65536..268435456>
  If MOZ_PROFILER_STARTUP is set, specifies the number of entries per
  process in the profiler's circular buffer when the profiler is first
  started.
  If unset, the platform default is used:
  8388608 entries per process, or 67108864 when MOZ_PROFILER_STARTUP is set.
  (8 bytes per entry -> 67108864 or 536870912 total bytes per process)

  MOZ_PROFILER_STARTUP_DURATION=<1..>
  If MOZ_PROFILER_STARTUP is set, specifies the maximum life time of
  entries in the the profiler's circular buffer when the profiler is
  first started, in seconds.
  If unset, the life time of the entries will only be restricted by
  MOZ_PROFILER_STARTUP_ENTRIES (or its default value), and no
  additional time duration restriction will be applied.

  MOZ_PROFILER_STARTUP_INTERVAL=<1..5000>
  If MOZ_PROFILER_STARTUP is set, specifies the sample interval,
  measured in milliseconds, when the profiler is first started.
  If unset, the platform default is used.

  MOZ_PROFILER_STARTUP_FEATURES_BITFIELD=<Number>
  If MOZ_PROFILER_STARTUP is set, specifies the profiling features, as
  the integer value of the features bitfield.
  If unset, the value from MOZ_PROFILER_STARTUP_FEATURES is used.

  MOZ_PROFILER_STARTUP_FEATURES=<Features>
  If MOZ_PROFILER_STARTUP is set, specifies the profiling features, as
  a comma-separated list of strings.
  Ignored if  MOZ_PROFILER_STARTUP_FEATURES_BITFIELD is set.
  If unset, the platform default is used.

    Features: (x=unavailable, D/d=default/unavailable,
               S/s=MOZ_PROFILER_STARTUP extra default/unavailable)
    d      1: "java" (Profile Java code, Android only)
    D      2: "js" (Get the JS engine to expose the JS stack to the profiler)
    D      4: "leaf" (Include the C++ leaf node if not stackwalking)
    S      8: "mainthreadio" (Add main thread I/O to the profile)
    -     16: "privacy" (Do not include user-identifiable information)
    D     32: "screenshots" (Take a snapshot of the window on every composition)
    -     64: "seqstyle" (Disable parallel traversal in styling)
    D    128: "stackwalk" (Walk the C++ stack, not available on all platforms)
    x    256: "tasktracer" (Start profiling with feature TaskTracer)
    D    512: "threads" (Profile the registered secondary threads)
    -   1024: "trackopts" (Have the JavaScript engine track JIT optimizations)
    x   2048: "jstracer" (Enable tracing of the JavaScript engine)
    -   4096: "jsallocations" (Have the JavaScript engine track allocations)
    -  16384: "nostacksampling" (Disable all stack sampling: Cancels "js", "leaf", "stackwalk" and labels)
    -  32768: "preferencereads" (Track when preferences are read)
    -  65536: "nativeallocations" (Collect the stacks from a smaller subset of all native allocations, biasing towards collecting larger allocations)
    - 131072: "ipcmessages" (Have the IPC layer track cross-process messages)
    -        "default" (All above D+S defaults)

  MOZ_PROFILER_STARTUP_FILTERS=<Filters>
  If MOZ_PROFILER_STARTUP is set, specifies the thread filters, as a
  comma-separated list of strings. A given thread will be sampled if
  any of the filters is a case-insensitive substring of the thread
  name. If unset, a default is used.

  MOZ_PROFILER_SHUTDOWN
  If set, the profiler saves a profile to the named file on shutdown.

  MOZ_PROFILER_SYMBOLICATE
  If set, the profiler will pre-symbolicate profiles.
  *Note* This will add a significant pause when gathering data, and
  is intended mainly for local development.

  MOZ_PROFILER_LUL_TEST
  If set to any value, runs LUL unit tests at startup.

  This platform supports native unwinding.

In my work of exposing the profiler to mochitests, I ended up controlling the profiler through these environment variables (see Bug 1628073). I even suggested in the mach help of the command line flag to run the MOZ_PROFILER_HELP for documentation on tweaking the values. I know curently mapping some CLI flags to the actual settings can be a bit awkward, especially as we change what those settings can do. Would it be feasible to use just these for Raptor? That would be great for having a much simpler and more maintainable API. If so I can file bugs for that work, and block this one with it.

Comment 7

[Henrik Skupin [:whimboo][⌚️UTC+2]]

Yes, that sounds great. The only problem I can see here is that we wouldn't be able to get separate profiles for warm page load tests, which do not restart Firefox in between each test. That way it will be hard to distinguish the individual page cycles. But to fix that we could add custom markers to the profile, and might have to reduce the interval or increase the entries to cover the full test job.

We will discuss that change in our next perftest triage meeting next Tuesday.

Comment 8

[Henrik Skupin [:whimboo][⌚️UTC+2]]

Oh sorry, that is actually in the devtools product. Greg, feel free to file a bug for Raptor and we can discuss anything else there. Thanks!

Comment 9

[Dave Hunt [:davehunt] [he/him] ⌚BST]

I agree with what's been said already. It would be great to modernise profile collection in Raptor. Would someone in the perf tools team be able to help us if this is needed in the short term?

Comment 10

[Greg Tatum [:gregtatum]]

Dave: Sounds great, I filed Bug 1640169. I probably have the most context here on the perf tools team, so I'm up for helping out.

Gerald: do you sense for the priority of removing the WebExtensions API on your end?

Comment 11

[Gerald Squelart (he/him) (not at Mozilla since 2022-09-15)]

Low priority for me, I was only hurt by the paper cut of having its tests fail when I modified the profiler feature list! And it may delay moving the buffer size from "entries" to "bytes", but it's not that important either.