1303313 - Implement Frame Throughput (FT) for Telemetry/Profiler

Status: RESOLVED INCOMPLETE

(Core :: Graphics, defect)

Description

[:Dominik Strohmeier [:dstrohmeier]]

In the context of Progressive Web Metrics, Frame Throughput (FT) measures jank-free scrolling and animation, i.e. it measures the "always 60fps" paradigm.

The probe will also help us to better study and understand those cases where we won't be able to ensure 60fps and validate effects like frame drop patters/jitter.

Chromium's working draft for Frame Throughput is here: https://docs.google.com/document/d/1Bot91txCWZUstt32_BBLo-_HUsvOVn_L5yaI5xre2M0/edit#

Comment 1

[:Harald Kirschner :digitarald]

We need to translate Chrome's spec to our reality and come up with something that is comparable to Chrome's measures but also something that users actually perceive - a very iterative process.

Milan, who would be best to help with this?

Comment 2

[Milan Sreckovic [:milan] (needinfo for best results)]

Right - we will lock down the architecture of the updated renderer before deciding what makes sense to measure, and how.  It will be about the end of October before this would happen.

Comment 3

[:Harald Kirschner :digitarald]

(In reply to Milan Sreckovic [:milan] from comment #2)
> Right - we will lock down the architecture of the updated renderer before
> deciding what makes sense to measure, and how.  It will be about the end of
> October before this would happen.

This measure has to be in both current and next architecture so we are able to baseline.

The measure is not depending on low level implementation details, so should not differ massively. We need to start reviewing the spec with our platform in mind to know for sure, as it might be Chrome specific in some cases that need to be addressed.

Comment 4

[:Harald Kirschner :digitarald]

Clarified the minimal scope for this bug, which is only telemetry and profiler. Developer facing numbers depend on validation and spec stabilizing.

Comment 5

[Milan Sreckovic [:milan] (needinfo for best results)]

This is probably the most graphics specific measurement out of the few mentioned in the progressive web metrics.  It is also very clearly the early days in the life of the document from comment 0.  It seems to be at the brainstorming level, and the details that exist are very much Chrome specific.

Kats, in the context of the Appendix that mentioned checkerboarding (which to me would be one of the most useful measures in this context), can you put in this bug what we currently measure?  Perhaps we can start by sharing that with the Chrome team.

Comment 6

[Kartikaya Gupta (email:[e-mail])]

I read through the checkerboarding appendix in that document and honestly it makes no sense to me. I don't understand why the metric is recursive.

Anyhow what we are measuring is the severity, duration, and peak of each checkerboard event. A checkerboard event is a set of contiguous frames where checkerboarding is nonzero. The duration of the event is simply the sum of all the frame times for each of the frames in the event. The peak is the maximum number of pixels that were checkerboarded in any given frame during the event. And the severity is what you get if, for each frame during the event, you multiply the number of pixels checkerboarded by the frame time, and then sum them together.

Comment 7

[Martin Best (:mbest)]

Just pinging this bug.  This is one of our core measures for Quantum work and this seems to be stalled.  What is the current status of this?

Comment 8

[Milan Sreckovic [:milan] (needinfo for best results)]

We're a bit stuck on what to do here that can be measured and will change value when something meaningful changes.  The document from comment 0 doesn't really help in that sense, as it's specific to Chrome architecture, or has some confusing information (see comment 6.)
We have also had trouble measuring anything between different browsers, so even if we come up with something here, it will be Firefox specific, and only applicable to Firefox (e.g., no number comparison to Chrome.)

Comment 9

[:Harald Kirschner :digitarald]

Filed bug 1338347 as subset of this bug, only focusing on the compositor aspect.

Comment 10

[:Harald Kirschner :digitarald]

We are not trying to match Chrome's spec for now. Work to have a similar telemetry probe happens in bug 1338347.