2085019 - Reduce visual metrics variance by keeping exact frame timestamps in browsertime videos

Status: ASSIGNED

(Testing :: Raptor, enhancement)

Description

[Andrew Creskey [:acreskey]]

Browsertime videos snap frame times to a 40ms grid on Linux/macOS (ffmpeg concat at 25fps) and 33ms on Windows (30fps capture).
Removing the snap may reduce visual metrics variance, especially for short loads like google-nav.

Comment 1

[Andrew Creskey [:acreskey]]

Perf-compare, just started

Comment 2

[Andrew Creskey [:acreskey]]

Updated PerfCompare, keeping millisecond frame timestamps:
Reduces nav-bench CoV by ~20% on Linux and ~28% on macOS with no change on Windows, which records via ffmpeg.

• Linux google-nav-load: before, 73 of 100 cycles read exactly 280ms, so the score could only move in 40ms steps. After, values spread over 250–330ms and CoV drops from 14.1% to 4.9%.
• mac duckduckgo-subnav: before, results split into two clusters (about 220ms, or 120–190ms) depending on which slot the paint fell into. After, every cycle lands at 150–180ms and CoV drops from 12.8% to 3.8%.

Comment 3

[Andrew Creskey [:acreskey]]

Attachment: WIP: Bug 2085019 - WIP: Keep millisecond frame timestamps in browsertime videos for nav-bench. r?kshampur,#perftest-reviewers!

The window recorder snaps video frames to a 40ms grid, which adds visual metrics noise; opt nav-bench in first.

Comment 4

[Andrew Creskey [:acreskey]]

I think the cleanest way is to fix this in upstream browsertime, logged here: https://github.com/sitespeedio/browsertime/issues/2640