2041090 - Add information about Speedometer 3 to performance docs

Status: NEW

(Core :: Performance: General, task)

Description

[Abhishek Nimalan [:animalan]]

It would be good to have a section in perfdocs discussing the Speedometer 3 benchmark. We should mention (not an exhaustive list):

• What is Speedometer 3?
• What does it measure?
• How to test patches with the Speedometer 3 benchmark / using perftest to collect Speedometer 3 information (metrics, profiles, comparisons, etc.)?
• A tutorial/example showing how to use Speedometer 3 information from Comment 0 to diagnose, investigate, and resolve a performance regression.

A dedicated Speedometer 3 section is useful because:

• It can be linked in Comment 0 and help developers understand how to make use of the benchmark information given in Comment 0.
• It can introduce new developers to Speedometer 3 benchmarking and how to consider performance in their patches
• It can provide better context to our AI tools

Comment 1

[Kash Shampur [:kshampur] ⌚EST]

(following up from sp3 meeting)
reviewing the meeting notes around May, i believe this documentation was going to be an action item for the work week. AFAICT i did not see this happen. Still this should be worked on. repurposing this bug :animalan filed for this.

we had determined in the meeting we should put it around here https://firefox-source-docs.mozilla.org/performance/index.html

Comment 2

[Kash Shampur [:kshampur] ⌚EST]

:fdoty or someone please close this as duplicate if there already was something (i was unable to find it)

Comment 3

[Frank Doty [:fdoty]]

mstange will answer question 3 & 4 from comment 0 in this bug for review. Here is what I plan on putting into the documentation for questions 1 and 2:

What is Speedometer 3?

Speedometer 3 is a cross-browser benchmark that measures web application responsiveness - how quickly a browser can react to user interactions in realistic web apps. It was released in March 2024 and is notable for being the first major browser benchmark jointly developed and governed by all three major engine teams: Apple (WebKit), Google (Chromium/Blink), and Mozilla (Gecko), hosted under a shared open-governance model. That shared ownership matters politically and technically: a good score is a target all three browsers agree is meaningful, rather than one vendor's definition of "fast."

It supersedes Speedometer 2.0 (2018) and 1.0 (2014). The current point release is 3.1 (March 2025), which brought harness-accuracy fixes rather than workload changes. The next iteration of Speedometer is expected to land in 2027.

What does it measure?

Speedometer runs a battery of test suites, each a small but complete web application, and drives them through scripted user interactions (adding to-do items, typing, clicking, editing, navigating). It measures the wall-clock time to complete each interaction end-to-end, including the synchronous work and the asynchronous rendering work the browser must finish before the UI is actually updated and ready for the next interaction.

• It's an interaction-latency benchmark, not a microbenchmark. The score reflects the full round trip of a user action: running the app's JavaScript, recalculating styles and layout, painting, and compositing. This exercises the whole browser stack, not just the JS engine.

• The workloads are representative of real frameworks. Test suites are built with the tools real developers use - React, Vue, Angular, Preact, Svelte, Lit, plus vanilla JS and jQuery, and non-trivial cases like a TodoMVC variant with a large/complex DOM, a news site, a charting app, and text/code editors. This is deliberate: it stresses the code paths real sites hit.

• What contributes to the score: DOM manipulation, style recalculation, layout/reflow, painting and compositing, JavaScript execution, and garbage-collection pauses that block interaction.

• The final score is a single number (higher is better) derived from the aggregate of how many of these interaction runs the browser can complete per unit time, reported with a confidence interval so run-to-run noise is visible.