2083927 - Create consts for `NODE_COUNT` use in performance tests

Status: RESOLVED FIXED

(Core :: Disability Access APIs, task)

Description

[Morgan Reschenberg [:morgan]]

Right now, our performance tests use a variety of test-specific numbers (500, 2000, 10000, etc.) in their test cases.
We should abstract these to constants to start, and put them in head.js in the perftests directory.

In the long run, it might make sense to support only a few numbers here (maybe SMALL, MEDIUM, and LARGE constants?) for our tests, and we should use them consistently.
I'll spin off another bug for that work, since it should come later and we might need to disable test alerts before we make the change.

To make an informed decision about what numbers generate significantly good or bad perf experiences, we probably want to first put our perfmetrics into telemetry probes so we can get a histogram of user experiences. Then, we can do a few rounds of testing on various -ref or dev hardware to see how modifying the node counts modifies the metrics values.

In the convo I had with the perf folks at the recent engineering summit, I was told we should target the 95th percentile of in-the-wild performance as our "poor performance" benchmark.

Comment 1

[Morgan Reschenberg [:morgan]]

Attachment: Bug 2083927: Add shared node count constants to a11y perftest head.js r?#accessibility-platform-reviewers!

Comment 2

[Pulsebot]

Pushed by [e-mail]:
https://github.com/mozilla-firefox/firefox/commit/bb82ad6f97b1
https://hg.mozilla.org/integration/autoland/rev/759e33e33aec
Add shared node count constants to a11y perftest head.js r=accessibility-platform-reviewers,eeejay

Comment 3

[Norisz Fay [:noriszfay]]

https://hg.mozilla.org/mozilla-central/rev/759e33e33aec