2084388 - Prototype/investigate a navigation test comprised of synthetic sites

Status: NEW

(Testing :: Performance, task, P3)

Description

[Kash Shampur [:kshampur] ⌚EST]

capturing a benchmark idea following a discussion on element. working name for now is "navigation-gauntlet" and would be separate from the current nav-bench.

two main things that would be involved for this new test

1. use a set of synthetic sites as the "workload" with the benchmark run against those. https://github.com/bgrins/zoo-sites and https://github.com/bgrins/the_zoo may be good starting points
2. distilled scenarios. each subtest is a focused, navigation scenario, and new ones get added over time as we come across them. Bug 2073063 is a good recent example

some things tbd:

• what metrics
• what are the distilled scenarios?
• how faithful synthetic soft navs can be vs a real sites
• does it catch the same regressions nav bench does (easy to check by comparing against a known one)
• upkeep of the sites/scenarios and that maintenance versus mitmproxy re-recording