1867781 - Promote to tier-2 the browsertime-tp6-webextensions and browsertime-tp6m-webextensions taskcluster perftests jobs

Status: NEW

(WebExtensions :: General, task, P2)

Description

[Luca Greco [:rpl] [:luca] [:lgreco]]

As part of Bug 1859549 we are introducing two new taskcluster jobs to run the raptor browsertime tp6 and tp6m perftests with real WebExtensions installed, these new jobs aim to:

• provide insights (and alerts) about the performance impact of the WebExtensions framework while being exercised by real extensions under more realistic scenarios (as an examle, unlike the existing talos real worlds extensions perfests, the tp6 and tp6m jobs are replaying real websites traffic using the real domain names instead of fake ones like in the talos tests, which do match real adblockers filter lists and trigger adblockers blocking behaviors that the talos tests cannot hit)
• provide a more direct comparison between metrics collected by the same raptor browsertime scenario executed with and without real webextensions
• collect and alert metrics from Firefox for Android with real third party extensions installed (instead of just built-ins like webcompat), which the existing talos real worlds extensions perfests cannot (because it does not actually support Firefox for Android builds)

We are currently planning to land the new taskcluster jobs initially as tier 3 jobs as part of Bug 1859549, and this followup is tracking promoting those jobs to tier-2 after having monitored how they behave after being landed as tier-3.

Until the jobs are promoted to tier-2 the jobs will not be code-sheriffed nor generate alerts.

Comment 1

[Greg Mierzwinski [:sparky]]

Something to add here is that the fenix tests will also need to wait for the monorepo changes to complete for alerting on autoland. That said, before those changes are complete, we can still enable alerts by running the tests in the firefox-android repo.

The monorepo changes don't impact sheriffing for these tests since we can get that on mozilla-central when the tests are promoted to tier-2.