1587828 - [meta] Refactor raptor-browsertime test INIs and taskcluster configs

Status: RESOLVED FIXED

(Testing :: Raptor, enhancement, P2)

Description

[Robert Wood [:rwood]]

The current structure of the raptor tests INIs and corresponding taskcluster configuration files works but is not really manageable with the large number of raptor test configurations, platforms, target hardware etc.

With the switch to using browsertime as the raptor engine (instead of the raptor web-ext) now is the ideal time to refactor the test INIs and tc configs. This refactor will be applied to raptor-browsertime tests; no need to convert all the exiting raptor test INIs and configs because they will eventually be ported over to raptor-browsertime tests anyway.

Ideally we will have a single raptor-browsertime page-load test INI that will have settings that apply to all browsertime page-load tests, with a list of all the URLs that will be measured. Then we will use taskcluster transforms to automatically create corresponding taskcluster raptor-browsertime test jobs (one for each URL to be measured).

This will require work on both the raptor framework side (still using the existing mozbase manifestparser to read the new test INI and keys) as well as taskcluster transforms side (to generate the raptor-browsertime jobs automatically).

The focus will be a solution for raptor-browsertime desktop jobs first, then android fenix (which runs in the github CI).

Comment 1

[Robert Wood [:rwood]]

Making this Meta a P2, individual dependent items will be P1 as they're worked on.