In Bug 1859549 we have introduced new tp6 and tp6m webextensions taskcluster jobs that runs these tp6 and tp6m perftests on Firefox Desktop and Firefox for Android instances with real webextensions installed.
As part of the previous Talos perftests framework we also have a
talos-realworld-webextensions (introduced from Bug 1575089), see talos docs here.
Talos perftests framework is now in maintenance mode and expected to be fully replaced by the new mozperftests and raptor perftests frameworks, and so once we have promoted the new tp6 / tp6m webextensions jobs to tier-2 (Bug 1867781) and confirmed that the new perftests are fully replacing the
talos-realworld-webextensions job, we will be removing the talos job in favor of the new ones.
The following are a few of the improvements that we expect the new tp6 / tp6m webextensions jobs to be providing compared to the previous
the new tp6m job runs on Firefox for Android, while the
talos-realworld-webextensions job runs only on Firefox Desktop
the new tp6/tp6m jobs use more recent dumps of top real websites to run the perftests on (compared to the one used by talos tests)
the new tp6/tp6m jobs replay the HTTP requests using the real websites domain, while talos does replay the HTTTP requests using fake talos domains, and so the new job are able to trigger more realistic adblockers behaviors (e.g. ublock is suspending and cancelling ads requests while running the tp6/tp6m amazon perf-test)
talos-realworld-webextensions job installs all the extensions included in a given talos-real-webextensions zip file, whereas the new jobs are using a condprof customization file which is falling back to download additonal xpi files from a given url (unless the expected xpi is already part of the firefox-addons.tar file fetched through a taskcluster fetch task dependency), which will allow to run these perftests on additional arbirary third party xpi files in a try push by just adding the new extension to the condprof customization file in try push itself