1627433 - Antitracking tests take way too long to run

Status: NEW

(Core :: Privacy: Anti-Tracking, defect, P3)

Description

[:Gijs (he/him)]

On a Windows shippable opt configuration on infra, they take nearly 15 minutes:

[task 2020-04-04T01:30:09.079Z] 01:30:09     INFO - Running manifest: toolkit\components\antitracking\test\browser\browser.ini
...
[task 2020-04-04T01:44:54.837Z] 01:44:54     INFO -  *** End BrowserChrome Test Results ***

(from https://firefoxci.taskcluster-artifacts.net/EU3d3itjRlOpHbMTYFDi-Q/0/public/logs/live_backing.log )

When, like today, I got pinged for a failure in one of the tests, just trying to reproduce it once (if running the individual test didn't repro it, as was the case here) takes all that time, during which I can't use the machine.

As a result, this isn't really a usable set of tests. As a first step, perhaps we can split them into directories, but a better way of looking at this might be to investigate where we're spending all this time and if we can do more testing either as xpcshell tests or without opening/closing quite as many tabs/windows (ie can we do more of the tests with a single helper page, instead of opening one tab/window for every configuration).

Of course, if we need extra motivation to work on this: this also just plain costs us a bunch of money.

Comment 1

[Firefox Bug Husbandry Bot]

Because this bug's Severity has not been changed from the default since it was filed, and it's Priority is P3 (Backlog,) indicating it has been triaged, the bug's Severity is being updated to S3 (normal.)