1617649 - consider adjusting wpt run-by-dir argument

Status: NEW

(Testing :: web-platform-tests, enhancement)

Description

[Joel Maher ( :jmaher ) (UTC -8)]

:jgraham points out that we could save some time by making a small adjustment to our parameters:
https://searchfox.org/mozilla-central/source/testing/mozharness/scripts/web_platform_tests.py#223

changing the value from 3 to 2 could save some runtime. We need to make sure there is not an increase in intermittent failures.

Comment 1

[James Graham [:jgraham]]

When applied on top of the changes in Bug 1617886 this seems to save another 30-35 minutes across all the chunks of a linux64-opt run. But it also looks like there may be some additional instability that needs to be investigated: https://treeherder.mozilla.org/#/jobs?repo=try&selectedJob=290376521&revision=978d97aa1c0920660a6536a2e0b31eed72aa27e5

Comment 2

[James Graham [:jgraham]]

Also, amusingly it ends up running all the WebDriver tests in a single chunk since they all have the path prefix webdriver/tests, so that probably needs special casing.

Comment 3

[James Graham [:jgraham]]

So it seems we end up crashing in the referrer-policy tests in a way that may be due to fd exhaustion, probably as a side effect of having more tests run the in the same process. I'm not sure what to do about that.