support list of test manifests which run in 'backlog' tasks of web-platform-tests
Categories
(Testing :: web-platform-tests, task)
Tracking
(Not tracked)
People
(Reporter: aryx, Unassigned)
Details
Web platform tests run as part of a 'normal' task by default. If they are expected to fail at the test level (not at test step), they run less often but as part of the 'backlog' tasks on mozilla-central. The latter does not provide a lists of test manifest covered at least partially which prevents them from matching when a user filters the tasks in Treeherder by test path.
|
||
Comment 1•2 years ago
|
||
I don't think this is a harness issue, but a test infra issue? Or at least if it's a harness issue I don't know what's missing for the backlog case.
which specific artifact or data in an artifact is needed.
I see _errorsummary.json artifacts on success and failed wpt backlog tasks:
https://treeherder.mozilla.org/jobs?repo=mozilla-central&searchStr=backlog&selectedTaskRun=NxpdywyzQmS6zq3YYdG1KA.0
|
Reporter | |
Comment 3•2 years ago
|
||
Treeherder requests e.g. https://firefox-ci-tc.services.mozilla.com/api/queue/v1/task/JpQAaguOR4WKbpVXOAofGg which lacks MOZHARNESS_TEST_PATHS in the payload.
we need to set the test-manifest-loader:
https://searchfox.org/mozilla-central/source/taskcluster/ci/test/web-platform.yml#124
that is set to NULL, so taskcluster will not parse manifests and therefore will not set test-manifests in the task attributes.
it appears it isn't as easy as changing that, a bit of digging and this might turn into a simple solution.
Description
•