1596942 - Improve wpt support in 'mach test-info report'

Status: RESOLVED WONTFIX

(Testing :: General, enhancement, P2)

Description

[Geoff Brown [:gbrown]]

Are disabled wpt tests reported as such?

Reports can be filtered by manifest annotations for manifestparser or reftest manifests, but what about wpt?

How are wpt tests reflected in a manifest report? Are they associated with components correctly? Are they/can they be broken down by directory?

Comment 1

[Geoff Brown [:gbrown]]

'mach test-info tests <wpt-test>' works correctly and reports the manifest as _tests/web-platform/meta/MANIFEST.json. ActiveData records are found in an abbreviated directory, like '/webvr/webvr-disabled-by-feature-policy.https.sub.html'.

Comment 2

[Geoff Brown [:gbrown]]

$ ./mach test-info report testing/web-platform/tests
Finding tests...
Resolver found 36252 tests, 1 manifests
{
"manifests": {},
"summary": {
"components": 63,
"failed tests": 0,
"manifests": 1,
"skipped tests": 0,
"tests": 36252
}
}
...so, no manifest summary for MANIFEST.json (but it would be boring anyway); and no skipped/failed info.

Comment 3

[Geoff Brown [:gbrown]]

$ ./mach test-info report testing/web-platform/tests/webvr --show-tests
Finding tests...
Resolver found 7 tests, 1 manifests
{
"tests": {
"Core::DOM: WebVR": [
{
"test": "testing/web-platform/tests/webvr/idlharness.https.html"
},
{
"test": "testing/web-platform/tests/webvr/webvr-disabled-by-feature-policy.https.sub.html"
},
{
"test": "testing/web-platform/tests/webvr/webvr-enabled-by-feature-policy-attribute-redirect-on-load.https.sub.html"
},
{
"test": "testing/web-platform/tests/webvr/webvr-enabled-by-feature-policy-attribute.https.sub.html"
},
{
"test": "testing/web-platform/tests/webvr/webvr-enabled-by-feature-policy.https.sub.html"
},
{
"test": "testing/web-platform/tests/webvr/webvr-enabled-on-self-origin-by-feature-policy.https.sub.html"
},
{
"test": "testing/web-platform/tests/webvr/webvr-supported-by-feature-policy.html"
}
]
}
}

Comment 4

[Geoff Brown [:gbrown]]

I'm reasonably happy with existing support and don't have specific ideas for how to improve this area.