1614643 - web-platform-tests have missing manifest information in errorsummary.log

Status: RESOLVED FIXED

(Firefox Build System :: Task Configuration, task)

Comment 1

[Andrew Halberstadt [:ahal]]

Attachment: Bug 1614643 - [web-platform-tests] Strip leading path separator when computing hash_path, r?jgraham!

This fixes a bug which miscomputes the depth. Say depth=1 and test_path="/Foo/bar.html"
for example.

We then run this line of code:
os.path.sep.join(os.path.dirname(test_path).split(os.path.sep, depth)[:depth])

Breaking it doen a bit:

$ dirname = os.path.dirname("/Foo/bar.html").split(os.path.sep, 1)
$ dirname
["", "Foo"]
$ os.path.sep.join(dirname[:1])
""

According to the specs, --run-by-dir=1 should split tests by the top-level
directory, but it will assign an empty string to everything. Similarly larger
depths will be off by one directory level.

This fixes the issue by stipping the leading separator (and adding it back again
at the end).

Comment 2

[Andrew Halberstadt [:ahal]]

Attachment: Bug 1614643 - [web-platform-tests] Add test groups to logger.suite_start, r?jgraham!

This adds "group" information to the 'suite_start' log message. Since WPT don't have
manifests in the same sense that we understand them for other harnesses, we use
directories as stand-ins.

The same algorithm that the 'DirectoryHashChunker' uses to find a directory is used
to determine which level of depth consitutes a "group". This depth is set via
--run-by-dir=<int> in both cases, but will default to leaf directories if not
specified.

Depends on D67625

Comment 3

[Andrew Halberstadt [:ahal]]

Attachment: Bug 1614643 - [wpt] Log test 'group' information in suite_start, r?jgraham

Comment 4

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/370e52ff13b2
[wpt] Log test 'group' information in suite_start, r=jgraham

Comment 5

[Web Platform Test Sync Bot [:wpt-sync] (Matrix: #interop:mozilla.org)]

Created web-platform-tests PR https://github.com/web-platform-tests/wpt/pull/22894 for changes under testing/web-platform/tests

Comment 6

[Web Platform Test Sync Bot [:wpt-sync] (Matrix: #interop:mozilla.org)]

Can't merge web-platform-tests PR due to failing upstream checks:
Github PR https://github.com/web-platform-tests/wpt/pull/22894
* Community-TC (pull_request) (https://community-tc.services.mozilla.com/tasks/groups/bpByq2b5Rz-A5-xDJWLkzg)

Comment 7

[Web Platform Test Sync Bot [:wpt-sync] (Matrix: #interop:mozilla.org)]

Upstream PR was closed without merging

Comment 8

[Cristian Brindusan [:cbrindusan]]

Backed out changeset 370e52ff13b2 (Bug 1614643) for wpt failures at open-features-tokenization-top-left.html.
https://hg.mozilla.org/integration/autoland/rev/d82faacc40bc4aa370afbc65278bf71d8b6732fa

Push with failures:
https://treeherder.mozilla.org/#/jobs?repo=autoland&resultStatus=pending%2Crunning%2Csuccess%2Ctestfailed%2Cbusted%2Cexception&revision=370e52ff13b2f6368d8cb7a3f7a1190cc18bc94d&selectedJob=297383519

Failure log:
https://treeherder.mozilla.org/logviewer.html#/jobs?job_id=297383519&repo=autoland&lineNumber=3083

Comment 9

[Cristian Brindusan [:cbrindusan]]

Please disregard the above message, your patch was backed-out due to an error I've made. I've relanded this one.

Comment 10

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/7d6948512ef0
[wpt] Log test 'group' information in suite_start, r=jgraham CLOSED TREE

Comment 11

[Web Platform Test Sync Bot [:wpt-sync] (Matrix: #interop:mozilla.org)]

Can't merge web-platform-tests PR due to failing upstream checks:
Github PR https://github.com/web-platform-tests/wpt/pull/22894
* Community-TC (pull_request) (https://community-tc.services.mozilla.com/tasks/groups/LCp6VWIPTNuVEKF9Sn0F2Q)

Comment 12

[Oana Pop-Rus]

https://hg.mozilla.org/mozilla-central/rev/7d6948512ef0

Comment 13

[Web Platform Test Sync Bot [:wpt-sync] (Matrix: #interop:mozilla.org)]

Upstream PR merged by moz-wptsync-bot