2017153 - Remove dead mochitest chunking code (--chunk-by-runtime, --chunk-by-dir, --total-chunks, --this-chunk)

Status: RESOLVED FIXED

(Testing :: Mochitest, enhancement)

Description

[Florian Quèze [:florian]]

Mochitest's runtests.py has code to split tests into chunks at runtime using --total-chunks/--this-chunk with either --chunk-by-runtime (loading manifest-runtimes-*.json files) or --chunk-by-dir.

This code is dead in CI: the taskgraph decision task now pre-assigns specific manifests to each chunk via chunk_manifests() in taskcluster/gecko_taskgraph/util/chunking.py, and communicates them to test tasks through MOZHARNESS_TEST_PATHS. When that env var is set, desktop_unittest.py passes the manifest paths directly to mochitest and skips --total-chunks/--this-chunk entirely. Since mochitest uses the default manifest loader (no test-manifest-loader: null), test-manifests are always resolved at taskgraph time, so the runtime chunking path is never taken.

This patch removes:

• The --total-chunks, --this-chunk, --chunk-by-runtime, --chunk-by-dir options from mochitest_options.py and their validation logic
• The chunking filter block in runtests.py (chunk_by_dir, chunk_by_runtime, chunk_by_slice) and the corresponding manifestparser imports
• The chunkByRuntime/chunkByDir assignments in mach_test_package_commands.py
• The suite_name pass-through in mach_commands.py (only needed for chunk-by-runtime)
• All --chunk-by-runtime and --chunk-by-dir=4 flags from mozharness configs (linux_unittest.py, mac_unittest.py, win_unittest.py, android_common.py)

Comment 1

[Florian Quèze [:florian]]

Attachment: Bug 2017153 - Remove dead mochitest chunking code. r=ahal

Chunking is now done at taskgraph time via chunk_manifests() in
chunking.py, with manifests passed to test tasks via
MOZHARNESS_TEST_PATHS. The --total-chunks/--this-chunk args are
no longer passed to mochitest, making the runtime chunking code
(--chunk-by-runtime, --chunk-by-dir) dead.

Comment 2

[Florian Quèze [:florian]]

Attachment: Bug 2017153 - Remove manifest-runtimes-*.json files, writeruntimes script, and stale references. r=ahal

These files provided runtime data for the now-removed mochitest chunking
code. The taskgraph fetches runtime data directly from TaskCluster
artifacts instead. The writeruntimes script queried the long-defunct
ActiveData service to generate these files.

Also removes a stale pyproject.toml exclusion and a sparse-profile entry
that is no longer needed since the taskgraph doesn't read from
testing/runtimes/.

Comment 3

[Pulsebot]

Pushed by [e-mail]:
https://github.com/mozilla-firefox/firefox/commit/98fdfabf3272
https://hg.mozilla.org/integration/autoland/rev/87d79951be29
Remove dead mochitest chunking code. r=ahal

Comment 4

[Pulsebot]

Pushed by [e-mail]:
https://github.com/mozilla-firefox/firefox/commit/bb7a4d5ebc72
https://hg.mozilla.org/integration/autoland/rev/19e0d495a041
Revert "Bug 2017153 - Remove dead mochitest chunking code. r=ahal" for causing mochitests failures.

Comment 5

[Serban Stanca [:SerbanS]]

Reverted this because it was causing mochitests failures.

• Revert link
• Push with failures
• Failure Log
• Failure line: No suite end message was emitted by this harness.

Comment 6

[Pulsebot]

Pushed by [e-mail]:
https://github.com/mozilla-firefox/firefox/commit/a68cd2b6fe63
https://hg.mozilla.org/integration/autoland/rev/00154d9b088f
Remove dead mochitest chunking code. r=ahal

Comment 7

[Pulsebot]

Pushed by [e-mail]:
https://github.com/mozilla-firefox/firefox/commit/bd3b6a9b9ce5
https://hg.mozilla.org/integration/autoland/rev/ece0c4f89ac1
Remove manifest-runtimes-*.json files, writeruntimes script, and stale references. r=ahal

Comment 8

[Cosmin Sabou [:CosminS]]

https://hg.mozilla.org/mozilla-central/rev/00154d9b088f

Comment 9

[amarc]

https://hg.mozilla.org/mozilla-central/rev/ece0c4f89ac1