2042832 - Remove --total-chunks/--this-chunk and chunk_by_slice from mochitest harness

Status: RESOLVED FIXED

(Testing :: Mochitest, task)

Description

[Florian Quèze [:florian]]

Bug 2017153 removed most of the dead static chunking code from the mochitest harness (--chunk-by-runtime, --chunk-by-dir, and the manifest-runtimes-*.json files), but had to scale back to keep --total-chunks, --this-chunk, and the chunk_by_slice filter in runtests.py and mochitest_options.py.

The reason was mochitest-webgl2-ext: its 4-chunk job fell through to the runtime chunking path because the suite's manifest was a single large generated file (dom/canvas/test/webgl-conf/generated-mochitest.toml) that couldn't be split at taskgraph time.

Bug 2040629 has since enabled dynamic chunking for all webgl mochitest jobs (once the webgl-conf manifest got split into per-cluster files). With that, no remaining mochitest task hits the --total-chunks/--this-chunk path:

• mochitest-browser-chrome-failures, mochitest-devtools-chrome-failures, and mochitest-valgrind all still have static chunks in kind.yml, but none set test-manifest-loader: null. Their manifests are pre-resolved by the taskgraph and passed via MOZHARNESS_TEST_PATHS, so desktop_unittest.py never falls through to the --total-chunks/--this-chunk branch.
• mochitest-valgrind also has run-on-projects: [], so it never runs by default anyway.

This bug re-does the part of bug 2017153 that was scaled back:

In testing/mochitest/mochitest_options.py:

• Remove the --total-chunks and --this-chunk option blocks
• Remove the totalChunks/thisChunk validation in validate()

In testing/mochitest/runtests.py:

• Remove the chunk_by_slice import from manifestparser.filters
• Remove the if options.totalChunks: filters.append(chunk_by_slice(...)) block

Comment 1

[Florian Quèze [:florian]]

Attachment: Bug 2042832 - Remove --total-chunks/--this-chunk and chunk_by_slice from mochitest harness, r=ahal.

Bug 2017153 removed most of the dead static chunking code from the mochitest
harness, but had to scale back to keep --total-chunks, --this-chunk, and
the chunk_by_slice filter because mochitest-webgl2-ext fell through to the
runtime chunking path (its tests all lived in a single large generated
manifest that couldn't be split at taskgraph time).

Bug 2040629 has since enabled dynamic chunking for all webgl mochitest jobs,
once the webgl-conf manifest was split into per-cluster files. No remaining
mochitest task hits the --total-chunks/--this-chunk path: the only suites
still on static chunks in kind.yml (mochitest-browser-chrome-failures,
mochitest-devtools-chrome-failures, mochitest-valgrind) do not set
test-manifest-loader: null, so taskgraph pre-resolves their manifests and
passes them via MOZHARNESS_TEST_PATHS, and desktop_unittest.py never
appends --total-chunks/--this-chunk to the harness command.

Comment 2

[Florian Quèze [:florian]]

Attachment: Bug 2042832 - Remove --total-chunks/--this-chunk and chunk_by_slice from xpcshell harness, r=ahal.

Same reasoning as the mochitest cleanup in the previous commit. Neither
xpcshell task in taskcluster/kinds/test/xpcshell.yml sets
test-manifest-loader: null, so taskgraph always pre-resolves their
manifests and passes them via MOZHARNESS_TEST_PATHS. desktop_unittest.py
never appends --total-chunks/--this-chunk to the xpcshell command,
self.totalChunks stays at its default of 1, and the chunk_by_slice call
in runxpcshelltests.py never fires.

Comment 3

[Florian Quèze [:florian]]

Attachment: Bug 2042832 - Remove unused chunk_by_slice filter from manifestparser, r=ahal.

With xpcshell and mochitest no longer importing chunk_by_slice, the
filter has no consumers anywhere in-tree.

Comment 4

[Pulsebot]

Pushed by [e-mail]:
https://github.com/mozilla-firefox/firefox/commit/a8a519b1350c
https://hg.mozilla.org/integration/autoland/rev/1101270c997f
Remove --total-chunks/--this-chunk and chunk_by_slice from mochitest harness, r=ahal.
https://github.com/mozilla-firefox/firefox/commit/4f3fb8bf93f3
https://hg.mozilla.org/integration/autoland/rev/9f5772d970da
Remove --total-chunks/--this-chunk and chunk_by_slice from xpcshell harness, r=ahal.
https://github.com/mozilla-firefox/firefox/commit/737b97bdb839
https://hg.mozilla.org/integration/autoland/rev/c1a08ce43124
Remove unused chunk_by_slice filter from manifestparser, r=ahal.

Comment 5

[Serban Stanca [:SerbanS]]

https://hg.mozilla.org/mozilla-central/rev/1101270c997f
https://hg.mozilla.org/mozilla-central/rev/9f5772d970da
https://hg.mozilla.org/mozilla-central/rev/c1a08ce43124