Add macosx1500-aarch64-vms test platform for headless VM workers
Categories
(Firefox Build System :: Task Configuration, task, P3)
Tracking
(firefox153 fixed)
| Tracking | Status | |
|---|---|---|
| firefox153 | --- | fixed |
People
(Reporter: rcurran, Assigned: rcurran)
References
(Blocks 1 open bug)
Details
Attachments
(13 files)
|
48 bytes,
text/x-phabricator-request
|
Details | Review | |
|
48 bytes,
text/x-phabricator-request
|
Details | Review | |
|
48 bytes,
text/x-phabricator-request
|
Details | Review | |
|
48 bytes,
text/x-phabricator-request
|
Details | Review | |
|
48 bytes,
text/x-phabricator-request
|
Details | Review | |
|
48 bytes,
text/x-phabricator-request
|
Details | Review | |
|
48 bytes,
text/x-phabricator-request
|
Details | Review | |
|
48 bytes,
text/x-phabricator-request
|
Details | Review | |
|
48 bytes,
text/x-phabricator-request
|
Details | Review | |
|
48 bytes,
text/x-phabricator-request
|
Details | Review | |
|
48 bytes,
text/x-phabricator-request
|
Details | Review | |
|
48 bytes,
text/x-phabricator-request
|
Details | Review | |
|
48 bytes,
text/x-phabricator-request
|
Details | Review |
|
Assignee | |
Comment 1•5 months ago
|
||
Wires the gecko-t-osx-1500-m-vms worker pool (Apple Virtualization
Framework VMs on M4 Mac minis) into the gecko taskgraph as a
first-class test platform, so try pushes don't need
--worker-override t-osx-1500-m4=releng-hardware/gecko-t-osx-1500-m-vms.
Changes:
-
taskcluster/gecko_taskgraph/transforms/test/worker.py
Add macosx1500-aarch64-vms -> t-osx-1500-m-vms in MACOSX_WORKER_TYPES.
Dispatcher for macosx1500-aarch64-vms is placed BEFORE the
macosx1500-aarch64 startswith match (the latter would otherwise
shadow it). -
taskcluster/test_configs/test-platforms.yml
Add macosx1500-aarch64-vms/{opt,debug}, building from the existing
macosx64-aarch64 builds and using the new vms-tests test set. -
taskcluster/test_configs/test-sets.yml
Add macosx1500-aarch64-vms-tests covering the suites that don't
require a real GPU / display / perf hardware: cppunittest,
mochitest-{a11y,browser-a11y,browser-media,browser-translations,
chrome,devtools-chrome,media,plain,remote}, web-platform-tests.
Excludes WebGL, GPU, reftest, talos, raptor, awsy, browsertime. -
taskcluster/test_configs/variants.yml
media-gpu variant now excludes "macosx ... vms" platforms so the
mda-gpu variant of mochitest-media doesn't get scheduled on the
no-GPU VM workers.
|
|
Assignee | |
Comment 2•5 months ago
|
||
Wires the gecko-t-osx-1500-m-vms worker pool (Apple Virtualization
Framework VMs on M4 Mac minis) into the gecko taskgraph as a
first-class test platform. The new macosx1500-aarch64-vms platform
(opt + debug + shippable + devedition variants) resolves through the
worker.py dispatcher to the t-osx-1500-m-vms alias and lands on
releng-hardware/gecko-t-osx-1500-m-vms natively, so try pushes no
longer need --worker-override.
Tasks on the new platform are scheduled on the standard set of
projects (autoland, trunk, try) — inheriting the same run-on-projects
defaults as the existing macosx1500-aarch64 platform. The test set
(macosx1500-aarch64-vms-tests) is intentionally narrow — cppunittest
and jittest only — to keep the additive CI load minimal while the
VM platform's broader test compatibility is improved (web-platform-tests
is pending Bug 2038639; display-dependent suites are excluded).
|
||
Comment 4•5 months ago
|
||
| bugherder | ||
|
|
Assignee | |
Comment 5•5 months ago
|
||
xpcshell is chrome-process and does not require a display or WindowServer,
so it runs cleanly on the headless VirtualMac2,1 VM workers. Already
running on bare-metal macosx1500-aarch64 via Bug 2032601 (D294786).
|
||
Updated•5 months ago
|
|
||
Comment 7•5 months ago
|
||
A patch has been attached on this bug, which was already closed. Filing a separate bug will ensure better tracking. If this was not by mistake and further action is needed, please alert the appropriate party. (Or: if the patch doesn't change behavior -- e.g. landing a test case, or fixing a typo -- then feel free to disregard this message)
|
|
Assignee | |
Comment 8•5 months ago
|
||
telemetry-tests-client is a chrome-process suite of telemetry-ping tests;
pure background-task workload with no UI rendering, so it runs cleanly on
the headless VirtualMac2,1 VM workers (validated via try push on the
gecko-t-osx-1500-m-vms pool).
Note: marionette-unittest was initially included in this expansion but
removed after a validation try push showed two display-dependent failures
(test_accessibility::test_click_raises_element_not_accessible and
test_switch_window::test_switch_to_unloaded_tab). Marionette-unittest can
be added in a follow-up with appropriate manifest-level skip-ifs.
|
||
Comment 10•5 months ago
|
||
| bugherder | ||
|
|
Assignee | |
Comment 11•5 months ago
|
||
jsreftest on macosx1500-aarch64-vms/debug consistently hits a null-pointer
shutdown crash in CompositorBridgeParent::DisconnectApzcTreeManager after
all tests in the chunk have passed ("57895/0/42 CRASH"). This is the
same APZ-on-headless-macOS family of bugs that's already blocking
web-platform-tests (Bug 2038639). opt builds finish before the crash
window and pass cleanly.
Excluding debug only via run-on-projects keeps the opt jsreftest coverage
on the VM platform (currently 100% pass rate). The exclusion can be
removed once the underlying APZ shutdown crash is fixed.
|
||
Comment 12•5 months ago
|
||
| bugherder | ||
|
|
Assignee | |
Comment 13•4 months ago
|
||
Sets a new mozinfo flag macos_vm = True when running on an Apple
Virtualization Framework VM (hw.model starting with VirtualMac), False
otherwise. Detection runs at mozinfo import time in
testing/mozbase/mozinfo/mozinfo/mozinfo.py via a single sysctl call,
alongside the existing apple_silicon and apple_catalina derivations.
This makes the flag visible to every test harness that imports mozinfo
(mochitest, xpcshell, marionette, reftest, jsreftest, cppunittest, etc.)
without requiring per-harness changes.
Motivation: the macosx1500-aarch64-vms test platform shares the
aarch64 mozinfo signature with bare-metal macosx1500-aarch64, so
skip-if conditions like 'os == "mac" && arch == "aarch64"' leak
into bare-metal coverage. This flag gives manifests a precise way to
exclude tests on the no-display VM workers without affecting
bare-metal aarch64 coverage. Three specific tests already known to
fail VM-only (test_accessibility.test_click_raises_element_not_accessible,
test_switch_window.test_switch_to_unloaded_tab,
browser_WindowManager.test_fullscreenWindow) will adopt this flag in
follow-up patches once it's validated in tree.
|
||
Comment 14•4 months ago
|
||
|
|
Assignee | |
Comment 15•4 months ago
|
||
Adds marionette-unittest to the macosx1500-aarch64-vms-tests test-set so the
opt build of the suite runs on the Apple Virtualization Framework VM workers.
Two tests fail in this environment because they depend on a real display, so
they're skipped via an in-file _MACOS_VM check (sysctl hw.model starts with
"VirtualMac"):
- test_accessibility.test_click_raises_element_not_accessible: requires
the macOS accessibility API which depends on the display server. - test_switch_window.test_switch_to_unloaded_tab: the in-app restart
times out on the headless VM workers.
Detection is done inline rather than via mozinfo because marionette's test
harness runs in a venv with a pre-installed mozinfo package that doesn't
include the macos_vm flag landed in D302479. Both tests pass on bare-metal
macosx1500-aarch64.
The debug build is excluded from the VM platform in taskcluster/kinds/test/
marionette.yml. Every test triggers a process shutdown assertion in
DisconnectApzcTreeManager (Bug 2039987 family — same APZ-on-headless issue
that's already excluding jsreftest-debug and mochitest-remote-debug).
|
|
Assignee | |
Comment 16•4 months ago
|
||
Adds mochitest-remote to the macosx1500-aarch64-vms-tests test-set so the
opt build of the suite runs on the Apple Virtualization Framework VM
workers.
One task in browser_WindowManager.js (test_fullscreenWindow) is skipped on
VMs via a per-task add_task({ skip_if }) check — fullscreen window operations
don't behave like on bare-metal hardware without a display. VM detection in
chrome JS uses Services.sysinfo.getProperty("appleModelId"), which returns
the sysctl hw.model value ("VirtualMac2,1" on AVF VMs). The other 16 tasks
in the file keep running on the VM platform.
The debug build is excluded from the VM platform in taskcluster/kinds/
mochitest/kind.yml: a GPU process leak-detection failure (Bug 2039987 family)
consistently fires after the test chunk finishes, so debug can't currently
run on the no-display VM workers regardless of which tests are selected.
Depends on D302617
|
||
Comment 17•4 months ago
|
||
| bugherder | ||
|
|
||
Comment 18•4 months ago
|
||
|
|
Assignee | |
Comment 19•4 months ago
|
||
Adds crashtest to the macosx1500-aarch64-vms-tests test-set so the opt build
runs on the Apple Virtualization Framework VM workers. Crashtest is a thin
manifest that loads a battery of content files and verifies Firefox doesn't
crash.
To support per-test gating on the new platform, the reftest manifest sandbox
gains a macosVm flag (mozinfo.macos_vm from D302479, exposed alongside the
existing appleSilicon flag). Seven crashtests are skipped on macosVm because
their media or focus dependencies hang on the no-display VM workers:
- dom/media/test/crashtests/video-replay-after-audio-end.html
- dom/media/tests/crashtests/1920264.html (already skipped on Android for
the same "no screenshare" reason) - dom/media/webrtc/tests/crashtests/1991492_2.html (ditto)
- dom/media/webrtc/tests/crashtests/1991494.html (ditto)
- editor/libeditor/crashtests/1429523.html
- editor/libeditor/crashtests/1429523.xhtml
- editor/libeditor/crashtests/1444630.html (already skipped on Android)
The debug build is preemptively excluded in taskcluster/kinds/reftest/kind.yml.
The compositor shutdown assertion in DisconnectApzcTreeManager (Bug 2039987
family) has hit every other debug suite added to this platform so far
(jsreftest, marionette-unittest, mochitest-remote). Debug can be re-enabled
once the underlying APZ assertion is fixed.
|
||
Comment 20•4 months ago
|
||
| bugherder | ||
|
|
||
Comment 21•4 months ago
|
||
|
||
Comment 22•4 months ago
|
||
| bugherder | ||
|
|
Assignee | |
Comment 23•4 months ago
|
||
Adds marionette-integration to the macosx1500-aarch64-vms-tests test-set so
the opt build runs on the Apple Virtualization Framework VM workers. This is
the sibling of marionette-unittest (already enabled via D302617); it runs the
integration-tests.toml manifest from the same harness.
The debug build is preemptively excluded in taskcluster/kinds/test/marionette.yml
for both the default and webrender-sw variants. The compositor shutdown
assertion in DisconnectApzcTreeManager (Bug 2039987 family) has hit every
other debug suite added to this platform so far (jsreftest, marionette-unittest,
mochitest-remote, crashtest). Debug can be re-enabled once the underlying APZ
assertion is fixed.
|
|
||
Updated•4 months ago
|
|
||
Comment 24•4 months ago
|
||
Not all the patches landed yet. Lets reopen the bug until the last work item is done.
|
|
||
Comment 25•4 months ago
|
||
|
|
Assignee | |
Comment 26•4 months ago
|
||
Adds mochitest-plain to the macosx1500-aarch64-vms-tests test-set so the opt
build runs on the Apple Virtualization Framework VM workers. mochitest-plain
is the largest desktop test suite by chunk count; this initial enablement
ships only the test-set + kind.yml changes. Per-test skip-ifs for display,
audio, and screen-capture-dependent tests will follow once the first VM run
exposes the actual failure set.
The debug build is preemptively excluded in taskcluster/kinds/mochitest/kind.yml.
The compositor shutdown assertion in DisconnectApzcTreeManager (Bug 2039987
family) has hit every other debug suite added to this platform so far
(jsreftest, marionette-unittest, mochitest-remote, crashtest, marionette-
integration). Debug can be re-enabled once the underlying APZ assertion is
fixed.
|
|
||
Comment 27•4 months ago
|
||
| bugherder | ||
|
|
||
Comment 28•4 months ago
|
||
|
||
Comment 29•4 months ago
|
||
| bugherder | ||
|
|
Assignee | |
Comment 30•4 months ago
|
||
Adds mochitest-chrome to the macosx1500-aarch64-vms-tests test-set so the opt
build runs on the Apple Virtualization Framework VM workers. This is the next
mochitest flavor after mochitest-plain and mochitest-remote, both already
enabled on this platform. This initial enablement ships only the test-set +
kind.yml changes; per-test skip-ifs for any display/focus-dependent chrome
tests will follow once the first VM run exposes the failure set.
The debug build is preemptively excluded in taskcluster/kinds/mochitest/kind.yml
(1proc variant). The compositor shutdown assertion in DisconnectApzcTreeManager
(Bug 2039987 family) has hit every other debug suite added to this platform so
far. Debug can be re-enabled once the underlying APZ assertion is fixed.
|
|
||
Comment 31•4 months ago
|
||
|
|
||
Comment 32•4 months ago
|
||
| bugherder | ||
|
||
Comment 33•4 months ago
|
||
jsreftest on macosx1500-aarch64-vms/debug consistently hits a null-pointer
shutdown crash in CompositorBridgeParent::DisconnectApzcTreeManager after
all tests in the chunk have passed ("57895/0/42 CRASH"). This is the
same APZ-on-headless-macOS family of bugs that's already blocking
web-platform-tests (Bug 2038639). opt builds finish before the crash
window and pass cleanly.
Excluding debug only via run-on-projects keeps the opt jsreftest coverage
on the VM platform (currently 100% pass rate). The exclusion can be
removed once the underlying APZ shutdown crash is fixed.
Original Revision: https://phabricator.services.mozilla.com/D301508
|
|
||
Updated•4 months ago
|
|
|
||
Comment 34•4 months ago
|
||
firefox-beta Uplift Approval Request
- User impact if declined/Reason for urgency: none
- Code covered by automated testing?: yes
- Fix verified in Nightly?: yes
- Needs manual QE testing?: no
- Steps to reproduce for manual QE testing:
- Risk associated with taking this patch: low
- Explanation of risk level: only changes machine type on which jsreftests are executed for macOS
- String changes made/needed?: none
- Is Android affected?: no
|
|
||
Updated•4 months ago
|
|
|
||
Comment 35•4 months ago
|
||
| uplift | ||
|
||
Updated•4 months ago
|
|
||
Comment 36•3 months ago
|
||
All the patches landed.
|
||
Updated•3 months ago
|
Description
•