Closed Bug 2015144 Opened 9 months ago Closed 5 days ago

Get pageload tests running on emulation and available for Try selection

Categories

(Testing :: Performance, task, P2)

task

Tracking

(firefox162 fixed)

RESOLVED FIXED
162 Branch
Tracking Status
firefox162 --- fixed

People

(Reporter: aglavic, Assigned: aglavic)

References

(Blocks 1 open bug)

Details

(Whiteboard: [fxp])

Attachments

(1 external file, 1 obsolete file)

Context:

  • As part of our emulator report in 1934319 we found that expanding tests to emulated devices is a worthwhile endeavour from a cost-benefit perspective.

What we are doing here:

  • Expanding our regularly run tests to include running tests on the pageload tests regularly.

Why we are doing it:

  • To see how well over a long period of time emulators catch regressions in CI.
Summary: Get applink tests running on emulation regularly on autoland → Get pageload tests running on emulation regularly on autoland
Severity: -- → S3
Priority: -- → P2

The host-utils*[!z|checksum] glob was meant to skip the tarball and the
checksum file, but its character class excludes any name ending in one of
those letters, including the host-utils-<version>.en-US.mac directory. On the
macOS hosts running the Android emulator this left no match, and mitmproxy
setup failed with an IndexError. Select the directory explicitly instead.

Add browsertime-tp6m-essential to the android-em-14-arm64 test set and run
the Fenix tasks on autoland at tier 3, without a conditioned profile like the
A55s.

GeckoView and the reference browser haven't been verified on the emulators,
so limit them to android-hw platforms.

Assignee: nobody → aglavic
Status: NEW → ASSIGNED
Attachment #9672582 - Attachment is obsolete: true
Attachment #9672583 - Attachment description: WIP: Bug 2015144 - Run the tp6m-essential pageload tests on Android emulators on autoland. r?#perftest-reviewers → Bug 2015144 - Run the tp6m-essential pageload tests on Android emulators on autoland. r?#perftest-reviewers
Attachment #9672583 - Attachment description: Bug 2015144 - Run the tp6m-essential pageload tests on Android emulators on autoland. r?#perftest-reviewers → Bug 2015144 - Add the tp6m-essential pageload tests on Android emulators for try. r?#perftest-reviewers
Pushed by aglavic@mozilla.com: https://github.com/mozilla-firefox/firefox/commit/e39301925286 https://hg.mozilla.org/integration/autoland/rev/2e0e2316f091 Add the tp6m-essential pageload tests on Android emulators for try. r=perftest-reviewers,kshampur
Summary: Get pageload tests running on emulation regularly on autoland → Get pageload tests running on emulation and available for Try selection
Status: ASSIGNED → RESOLVED
Closed: 5 days ago
Resolution: --- → FIXED
Target Milestone: --- → 162 Branch
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Created:
Updated:
Size: