Closed Bug 1814448 Opened 1 year ago Closed 1 year ago

Reduce the frequency of android browsertime tests on the toolchains branch

Categories

(Testing :: Raptor, task, P1)

task

Tracking

(firefox111 fixed)

RESOLVED FIXED
111 Branch
Tracking Status
firefox111 --- fixed

People

(Reporter: sparky, Assigned: sparky)

Details

Attachments

(1 file)

I found that we are running Android performance tests on every push in the toolchains branch: https://treeherder.mozilla.org/jobs?repo=toolchains&searchStr=android%2Cbrowsertime

We should either disable them completely, or reduce them to a small handful of tests. If we decide to not disable them completely, then we should reduce their frequency to once per week given our limited capacity for these.

:glandium, what do you think about disabling these performance tests on the toolchains branch?

Flags: needinfo?(mh+mozilla)

The toolchains branch receives pushes at most once a day, but more often twice a week or less. Is that really a problem?

Flags: needinfo?(mh+mozilla)

Unfortunately yes, we only have 27 devices available for testing on all of our branches. I'm looking into increasing our capacity but it'll be some time before we get that (assuming we do). Could we instead reduce the tests we run on each commit there to only the browsertime essential tests (6-7 tasks)? That way you'll still have android coverage on this branch.

Flags: needinfo?(mh+mozilla)

If you can make it happen, sure.

Flags: needinfo?(mh+mozilla)

Ok, thanks :glandium! Patch incoming.

This patch disables most of the android perftests on the toolchain branch. After this patch, only 1 pageload test, and 1 benchmark test will run on android on that branch.

Assignee: nobody → gmierz2
Status: NEW → ASSIGNED
Pushed by gmierz2@outlook.com:
https://hg.mozilla.org/integration/autoland/rev/430910bbea14
Run a subset of Android perftests on the toolchain branch. r=perftest-reviewers,AlexandruIonescu
Status: ASSIGNED → RESOLVED
Closed: 1 year ago
Resolution: --- → FIXED
Target Milestone: --- → 111 Branch
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Created:
Updated:
Size: