Closed Bug 1548659 Opened 6 months ago Closed 5 months ago

Move Fennec tests on mozilla-central to Tier 2 and reduced frequency after the Gecko 69 version bump

Categories

(Firefox for Android :: Testing, task, P2)

task

Tracking

()

RESOLVED FIXED
Firefox 69
Tracking Status
firefox69 --- fixed

People

(Reporter: RyanVM, Assigned: egao)

References

(Blocks 1 open bug)

Details

Attachments

(1 file)

Once the Gecko version number is bumped to 69 on mozilla-central, we can drop all Fennec test suites (NOT GeckoView) to Tier 2 and lower run frequency. Per the ESR migration plan, we need to ensure that they remain functional until the beginning of the Gecko 71 cycle, but we will no longer need per-push coverage.

Priority: -- → P2

:egao, want to take this one as well :)

Flags: needinfo?(egao)
Blocks: 1522221

The affected tests are the android-common-tests and android-opt-tests:

https://searchfox.org/mozilla-central/rev/a887c90ea9c19a0b5529a1f5fa351929944887ba/taskcluster/ci/test/test-sets.yml#432-446

Or anything on android-em-4.3-arm7-api-16 ("Android 4.3 API16+" on treeherder).

I believe we want to continue all the Android builds per-push, in support of geckoview.

I think builds are ok for now, lets reduce tests to tier-2 and some can be turned off if they are running on android 7.0 as tier-1.

for tests that we are not planning to port (robocop, etc.) I would vote for leaving them for now- when more tests are ported to android 7.0, we can revisit the remaining 4.3 and see where we are.

(In reply to Joel Maher ( :jmaher ) (UTC-4) from comment #4)

I think builds are ok for now, lets reduce tests to tier-2 and some can be turned off if they are running on android 7.0 as tier-1.

Per our previous discussion, we don't want to turn any suites off at this point in time. Just demote to Tier 2 with reduced frequency. We need to ensure that 4.3 tests remain not-broken until the start of the Gecko 71 cycle.

ok, I was remembering incorrectly- thanks for correcting :ryanVM

:ryanvm, do we need pgo and debug, or just pgo?

Flags: needinfo?(ryanvm)

We should keep both IMO.

Also, I see a number of test suites being demoted here which don't we don't have running as Tier 1 on newer platforms (like reftests, mochitest-plain, etc). We need to be careful about effectively reducing our overall mobile test coverage here. For example, I can't imagine our Mobile team being very happy if the outcome here is that we could break layout on GV and not revert the change because the only failing test suite happens to be Tier 2 running in a 4.3 emulator. The intent of this work is cutting down on redundant suites, not just demoting everything-4.3.

Flags: needinfo?(ryanvm)

I've updated the original patch (which was a temporary WiP version) with something that should work as intended.

Flags: needinfo?(egao)
Assignee: nobody → egao
Attachment #9066588 - Attachment description: Bug 1548659 - demote all android-em-4.3 tests to tier 2 → Bug 1548659 - demote all android-em-4.3 tests to tier 2 and restrict platforms
Blocks: 1553955
Pushed by egao@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/22fad4a26360
demote all android-em-4.3 tests to tier 2 and restrict platforms r=jmaher,RyanVM,gbrown

:arthur_iakab - I identified the issue with the patch, it was a wrong keyword being used for default value.

I have pushed a new try run to verify that decision task is not busted, if that passes I will re-attempt a landing.

Flags: needinfo?(egao)
Pushed by egao@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/7717ce982073
demote all android-em-4.3 tests to tier 2 and restrict platforms r=jmaher,RyanVM,gbrown
Status: NEW → RESOLVED
Closed: 5 months ago
Resolution: --- → FIXED
Target Milestone: --- → Firefox 69
You need to log in before you can comment on or make changes to this bug.