1883499 - Update task schedule optimizations for firefox-android tasks

Status: RESOLVED FIXED

(Release Engineering :: General, enhancement)

Description

[Geoff Brown [:gbrown]]

On firefox-android, the gradle_optimization taskgraph transform adds skip-unless-changed optimizations such that non-release taskgraphs only run fenix tasks if fenix files have changed, focus tasks if focus files have changed, and android-components tasks if a-c files have changed.

In discussions with Aaron and Jackie today, we noted that Geckoview changes can cause test regressions for firefox-android, and such regressions may slip by because of the skip-unless-changed optimizations. Let's update the transform such that geckoview changes trigger A-C, Focus, and Fenix tests.

There is a similar concern for purely Gecko changes: maybe we should remove the skip-unless-changed entirely, so that every push runs A-C/Focus/Fenix tasks (modulo autoland task optimization). However, that seems unwarranted at this time: We think there's less chance of Gecko changes causing firefox-android bustage.

Comment 1

[Geoff Brown [:gbrown]]

There's also discussion about using SCHEDULES for task optimization in bug 1882468.

Comment 2

[Geoff Brown [:gbrown]]

There are (at least) two obvious approaches to including geckoview changes as triggers for firefox-android tasks:

1. If anything in mobile/android/** changes, run all A-C, Focus, and Fenix builds and tests.
2. If anything in mobile/android/geckoview/** or mobile/android/android-components/** changes, run A-C tasks;
   if anything in mobile/android/geckoview/** or mobile/android/focus-android/** changes, run Focus tasks;
   if anything in mobile/android/geckoview/** or mobile/android/fenix/** changes, run Fenix tasks.

Preferences?

Comment 3

[Geoff Brown [:gbrown]]

(In reply to Geoff Brown [:gbrown] from comment #0)

There is a similar concern for purely Gecko changes: maybe we should remove the skip-unless-changed entirely, so that every push runs A-C/Focus/Fenix tasks (modulo autoland task optimization). However, that seems unwarranted at this time: We think there's less chance of Gecko changes causing firefox-android bustage.

On the other hand...

• Gecko changes can cause firefox-android bustage
• If firefox-android tasks are eliminated from a series of autoland pushes by the machine-learning optimization and the next backstop push has f-a tasks eliminated by skip-unless-changed, those f-a tasks may go for many pushes without running, allowing faults to go undetected for too long and making regression-range finding more difficult.

Quoting Aryx from a discussion yesterday: "What's needed from the sheriffing side: Backstop pushes always run the desktop, geckoview, fenix and focus tasks."

I think that means we should remove the existing firefox-android skip-unless-changed optimizations entirely, treating firefox-android builds and tests the same as Gecko builds and tests.

That might (depending partly on the behavior of the autoland machine-learning) result in a significant increase in firebase tasks; we could reduce the firebase load by running those tests only on backstop pushes (skip-unless-backstop optimization).

Aaron, Aryx: How's that sound?

Comment 4

[Sebastian Hengst [:aryx] (needinfo me if it's about an intermittent or backout)]

Based on this code, tasks available (= run-on-projects includes autoland, built-projects or trunk as far as I understand it) will run for backstop pushes.

Comment 5

[Aaron Train [:aaronmt]]

(In reply to Geoff Brown [:gbrown] from comment #3)

(In reply to Geoff Brown [:gbrown] from comment #0)

There is a similar concern for purely Gecko changes: maybe we should remove the skip-unless-changed entirely, so that every push runs A-C/Focus/Fenix tasks (modulo autoland task optimization). However, that seems unwarranted at this time: We think there's less chance of Gecko changes causing firefox-android bustage.

On the other hand...

• Gecko changes can cause firefox-android bustage
• If firefox-android tasks are eliminated from a series of autoland pushes by the machine-learning optimization and the next backstop push has f-a tasks eliminated by skip-unless-changed, those f-a tasks may go for many pushes without running, allowing faults to go undetected for too long and making regression-range finding more difficult.

Quoting Aryx from a discussion yesterday: "What's needed from the sheriffing side: Backstop pushes always run the desktop, geckoview, fenix and focus tasks."

I think that means we should remove the existing firefox-android skip-unless-changed optimizations entirely, treating firefox-android builds and tests the same as Gecko builds and tests.

That might (depending partly on the behavior of the autoland machine-learning) result in a significant increase in firebase tasks; we could reduce the firebase load by running those tests only on backstop pushes (skip-unless-backstop optimization).

Aaron, Aryx: How's that sound?

This sound's good to me as I understand it. I'd rather we scale back extraneous scheduling (it does cost money/time after all) rather than the not schedule enough. In any case where signed Android APK's are built with code changes, we should schedule. I'll pass this to Jackie if he has any additional thought here.

Comment 6

[Julien Cristau [:jcristau]]

Attachment: Bug 1883499 - Update task schedule optimizations for firefox-android

Comment 7

[Julien Cristau [:jcristau]]

https://hg.mozilla.org/mozilla-central/rev/67157e0e6034