1803141 - firefox-android: Have Focus dependencies known by taskgraph.

Status: RESOLVED FIXED

(Release Engineering :: General, task)

Description

[Johan Lorenzo [:jlorenzo]]

In mozilla-mobile/android-components#7617, we optimized the number of Android-Components tasks scheduled by stripping away the components that remained unchanged. This happened with a few shortcomings like bug 1642977. Another one is this implementation betrays the core design of taskgraph. Taskgraph first generates the full graph before optimizing it[1]. In the current implementation, jobs don't even get created so they never make the full graph.

With Fenix coming to the Android monorepo, we want to be smarter about the way we schedule tasks. For instance a change in Fenix shouldn't trigger Focus tasks and the other way around. Similarly for A-C: a change in a component used by Fenix must trigger Fenix tasks. This means, we need to revisit how we make links between Taskcluster tasks and we should stop preventing them from reaching the full graph. Otherwise, Fenix is unable to when it needs to be rebuilt, for instance.

We can leverage what was done in mozilla-mobile/android-components#7617 by calling gradle and parsing its output. For instance:

cd focus-android
./gradlew 'app:dependencies' --configuration 'implementation'

gave some interesting results. We should be able to use taskgraph's skip-unless-change with a list of generated directories based on the output of such commands. That said, we should first handle bug 1642977. That bug will ensure the complete-push/complete-pr have the right dependencies.

[1] https://johanlorenzo.github.io/blog/2019/10/24/taskgraph-is-now-deployed-to-the-biggest-mozilla-mobile-projects.html

Comment 1

[BMO Automation]

Attachment: [mozilla-mobile/firefox-android] Bug 1803141 - part 1-6: Build/test components only when they change (#253)

Comment 2

[BMO Automation]

Attachment: [mozilla-mobile/firefox-android] Bug 1803141 - part 7: Build focus only when it or one of its dependen… (#284)

Comment 3

[BMO Automation]

Attachment: [mozilla-mobile/firefox-android] Bug 1803141 - part 8: Build focus only when it or one of its dependen… (backport #284) (#308)

Comment 4

[BMO Automation]

Attachment: [mozilla-mobile/firefox-android] Bug 1803141 - part 1-7: Build/test components only when they change (backport #253) (#330)

Comment 5

[BMO Automation]

Attachment: [mozilla-mobile/firefox-android] Bug 1803141 - part 8: Build focus only when it or one of its dependen… (backport #284) (#363)

Comment 6

[Johan Lorenzo [:jlorenzo]]

Status update: We ended up implementing comment 0 for the focus migration (bug 1797704). We even leveraged the output to the decision task faster (bug 1788606). Fenix is not implemented yet but will reuse what we did in these 2 bugs. The next step is to build sample-browser and focus-debug only when changed (bug 1806601).

Thus, let's close this current bug as fixed.

Comment 7

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/0cb0ae65c33b
part 1: Remove unused vars/imports in `build_config` loader
https://hg.mozilla.org/integration/autoland/rev/e08d0b1e244a
part 2: Make gradle dependencies parsing more generic
https://hg.mozilla.org/integration/autoland/rev/6fd2079ad50f
part 3: Optimize away build tasks during taskgraph optimization phase
https://hg.mozilla.org/integration/autoland/rev/d1608688694b
part 4: Factorize UI tests commons in a transform
https://hg.mozilla.org/integration/autoland/rev/672da83ff5fa
part 5: Re-enable tests on features-downloads and features-recentlyclosed
https://hg.mozilla.org/integration/autoland/rev/4691b8c4042b
part 6: Run UI tests only when components under tests are changed
https://hg.mozilla.org/integration/autoland/rev/d15f64f5c04a
part 7: Build samples-browser only when changed
https://hg.mozilla.org/integration/autoland/rev/48c34c5ff98f
part 8: Build focus only when it or one of its dependencies change