1807237 - firefox-android: Optimize gradle dependencies toolchain task on android-components

Status: RESOLVED FIXED

(Release Engineering :: General, enhancement)

Description

[Johan Lorenzo [:jlorenzo]]

In bug 1788606, we optimized gradle calls in the decision task. There's another gradle-based task that remains outstanding: the one that caches gradle dependencies. It was introduced in bug 1625126 and has regularly been called out as being too long, especially on Android-Components.

RELENG-736 suggests a solution: use a Nexus to proxy these dependencies. I think we shouldn't jump to this solution yet because of the lack of experience Mozilla has in monitoring such proxy. It's an easy target for supply-chain attacks.

Before, I think we should try something else: how about we split the toolchain tasks into smaller ones? Bug 1803141 made it easier to know what internal dependencies a gradle project has. So we can have 1 cache task per gradle project whose hash depends on its internal dependencies. This way, we don't wait on a single big task that caches all possibles deps.

I'll see how good this proposal is. I'd like to reduce the duration of this task before we migrate Fenix to the monorepo (bug 1803130). That duration delayed the migration by several hours on the day of the migration.

Comment 1

[Johan Lorenzo [:jlorenzo]]

Possibly duplicates bug 1619659.

Comment 2

[BMO Automation]

Attachment: [mozilla-mobile/firefox-android] Bug 1807237 (#490)

Comment 3

[Chris Peterson [:cpeterson]]

Johan, can we close this gradle bug?

You landed parts 1-9:
https://github.com/mozilla-mobile/firefox-android/commit/ecf523fb651dfe1ea4d4011f4e02129fa02a4eb6
...
https://github.com/mozilla-mobile/firefox-android/commit/366d63a61255bec91ed80be70b9a2a777d2b3883

Comment 4

[Johan Lorenzo [:jlorenzo]]

Sorry for the delay. I agree, let's close this bug as is. It definitely made things better on Android-Components. We're working on the Fenix migration to the monorepo and I see this task takes about an hour. We don't be able to optimize it the same way. Thus, I'll open a followup bug once we're done with the migration.

Comment 5

[BMO Automation]

Attachment: [mozilla-mobile/firefox-android] Bug 1807237 - Optimize task that caches external gradle dependencies (backport #490) (#681)

Comment 6

[BMO Automation]

Attachment: [mozilla-mobile/firefox-android] Bug 1807237 - Optimize task that caches external gradle dependencies (backport #490) (#682)

Comment 7

[Johan Lorenzo [:jlorenzo]]

(In reply to Johan Lorenzo [:jlorenzo] from comment #4)

Thus, I'll open a followup bug once we're done with the migration.

Filed as bug 1820122.

Comment 8

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/290a50aba5fc
part 1: Add missing `release_type` in params files
https://hg.mozilla.org/integration/autoland/rev/8f0385b12401
part 2: Upgrade taskgraph to get faster hashes
https://hg.mozilla.org/integration/autoland/rev/301e9ea00b37
part 3: Make `complete` depend on docker-images tasks
https://hg.mozilla.org/integration/autoland/rev/36b4b8341bce
part 4: Rename `build_config:loader` into `build_config:components_loader`
https://hg.mozilla.org/integration/autoland/rev/f7fb1b230878
part 5: Move too long treeherder group in `build_config` loader
https://hg.mozilla.org/integration/autoland/rev/775dd5ed6f00
part 6: Fix jacocoFocusDebugTestReport gradle task
https://hg.mozilla.org/integration/autoland/rev/03f5257d9a56
part 7: Do not let `build-components` tasks set `soft-dependencies`
https://hg.mozilla.org/integration/autoland/rev/be5c93752941
part 8: Rename `android-gradle-dependencies` into `external-gradle-dependencies`
https://hg.mozilla.org/integration/autoland/rev/64b08ecad85d
part 9: Split `external-gradle-dependencies` task per component