1820125 - firefox-android: `.buildconfig.yml` doesn't contain transitive dependencies

Status: RESOLVED FIXED

(Release Engineering :: General, defect)

Description

[Johan Lorenzo [:jlorenzo]]

What happened then?

In bug 1788606, we output the internal upstream dependencies of each Android-Component and browser app in .buildconfig.yml. This enables the decision task to run faster. Before then, the decision task used the gradle parser wrote in [1], 3 years ago.

In bug 1807237, we split the external-gradle-dependencies task into multiple ones (one per Android-Component or browser app). This reduced the overall time to get gradle upstream dependencies cached on Taskcluster.

What's going on today?

The combination of these 2 optimizations unveiled a 3-year-old bug in the said gradle parser. This PR[2] is busted because the gradle cache for compose-engine doesn't contain a transitive dependency (com.google.android.material) declared in support-ktx:

[task 2023-03-02T09:50:02.211Z] FAILURE: Build completed with 2 failures.
[task 2023-03-02T09:50:02.211Z] 
[task 2023-03-02T09:50:02.211Z] 1: Task failed with an exception.
[task 2023-03-02T09:50:02.211Z] -----------
[task 2023-03-02T09:50:02.211Z] * What went wrong:
[task 2023-03-02T09:50:02.211Z] Execution failed for task ':compose-engine:mapReleaseSourceSetPaths'.
[task 2023-03-02T09:50:02.211Z] > Could not resolve all files for configuration ':compose-engine:releaseRuntimeClasspath'.
[task 2023-03-02T09:50:02.211Z]    > Could not find com.google.android.material:material:1.2.1.
[task 2023-03-02T09:50:02.211Z]      Required by:
[task 2023-03-02T09:50:02.211Z]          project :compose-engine > project :browser-state > project :support-utils
[task 2023-03-02T09:50:02.211Z]          project :compose-engine > project :browser-state > project :support-ktx

The thing is: support-ktx is not declared as an upstream dependency of compose-engine[4]. The reason is: it was never explicitly declared as such in the corresponding build.gradle file. Tom White worked around the problem by making support-ktx explicit.

How can we improve the situation?

We must make the gradle parser smarter. It must detect every internal upstream dependencies including transitive ones.

While today's bug is the result of 2 successive optimizations, we've been definitely prone to this class of bugs since the introduction of this parser, 3 years ago. This means, we've been lucky that no one introduced a bug that was discovered after the PR got merged to main.

[1] https://github.com/mozilla-mobile/android-components/pull/7617
[2] https://github.com/mozilla-mobile/firefox-android/pull/948
[3] https://github.com/mozilla-mobile/firefox-android/pull/948/checks?check_run_id=11710964658
[4] https://github.com/mozilla-mobile/firefox-android/blob/0281b2c8b919888ca181410510b184f085fdbc6e/android-components/.buildconfig.yml#L185-L193

Comment 1

[Johan Lorenzo [:jlorenzo]]

Bug 1795152 is about making the parser better. This current bug won't change the output of gradle, though. It will just be better at finding the local dependencies.

Comment 2

[BMO Automation]

Attachment: [mozilla-mobile/firefox-android] Bug 1820125 - Let `.buildconfig.yml` contain transitive dependencies (#1090)

Comment 3

[BMO Automation]

Authored by https://github.com/JohanLorenzo
https://github.com/mozilla-mobile/firefox-android/commit/55ff47dbd8f64f159f6a6eeb70918b657d65e2aa
Bug 1820125 - part 1: Parse gradle dependencies to take transitive dependencies into account

Authored by https://github.com/JohanLorenzo
https://github.com/mozilla-mobile/firefox-android/commit/15b47e9549094d439d950660da5950926104958d
Bug 1820125 - part 2: Do not make fenix depend on `mozilla-lint-rules`

Authored by https://github.com/JohanLorenzo
https://github.com/mozilla-mobile/firefox-android/commit/39e9862c62c354a183f4d5070ec77b0d30810462
Bug 1820125 - part 3: Do not run gradle threads in parallel to ensure gradle output is sane

Authored by https://github.com/JohanLorenzo
https://github.com/mozilla-mobile/firefox-android/commit/4c846496a4f284f99878621a02f2184f242f64e4
Bug 1820125 - part 4: Generate `.buildconfig.yml` files with the new parser