firefox-android: Make `external-gradle-dependencies` task more robust
Categories
(Release Engineering :: General, enhancement)
Tracking
(firefox113 fixed)
| Tracking | Status | |
|---|---|---|
| firefox113 | --- | fixed |
People
(Reporter: jlorenzo, Assigned: jlorenzo)
References
Details
Attachments
(4 files)
How we'd always worked around the problem...
Ever since the introduction of the toolchain-android-gradle-dependencies task in bug 1625126 (then renamed external-gradle-dependencies in bug 1807237), we've intermittently experienced busted caches.
Oftentimes, it's just a network error that prevents a gradle dependencies from being fetch which leads to errors like this one[1]:
[task 2023-03-02T10:45:15.414Z] FAILURE: Build failed with an exception.
[task 2023-03-02T10:45:15.414Z]
[task 2023-03-02T10:45:15.414Z] * What went wrong:
[task 2023-03-02T10:45:15.414Z] Execution failed for task ':support-rusthttp:generateDebugUnitTestStubRFile'.
[task 2023-03-02T10:45:15.414Z] > Could not resolve all files for configuration ':support-rusthttp:debugUnitTestRuntimeClasspath'.
[task 2023-03-02T10:45:15.414Z] > Failed to transform monitor-1.4.0.aar (androidx.test:monitor:1.4.0) to match attributes {artifactType=android-symbol-with-package-name, org.gradle.category=library, org.gradle.libraryelements=jar, org.gradle.status=release, org.gradle.usage=java-runtime}.
[task 2023-03-02T10:45:15.414Z] > Could not find monitor-1.4.0.aar (androidx.test:monitor:1.4.0).
[task 2023-03-02T10:45:15.415Z] Searched in the following locations:
[task 2023-03-02T10:45:15.415Z] file:/builds/worker/fetches/external-gradle-dependencies/google/androidx/test/monitor/1.4.0/monitor-1.4.0.aar
[task 2023-03-02T10:45:15.415Z] > Failed to transform monitor-1.4.0.aar (androidx.test:monitor:1.4.0) to match attributes {artifactType=android-symbol-with-package-name, org.gradle.status=release}.
[task 2023-03-02T10:45:15.416Z] > Could not find monitor-1.4.0.aar (androidx.test:monitor:1.4.0).
[task 2023-03-02T10:45:15.416Z] Searched in the following locations:
[task 2023-03-02T10:45:15.416Z] file:/builds/worker/fetches/external-gradle-dependencies/google/androidx/test/monitor/1.4.0/monitor-1.4.0.aar
[task 2023-03-02T10:45:15.416Z]
This is usually fixed by rerunning the corresponding external-gradle-dependencies task, and then rerunning the failed build task. It's worked so far.
... until we couldn't anymore
2 days ago, reruns didn't work anymore. Neither did rebuilding the cache by introducing a small change. This led to bug 1819776.
I took a closer look at the external-gradle-dependencies tasks and they're all crippled with these silent errors[2]:
[task 2023-03-01T21:47:23.445Z] > Task :app:compileFenixDebugUnitTestKotlin FAILED
[task 2023-03-01T21:47:23.447Z] e: /builds/worker/checkouts/vcs/fenix/app/src/test/java/org/mozilla/fenix/tabstray/TabSheetBehaviorManagerTest.kt: (47, 50): Type mismatch: inferred type is BottomSheetBehavior<ConstraintLayout> but BottomSheetBehavior<View> was expected
[...]
[task 2023-03-01T21:47:23.459Z] e: /builds/worker/checkouts/vcs/fenix/app/src/test/java/org/mozilla/fenix/tabstray/TabSheetBehaviorManagerTest.kt: (389, 51): Type mismatch: inferred type is BottomSheetBehavior<ConstraintLayout> but BottomSheetBehavior<View> was expected
[task 2023-03-01T21:47:23.459Z]
[task 2023-03-01T21:47:23.459Z] > Task :feature-intent:generateReleaseUnitTestStubRFile
[task 2023-03-01T21:47:23.944Z] > Task :feature-intent:compileDebugUnitTestKotlin
[task 2023-03-01T21:47:27.044Z]
[task 2023-03-01T21:47:27.045Z] > Task :app:compileFenixBetaUnitTestKotlin FAILED
[task 2023-03-01T21:47:27.046Z] e: /builds/worker/checkouts/vcs/fenix/app/src/test/java/org/mozilla/fenix/tabstray/TabSheetBehaviorManagerTest.kt: (47, 50): Type mismatch: inferred type is BottomSheetBehavior<ConstraintLayout> but BottomSheetBehavior<View> was expected
[...]
[task 2023-03-01T21:47:27.052Z] e: /builds/worker/checkouts/vcs/fenix/app/src/test/java/org/mozilla/fenix/tabstray/TabSheetBehaviorManagerTest.kt: (389, 51): Type mismatch: inferred type is BottomSheetBehavior<ConstraintLayout> but BottomSheetBehavior<View> was expected
[task 2023-03-01T21:47:30.444Z]
The reason why these compilation errors were silently ignored is simple: for the sake of being less prone to intermittent failures, we allow some gradle commands to fail[3].
What should we do?
- First immediate action: make these errors fatal and turn the
external-gradle-dependenciestasks red whenever a test fail. - Make these tasks more robust to intermittent failures, including network errors.
Regarding #2, I think we should run some shell commands to make sure we don't have any .part files present, nor any empty files, nor any .pom files alone. If that occurs, then we should raise a specific exit code and let Taskcluster rerun the task.
[1] https://github.com/mozilla-mobile/firefox-android/pull/948/checks?check_run_id=11710965714
[2] https://firefox-ci-tc.services.mozilla.com/tasks/SqtLbtwiRFCVYZk4vPdVDg/runs/0/logs/public/logs/live.log#L29018
[3] https://github.com/mozilla-mobile/firefox-android/blob/aa3adc0e486c55efa18f26e4a29615f2ba4aac5a/taskcluster/scripts/toolchain/external-gradle-dependencies.sh#L55
|
||
Comment 1•2 years ago
|
||
|
Assignee | |
Updated•2 years ago
|
|
|
||
Comment 2•2 years ago
|
||
|
|
||
Comment 3•2 years ago
|
||
|
|
||
Comment 4•2 years ago
|
||
|
||
Comment 5•2 years ago
|
||
Description
•