Bug 1820129 Comment 0 Edit History

Note: The actual edited comment in the bug view page will always show the original commenter’s name and original timestamp.

# How we've always worked around the problem

Ever since the introduction of the `toolchain-android-gradle-dependencies` task in bug 1625126 (then renamed `external-gradle-dependencies` in bug 1807237), we've intermittently experienced busted caches. 

Oftentimes, it's just a network error that prevents a gradle dependencies from being fetch which leads to errors like this one[1]: 

```
[task 2023-03-02T10:45:15.414Z] FAILURE: Build failed with an exception.
[task 2023-03-02T10:45:15.414Z] 
[task 2023-03-02T10:45:15.414Z] * What went wrong:
[task 2023-03-02T10:45:15.414Z] Execution failed for task ':support-rusthttp:generateDebugUnitTestStubRFile'.
[task 2023-03-02T10:45:15.414Z] > Could not resolve all files for configuration ':support-rusthttp:debugUnitTestRuntimeClasspath'.
[task 2023-03-02T10:45:15.414Z]    > Failed to transform monitor-1.4.0.aar (androidx.test:monitor:1.4.0) to match attributes {artifactType=android-symbol-with-package-name, org.gradle.category=library, org.gradle.libraryelements=jar, org.gradle.status=release, org.gradle.usage=java-runtime}.
[task 2023-03-02T10:45:15.414Z]       > Could not find monitor-1.4.0.aar (androidx.test:monitor:1.4.0).
[task 2023-03-02T10:45:15.415Z]         Searched in the following locations:
[task 2023-03-02T10:45:15.415Z]             file:/builds/worker/fetches/external-gradle-dependencies/google/androidx/test/monitor/1.4.0/monitor-1.4.0.aar
[task 2023-03-02T10:45:15.415Z]    > Failed to transform monitor-1.4.0.aar (androidx.test:monitor:1.4.0) to match attributes {artifactType=android-symbol-with-package-name, org.gradle.status=release}.
[task 2023-03-02T10:45:15.416Z]       > Could not find monitor-1.4.0.aar (androidx.test:monitor:1.4.0).
[task 2023-03-02T10:45:15.416Z]         Searched in the following locations:
[task 2023-03-02T10:45:15.416Z]             file:/builds/worker/fetches/external-gradle-dependencies/google/androidx/test/monitor/1.4.0/monitor-1.4.0.aar
[task 2023-03-02T10:45:15.416Z] 
```

This is usually fixed by rerunning the corresponding `external-gradle-dependencies` task, and then rerunning the failed build task. It's worked so far.

# Until we couldn't anymore

2 days ago, reruns didn't work anymore. Neither did rebuilding the cache by introducing a small change.

I took a closer look at the `external-gradle-dependencies` tasks and they're all crippled with these silent errors[2]:

```
[task 2023-03-01T21:47:23.445Z] > Task :app:compileFenixDebugUnitTestKotlin FAILED
[task 2023-03-01T21:47:23.447Z] e: /builds/worker/checkouts/vcs/fenix/app/src/test/java/org/mozilla/fenix/tabstray/TabSheetBehaviorManagerTest.kt: (47, 50): Type mismatch: inferred type is BottomSheetBehavior<ConstraintLayout> but BottomSheetBehavior<View> was expected
[...]
[task 2023-03-01T21:47:23.459Z] e: /builds/worker/checkouts/vcs/fenix/app/src/test/java/org/mozilla/fenix/tabstray/TabSheetBehaviorManagerTest.kt: (389, 51): Type mismatch: inferred type is BottomSheetBehavior<ConstraintLayout> but BottomSheetBehavior<View> was expected
[task 2023-03-01T21:47:23.459Z] 
[task 2023-03-01T21:47:23.459Z] > Task :feature-intent:generateReleaseUnitTestStubRFile
[task 2023-03-01T21:47:23.944Z] > Task :feature-intent:compileDebugUnitTestKotlin
[task 2023-03-01T21:47:27.044Z] 
[task 2023-03-01T21:47:27.045Z] > Task :app:compileFenixBetaUnitTestKotlin FAILED
[task 2023-03-01T21:47:27.046Z] e: /builds/worker/checkouts/vcs/fenix/app/src/test/java/org/mozilla/fenix/tabstray/TabSheetBehaviorManagerTest.kt: (47, 50): Type mismatch: inferred type is BottomSheetBehavior<ConstraintLayout> but BottomSheetBehavior<View> was expected
[...]
[task 2023-03-01T21:47:27.052Z] e: /builds/worker/checkouts/vcs/fenix/app/src/test/java/org/mozilla/fenix/tabstray/TabSheetBehaviorManagerTest.kt: (389, 51): Type mismatch: inferred type is BottomSheetBehavior<ConstraintLayout> but BottomSheetBehavior<View> was expected
[task 2023-03-01T21:47:30.444Z]
```

The reason why these compilation errors were silently ignored is simple: for the sake of being less prone to intermittent failures, we allow some gradle commands to fail[3].

# What should we do?

 1. First immediate action: make these errors fatal and turn the `external-gradle-dependencies` tasks red whenever a test fail. 
 2. Make these tasks more robust to intermittent failures, including network errors. 

Regarding #2, I think we should run some shell commands to make sure we don't have any `.part` files present, nor any empty files, nor any `.pom` files alone. If that occurs, then we should raise a specific exit code and let Taskcluster rerun the task. 


[1] https://github.com/mozilla-mobile/firefox-android/pull/948/checks?check_run_id=11710965714
[2] https://firefox-ci-tc.services.mozilla.com/tasks/SqtLbtwiRFCVYZk4vPdVDg/runs/0/logs/public/logs/live.log#L29018
[3] https://github.com/mozilla-mobile/firefox-android/blob/aa3adc0e486c55efa18f26e4a29615f2ba4aac5a/taskcluster/scripts/toolchain/external-gradle-dependencies.sh#L55
# How we've always worked around the problem...

Ever since the introduction of the `toolchain-android-gradle-dependencies` task in bug 1625126 (then renamed `external-gradle-dependencies` in bug 1807237), we've intermittently experienced busted caches. 

Oftentimes, it's just a network error that prevents a gradle dependencies from being fetch which leads to errors like this one[1]: 

```
[task 2023-03-02T10:45:15.414Z] FAILURE: Build failed with an exception.
[task 2023-03-02T10:45:15.414Z] 
[task 2023-03-02T10:45:15.414Z] * What went wrong:
[task 2023-03-02T10:45:15.414Z] Execution failed for task ':support-rusthttp:generateDebugUnitTestStubRFile'.
[task 2023-03-02T10:45:15.414Z] > Could not resolve all files for configuration ':support-rusthttp:debugUnitTestRuntimeClasspath'.
[task 2023-03-02T10:45:15.414Z]    > Failed to transform monitor-1.4.0.aar (androidx.test:monitor:1.4.0) to match attributes {artifactType=android-symbol-with-package-name, org.gradle.category=library, org.gradle.libraryelements=jar, org.gradle.status=release, org.gradle.usage=java-runtime}.
[task 2023-03-02T10:45:15.414Z]       > Could not find monitor-1.4.0.aar (androidx.test:monitor:1.4.0).
[task 2023-03-02T10:45:15.415Z]         Searched in the following locations:
[task 2023-03-02T10:45:15.415Z]             file:/builds/worker/fetches/external-gradle-dependencies/google/androidx/test/monitor/1.4.0/monitor-1.4.0.aar
[task 2023-03-02T10:45:15.415Z]    > Failed to transform monitor-1.4.0.aar (androidx.test:monitor:1.4.0) to match attributes {artifactType=android-symbol-with-package-name, org.gradle.status=release}.
[task 2023-03-02T10:45:15.416Z]       > Could not find monitor-1.4.0.aar (androidx.test:monitor:1.4.0).
[task 2023-03-02T10:45:15.416Z]         Searched in the following locations:
[task 2023-03-02T10:45:15.416Z]             file:/builds/worker/fetches/external-gradle-dependencies/google/androidx/test/monitor/1.4.0/monitor-1.4.0.aar
[task 2023-03-02T10:45:15.416Z] 
```

This is usually fixed by rerunning the corresponding `external-gradle-dependencies` task, and then rerunning the failed build task. It's worked so far.

# ... until we couldn't anymore

2 days ago, reruns didn't work anymore. Neither did rebuilding the cache by introducing a small change.

I took a closer look at the `external-gradle-dependencies` tasks and they're all crippled with these silent errors[2]:

```
[task 2023-03-01T21:47:23.445Z] > Task :app:compileFenixDebugUnitTestKotlin FAILED
[task 2023-03-01T21:47:23.447Z] e: /builds/worker/checkouts/vcs/fenix/app/src/test/java/org/mozilla/fenix/tabstray/TabSheetBehaviorManagerTest.kt: (47, 50): Type mismatch: inferred type is BottomSheetBehavior<ConstraintLayout> but BottomSheetBehavior<View> was expected
[...]
[task 2023-03-01T21:47:23.459Z] e: /builds/worker/checkouts/vcs/fenix/app/src/test/java/org/mozilla/fenix/tabstray/TabSheetBehaviorManagerTest.kt: (389, 51): Type mismatch: inferred type is BottomSheetBehavior<ConstraintLayout> but BottomSheetBehavior<View> was expected
[task 2023-03-01T21:47:23.459Z] 
[task 2023-03-01T21:47:23.459Z] > Task :feature-intent:generateReleaseUnitTestStubRFile
[task 2023-03-01T21:47:23.944Z] > Task :feature-intent:compileDebugUnitTestKotlin
[task 2023-03-01T21:47:27.044Z] 
[task 2023-03-01T21:47:27.045Z] > Task :app:compileFenixBetaUnitTestKotlin FAILED
[task 2023-03-01T21:47:27.046Z] e: /builds/worker/checkouts/vcs/fenix/app/src/test/java/org/mozilla/fenix/tabstray/TabSheetBehaviorManagerTest.kt: (47, 50): Type mismatch: inferred type is BottomSheetBehavior<ConstraintLayout> but BottomSheetBehavior<View> was expected
[...]
[task 2023-03-01T21:47:27.052Z] e: /builds/worker/checkouts/vcs/fenix/app/src/test/java/org/mozilla/fenix/tabstray/TabSheetBehaviorManagerTest.kt: (389, 51): Type mismatch: inferred type is BottomSheetBehavior<ConstraintLayout> but BottomSheetBehavior<View> was expected
[task 2023-03-01T21:47:30.444Z]
```

The reason why these compilation errors were silently ignored is simple: for the sake of being less prone to intermittent failures, we allow some gradle commands to fail[3].

# What should we do?

 1. First immediate action: make these errors fatal and turn the `external-gradle-dependencies` tasks red whenever a test fail. 
 2. Make these tasks more robust to intermittent failures, including network errors. 

Regarding #2, I think we should run some shell commands to make sure we don't have any `.part` files present, nor any empty files, nor any `.pom` files alone. If that occurs, then we should raise a specific exit code and let Taskcluster rerun the task. 


[1] https://github.com/mozilla-mobile/firefox-android/pull/948/checks?check_run_id=11710965714
[2] https://firefox-ci-tc.services.mozilla.com/tasks/SqtLbtwiRFCVYZk4vPdVDg/runs/0/logs/public/logs/live.log#L29018
[3] https://github.com/mozilla-mobile/firefox-android/blob/aa3adc0e486c55efa18f26e4a29615f2ba4aac5a/taskcluster/scripts/toolchain/external-gradle-dependencies.sh#L55
# How we'dalways worked around the problem...

Ever since the introduction of the `toolchain-android-gradle-dependencies` task in bug 1625126 (then renamed `external-gradle-dependencies` in bug 1807237), we've intermittently experienced busted caches. 

Oftentimes, it's just a network error that prevents a gradle dependencies from being fetch which leads to errors like this one[1]: 

```
[task 2023-03-02T10:45:15.414Z] FAILURE: Build failed with an exception.
[task 2023-03-02T10:45:15.414Z] 
[task 2023-03-02T10:45:15.414Z] * What went wrong:
[task 2023-03-02T10:45:15.414Z] Execution failed for task ':support-rusthttp:generateDebugUnitTestStubRFile'.
[task 2023-03-02T10:45:15.414Z] > Could not resolve all files for configuration ':support-rusthttp:debugUnitTestRuntimeClasspath'.
[task 2023-03-02T10:45:15.414Z]    > Failed to transform monitor-1.4.0.aar (androidx.test:monitor:1.4.0) to match attributes {artifactType=android-symbol-with-package-name, org.gradle.category=library, org.gradle.libraryelements=jar, org.gradle.status=release, org.gradle.usage=java-runtime}.
[task 2023-03-02T10:45:15.414Z]       > Could not find monitor-1.4.0.aar (androidx.test:monitor:1.4.0).
[task 2023-03-02T10:45:15.415Z]         Searched in the following locations:
[task 2023-03-02T10:45:15.415Z]             file:/builds/worker/fetches/external-gradle-dependencies/google/androidx/test/monitor/1.4.0/monitor-1.4.0.aar
[task 2023-03-02T10:45:15.415Z]    > Failed to transform monitor-1.4.0.aar (androidx.test:monitor:1.4.0) to match attributes {artifactType=android-symbol-with-package-name, org.gradle.status=release}.
[task 2023-03-02T10:45:15.416Z]       > Could not find monitor-1.4.0.aar (androidx.test:monitor:1.4.0).
[task 2023-03-02T10:45:15.416Z]         Searched in the following locations:
[task 2023-03-02T10:45:15.416Z]             file:/builds/worker/fetches/external-gradle-dependencies/google/androidx/test/monitor/1.4.0/monitor-1.4.0.aar
[task 2023-03-02T10:45:15.416Z] 
```

This is usually fixed by rerunning the corresponding `external-gradle-dependencies` task, and then rerunning the failed build task. It's worked so far.

# ... until we couldn't anymore

2 days ago, reruns didn't work anymore. Neither did rebuilding the cache by introducing a small change.

I took a closer look at the `external-gradle-dependencies` tasks and they're all crippled with these silent errors[2]:

```
[task 2023-03-01T21:47:23.445Z] > Task :app:compileFenixDebugUnitTestKotlin FAILED
[task 2023-03-01T21:47:23.447Z] e: /builds/worker/checkouts/vcs/fenix/app/src/test/java/org/mozilla/fenix/tabstray/TabSheetBehaviorManagerTest.kt: (47, 50): Type mismatch: inferred type is BottomSheetBehavior<ConstraintLayout> but BottomSheetBehavior<View> was expected
[...]
[task 2023-03-01T21:47:23.459Z] e: /builds/worker/checkouts/vcs/fenix/app/src/test/java/org/mozilla/fenix/tabstray/TabSheetBehaviorManagerTest.kt: (389, 51): Type mismatch: inferred type is BottomSheetBehavior<ConstraintLayout> but BottomSheetBehavior<View> was expected
[task 2023-03-01T21:47:23.459Z] 
[task 2023-03-01T21:47:23.459Z] > Task :feature-intent:generateReleaseUnitTestStubRFile
[task 2023-03-01T21:47:23.944Z] > Task :feature-intent:compileDebugUnitTestKotlin
[task 2023-03-01T21:47:27.044Z] 
[task 2023-03-01T21:47:27.045Z] > Task :app:compileFenixBetaUnitTestKotlin FAILED
[task 2023-03-01T21:47:27.046Z] e: /builds/worker/checkouts/vcs/fenix/app/src/test/java/org/mozilla/fenix/tabstray/TabSheetBehaviorManagerTest.kt: (47, 50): Type mismatch: inferred type is BottomSheetBehavior<ConstraintLayout> but BottomSheetBehavior<View> was expected
[...]
[task 2023-03-01T21:47:27.052Z] e: /builds/worker/checkouts/vcs/fenix/app/src/test/java/org/mozilla/fenix/tabstray/TabSheetBehaviorManagerTest.kt: (389, 51): Type mismatch: inferred type is BottomSheetBehavior<ConstraintLayout> but BottomSheetBehavior<View> was expected
[task 2023-03-01T21:47:30.444Z]
```

The reason why these compilation errors were silently ignored is simple: for the sake of being less prone to intermittent failures, we allow some gradle commands to fail[3].

# What should we do?

 1. First immediate action: make these errors fatal and turn the `external-gradle-dependencies` tasks red whenever a test fail. 
 2. Make these tasks more robust to intermittent failures, including network errors. 

Regarding #2, I think we should run some shell commands to make sure we don't have any `.part` files present, nor any empty files, nor any `.pom` files alone. If that occurs, then we should raise a specific exit code and let Taskcluster rerun the task. 


[1] https://github.com/mozilla-mobile/firefox-android/pull/948/checks?check_run_id=11710965714
[2] https://firefox-ci-tc.services.mozilla.com/tasks/SqtLbtwiRFCVYZk4vPdVDg/runs/0/logs/public/logs/live.log#L29018
[3] https://github.com/mozilla-mobile/firefox-android/blob/aa3adc0e486c55efa18f26e4a29615f2ba4aac5a/taskcluster/scripts/toolchain/external-gradle-dependencies.sh#L55
# How we'd always worked around the problem...

Ever since the introduction of the `toolchain-android-gradle-dependencies` task in bug 1625126 (then renamed `external-gradle-dependencies` in bug 1807237), we've intermittently experienced busted caches. 

Oftentimes, it's just a network error that prevents a gradle dependencies from being fetch which leads to errors like this one[1]: 

```
[task 2023-03-02T10:45:15.414Z] FAILURE: Build failed with an exception.
[task 2023-03-02T10:45:15.414Z] 
[task 2023-03-02T10:45:15.414Z] * What went wrong:
[task 2023-03-02T10:45:15.414Z] Execution failed for task ':support-rusthttp:generateDebugUnitTestStubRFile'.
[task 2023-03-02T10:45:15.414Z] > Could not resolve all files for configuration ':support-rusthttp:debugUnitTestRuntimeClasspath'.
[task 2023-03-02T10:45:15.414Z]    > Failed to transform monitor-1.4.0.aar (androidx.test:monitor:1.4.0) to match attributes {artifactType=android-symbol-with-package-name, org.gradle.category=library, org.gradle.libraryelements=jar, org.gradle.status=release, org.gradle.usage=java-runtime}.
[task 2023-03-02T10:45:15.414Z]       > Could not find monitor-1.4.0.aar (androidx.test:monitor:1.4.0).
[task 2023-03-02T10:45:15.415Z]         Searched in the following locations:
[task 2023-03-02T10:45:15.415Z]             file:/builds/worker/fetches/external-gradle-dependencies/google/androidx/test/monitor/1.4.0/monitor-1.4.0.aar
[task 2023-03-02T10:45:15.415Z]    > Failed to transform monitor-1.4.0.aar (androidx.test:monitor:1.4.0) to match attributes {artifactType=android-symbol-with-package-name, org.gradle.status=release}.
[task 2023-03-02T10:45:15.416Z]       > Could not find monitor-1.4.0.aar (androidx.test:monitor:1.4.0).
[task 2023-03-02T10:45:15.416Z]         Searched in the following locations:
[task 2023-03-02T10:45:15.416Z]             file:/builds/worker/fetches/external-gradle-dependencies/google/androidx/test/monitor/1.4.0/monitor-1.4.0.aar
[task 2023-03-02T10:45:15.416Z] 
```

This is usually fixed by rerunning the corresponding `external-gradle-dependencies` task, and then rerunning the failed build task. It's worked so far.

# ... until we couldn't anymore

2 days ago, reruns didn't work anymore. Neither did rebuilding the cache by introducing a small change.

I took a closer look at the `external-gradle-dependencies` tasks and they're all crippled with these silent errors[2]:

```
[task 2023-03-01T21:47:23.445Z] > Task :app:compileFenixDebugUnitTestKotlin FAILED
[task 2023-03-01T21:47:23.447Z] e: /builds/worker/checkouts/vcs/fenix/app/src/test/java/org/mozilla/fenix/tabstray/TabSheetBehaviorManagerTest.kt: (47, 50): Type mismatch: inferred type is BottomSheetBehavior<ConstraintLayout> but BottomSheetBehavior<View> was expected
[...]
[task 2023-03-01T21:47:23.459Z] e: /builds/worker/checkouts/vcs/fenix/app/src/test/java/org/mozilla/fenix/tabstray/TabSheetBehaviorManagerTest.kt: (389, 51): Type mismatch: inferred type is BottomSheetBehavior<ConstraintLayout> but BottomSheetBehavior<View> was expected
[task 2023-03-01T21:47:23.459Z] 
[task 2023-03-01T21:47:23.459Z] > Task :feature-intent:generateReleaseUnitTestStubRFile
[task 2023-03-01T21:47:23.944Z] > Task :feature-intent:compileDebugUnitTestKotlin
[task 2023-03-01T21:47:27.044Z] 
[task 2023-03-01T21:47:27.045Z] > Task :app:compileFenixBetaUnitTestKotlin FAILED
[task 2023-03-01T21:47:27.046Z] e: /builds/worker/checkouts/vcs/fenix/app/src/test/java/org/mozilla/fenix/tabstray/TabSheetBehaviorManagerTest.kt: (47, 50): Type mismatch: inferred type is BottomSheetBehavior<ConstraintLayout> but BottomSheetBehavior<View> was expected
[...]
[task 2023-03-01T21:47:27.052Z] e: /builds/worker/checkouts/vcs/fenix/app/src/test/java/org/mozilla/fenix/tabstray/TabSheetBehaviorManagerTest.kt: (389, 51): Type mismatch: inferred type is BottomSheetBehavior<ConstraintLayout> but BottomSheetBehavior<View> was expected
[task 2023-03-01T21:47:30.444Z]
```

The reason why these compilation errors were silently ignored is simple: for the sake of being less prone to intermittent failures, we allow some gradle commands to fail[3].

# What should we do?

 1. First immediate action: make these errors fatal and turn the `external-gradle-dependencies` tasks red whenever a test fail. 
 2. Make these tasks more robust to intermittent failures, including network errors. 

Regarding #2, I think we should run some shell commands to make sure we don't have any `.part` files present, nor any empty files, nor any `.pom` files alone. If that occurs, then we should raise a specific exit code and let Taskcluster rerun the task. 


[1] https://github.com/mozilla-mobile/firefox-android/pull/948/checks?check_run_id=11710965714
[2] https://firefox-ci-tc.services.mozilla.com/tasks/SqtLbtwiRFCVYZk4vPdVDg/runs/0/logs/public/logs/live.log#L29018
[3] https://github.com/mozilla-mobile/firefox-android/blob/aa3adc0e486c55efa18f26e4a29615f2ba4aac5a/taskcluster/scripts/toolchain/external-gradle-dependencies.sh#L55
# How we'd always worked around the problem...

Ever since the introduction of the `toolchain-android-gradle-dependencies` task in bug 1625126 (then renamed `external-gradle-dependencies` in bug 1807237), we've intermittently experienced busted caches. 

Oftentimes, it's just a network error that prevents a gradle dependencies from being fetch which leads to errors like this one[1]: 

```
[task 2023-03-02T10:45:15.414Z] FAILURE: Build failed with an exception.
[task 2023-03-02T10:45:15.414Z] 
[task 2023-03-02T10:45:15.414Z] * What went wrong:
[task 2023-03-02T10:45:15.414Z] Execution failed for task ':support-rusthttp:generateDebugUnitTestStubRFile'.
[task 2023-03-02T10:45:15.414Z] > Could not resolve all files for configuration ':support-rusthttp:debugUnitTestRuntimeClasspath'.
[task 2023-03-02T10:45:15.414Z]    > Failed to transform monitor-1.4.0.aar (androidx.test:monitor:1.4.0) to match attributes {artifactType=android-symbol-with-package-name, org.gradle.category=library, org.gradle.libraryelements=jar, org.gradle.status=release, org.gradle.usage=java-runtime}.
[task 2023-03-02T10:45:15.414Z]       > Could not find monitor-1.4.0.aar (androidx.test:monitor:1.4.0).
[task 2023-03-02T10:45:15.415Z]         Searched in the following locations:
[task 2023-03-02T10:45:15.415Z]             file:/builds/worker/fetches/external-gradle-dependencies/google/androidx/test/monitor/1.4.0/monitor-1.4.0.aar
[task 2023-03-02T10:45:15.415Z]    > Failed to transform monitor-1.4.0.aar (androidx.test:monitor:1.4.0) to match attributes {artifactType=android-symbol-with-package-name, org.gradle.status=release}.
[task 2023-03-02T10:45:15.416Z]       > Could not find monitor-1.4.0.aar (androidx.test:monitor:1.4.0).
[task 2023-03-02T10:45:15.416Z]         Searched in the following locations:
[task 2023-03-02T10:45:15.416Z]             file:/builds/worker/fetches/external-gradle-dependencies/google/androidx/test/monitor/1.4.0/monitor-1.4.0.aar
[task 2023-03-02T10:45:15.416Z] 
```

This is usually fixed by rerunning the corresponding `external-gradle-dependencies` task, and then rerunning the failed build task. It's worked so far.

# ... until we couldn't anymore

2 days ago, reruns didn't work anymore. Neither did rebuilding the cache by introducing a small change. This led to bug 1819776.

I took a closer look at the `external-gradle-dependencies` tasks and they're all crippled with these silent errors[2]:

```
[task 2023-03-01T21:47:23.445Z] > Task :app:compileFenixDebugUnitTestKotlin FAILED
[task 2023-03-01T21:47:23.447Z] e: /builds/worker/checkouts/vcs/fenix/app/src/test/java/org/mozilla/fenix/tabstray/TabSheetBehaviorManagerTest.kt: (47, 50): Type mismatch: inferred type is BottomSheetBehavior<ConstraintLayout> but BottomSheetBehavior<View> was expected
[...]
[task 2023-03-01T21:47:23.459Z] e: /builds/worker/checkouts/vcs/fenix/app/src/test/java/org/mozilla/fenix/tabstray/TabSheetBehaviorManagerTest.kt: (389, 51): Type mismatch: inferred type is BottomSheetBehavior<ConstraintLayout> but BottomSheetBehavior<View> was expected
[task 2023-03-01T21:47:23.459Z] 
[task 2023-03-01T21:47:23.459Z] > Task :feature-intent:generateReleaseUnitTestStubRFile
[task 2023-03-01T21:47:23.944Z] > Task :feature-intent:compileDebugUnitTestKotlin
[task 2023-03-01T21:47:27.044Z] 
[task 2023-03-01T21:47:27.045Z] > Task :app:compileFenixBetaUnitTestKotlin FAILED
[task 2023-03-01T21:47:27.046Z] e: /builds/worker/checkouts/vcs/fenix/app/src/test/java/org/mozilla/fenix/tabstray/TabSheetBehaviorManagerTest.kt: (47, 50): Type mismatch: inferred type is BottomSheetBehavior<ConstraintLayout> but BottomSheetBehavior<View> was expected
[...]
[task 2023-03-01T21:47:27.052Z] e: /builds/worker/checkouts/vcs/fenix/app/src/test/java/org/mozilla/fenix/tabstray/TabSheetBehaviorManagerTest.kt: (389, 51): Type mismatch: inferred type is BottomSheetBehavior<ConstraintLayout> but BottomSheetBehavior<View> was expected
[task 2023-03-01T21:47:30.444Z]
```

The reason why these compilation errors were silently ignored is simple: for the sake of being less prone to intermittent failures, we allow some gradle commands to fail[3].

# What should we do?

 1. First immediate action: make these errors fatal and turn the `external-gradle-dependencies` tasks red whenever a test fail. 
 2. Make these tasks more robust to intermittent failures, including network errors. 

Regarding #2, I think we should run some shell commands to make sure we don't have any `.part` files present, nor any empty files, nor any `.pom` files alone. If that occurs, then we should raise a specific exit code and let Taskcluster rerun the task. 


[1] https://github.com/mozilla-mobile/firefox-android/pull/948/checks?check_run_id=11710965714
[2] https://firefox-ci-tc.services.mozilla.com/tasks/SqtLbtwiRFCVYZk4vPdVDg/runs/0/logs/public/logs/live.log#L29018
[3] https://github.com/mozilla-mobile/firefox-android/blob/aa3adc0e486c55efa18f26e4a29615f2ba4aac5a/taskcluster/scripts/toolchain/external-gradle-dependencies.sh#L55

Back to Bug 1820129 Comment 0