Retry failed `nimbus-fml.zip` downloads in the Nimbus Gradle Plugin
Categories
(Application Services :: Nimbus SDK, enhancement)
Tracking
(Not tracked)
People
(Reporter: lina, Unassigned)
References
(Blocks 1 open bug)
Details
Downloading nimbus-fml.zip can fail, especially on TaskCluster workers with flakier-than-usual networking (e.g., bug 1889661). I think the URL.withInputStream() API that we're using now will raise an IOException on such a failure, and then fail the entire task if the archive couldn't be downloaded. That'll cause oranges for the gradle-dependencies task in Treeherder.
I wonder if we could make this situation a little better by:
- Retrying the download with exponential backoff on a socket (or 500-class HTTP status code) error.
- Optionally, using HTTP range requests to resume interrupted downloads, so that we don't have to start the 67 MB download from the beginning on each retry.
archive.mozilla.orgadvertises support for these, but TaskCluster doesn't, so there might be diminishing returns here.
For both of these, we need something a little more advanced than URL.withInputStream(). Luckily, Java 11 added a new java.net.http.HttpClient API that we can use (here's a nice quick start guide and recipes for it), without needing to pull in HttpComponents / OkHttp / Jetty. Our version of AGP requires Java 17 (bug 1831352), so I think that means we can use HttpClient! 🎉
Description
•