1464254 - Update verify should grep for "WARN:" and "FAIL:" so that it ignore download retries

Status: RESOLVED FIXED

(Release Engineering :: Release Automation: Updates, enhancement)

Description

[Nick Thomas [:nthomas] (UTC+12)]

eg https://taskcluster-artifacts.net/IbTgSFruT-29bWGvcGJySg/0/public/logs/live_backing.log

Downloading 'http://archive.mozilla.org/pub/devedition/candidates/61.0b8-candidates/build1/update/linux-x86_64/es-AR/firefox-61.0b8.complete.mar' and placing in cache...
...
     0K ........ ........ ........ ........ ........ ........  7% 50.0K 12m49s
  3072K ........ ........ ........ ........ ........ ........ 14% 48.1K 12m2s
  6144K ........ ........ ........ ........ ........ ........ 22% 48.5K 11m1s
  9216K ........ ........ ........ ........ ........ ........ 29% 59.4K 9m31s
 12288K ........ ........ ........ ........ ........ .....WARNING: Timeout (300) exceeded, killing process 7021

but then a retry of the download succeeds.

Comment 1

[Nick Thomas [:nthomas] (UTC+12)]

Attachment: More targetted grep

(As you know ...) chunked-verify.sh is the outer wrapper of all the scripts doing update verify, and does a grep over the log at the very end of the task. It looking for any lines issued by https://hg.mozilla.org/build/tools/file/default/release/updates/verify.sh#l233 AIUI.

By making the grep more specific we'll avoid retry warnings, as well as the occasional failure I've seen when the (random!) X-Amz-Cf-Id header from AWS downloads contain WARN or FAIL.

Comment 2

[Nick Thomas [:nthomas] (UTC+12)]

https://hg.mozilla.org/build/tools/rev/7a96783ecbc7109b1f7494865c2b1153a56e7b6d