696945 - retry downloads of builds, tests, and symbols on mozilla-tests

Status: RESOLVED FIXED

(Release Engineering :: General, defect, P3)

Description

[John Ford [:jhford] CET/CEST Berlin Time]

Today, someone restart apache without warning and it caused some major tree horkage.  We should retry downloading the build, test and symbols files when this happens. 

The log looked like.  This turned the tests red instead of purple and did not set RETRY, so the build never retried.

--15:49:35--  http://stage.mozilla.org/pub/mozilla.org/firefox/tinderbox-builds/mozilla-inbound-win32-debug/1319490284/firefox-10.0a1.en-US.win32.crashreporter-symbols.zip
           => `firefox-10.0a1.en-US.win32.crashreporter-symbols.zip'
Resolving stage.mozilla.org... 10.2.74.116
Connecting to stage.mozilla.org|10.2.74.116|:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 22,744,133 (22M) [application/zip]

    0K ........                                               2%   12.32 MB/s

15:49:35 (12.32 MB/s) - Connection closed at byte 565534. Retrying.

--15:49:35--  http://stage.mozilla.org/pub/mozilla.org/firefox/tinderbox-builds/mozilla-inbound-win32-debug/1319490284/firefox-10.0a1.en-US.win32.crashreporter-symbols.zip
  (try: 2) => `firefox-10.0a1.en-US.win32.crashreporter-symbols.zip'
Connecting to stage.mozilla.org|10.2.74.116|:80... connected.
HTTP request sent, awaiting response... Read error (Connection reset by peer) in headers.
Retrying.

--15:49:37--  http://stage.mozilla.org/pub/mozilla.org/firefox/tinderbox-builds/mozilla-inbound-win32-debug/1319490284/firefox-10.0a1.en-US.win32.crashreporter-symbols.zip
  (try: 3) => `firefox-10.0a1.en-US.win32.crashreporter-symbols.zip'
Connecting to stage.mozilla.org|10.2.74.116|:80... failed: Connection refused.
Resolving stage.mozilla.org... 10.2.74.116
Connecting to stage.mozilla.org|10.2.74.116|:80... failed: Connection refused.
program finished with exit code 1
elapsedTime=3.594000
======== Finished download failed (results: 2, elapsed: 3 secs) ========

Comment 1

[[e-mail] (:bhearsum)]

We use wget --tries=5 now. I think this is done.