Closed Bug 1319583 Opened 3 years ago Closed 3 years ago

No taskcluster retry on Android INFRA-ERROR


(Testing :: General, defect)

Version 3
Not set


(firefox53 fixed)

Tracking Status
firefox53 --- fixed


(Reporter: gbrown, Assigned: gbrown)



(1 file)

Even though "INFRA-ERROR" is included in the retry regex:

such failures do not appear to trigger a retry.

[task 2016-11-22T22:21:29.192215Z] 22:21:29    FATAL - INFRA-ERROR: Unable to start emulator after 5 attempts
[task 2016-11-22T22:21:29.192635Z] 22:21:29    FATAL - Running post_fatal callback...
[task 2016-11-22T22:21:29.193006Z] 22:21:29    FATAL - Exiting -1
[task 2016-11-22T22:21:29.757442Z] cleanup
[task 2016-11-22T22:21:29.757495Z] + cleanup
[task 2016-11-22T22:21:29.757511Z] + local rv=255
[task 2016-11-22T22:21:29.757528Z] + [[ -s /home/worker/.xsession-errors ]]
[task 2016-11-22T22:21:29.757974Z] + cp /home/worker/.xsession-errors /home/worker/artifacts/public/xsession-errors.log
[task 2016-11-22T22:21:29.764425Z] + cleanup_xvfb
[task 2016-11-22T22:21:29.764650Z] pidof Xvfb
[task 2016-11-22T22:21:29.764834Z] ++ pidof Xvfb
[task 2016-11-22T22:21:29.768987Z] + local xvfb_pid=33
[task 2016-11-22T22:21:29.769025Z] + local vnc=false
[task 2016-11-22T22:21:29.774378Z] + local interactive=false
[task 2016-11-22T22:21:29.774457Z] + '[' -n 33 ']'
[task 2016-11-22T22:21:29.775133Z] + [[ false == false ]]
[task 2016-11-22T22:21:29.775391Z] + [[ false == false ]]
[task 2016-11-22T22:21:29.775653Z] + kill 33
[task 2016-11-22T22:21:29.775688Z] + screen -XS xvfb quit
[task 2016-11-22T22:21:29.833340Z] No screen session found.
[task 2016-11-22T22:21:29.833753Z] + true
[task 2016-11-22T22:21:29.833806Z] + exit 255
[taskcluster 2016-11-22 22:21:31.563Z] === Task Finished ===
[taskcluster 2016-11-22 22:21:43.614Z] Unsuccessful task run with exit code: 255 completed in 1802.627 seconds
self.fatal() takes an optional exit_code parameter; if not specified (as here), it exits with the default, -1. Regardless of exit code specification, fatal() exits without any of the log parsing and exit code setting that would normally recognize "INFRA-ERROR" and trigger a TBPL_RETRY exit.
Summary: No taskcluster retry on Android x86 INFRA-ERROR → No taskcluster retry on Android INFRA-ERROR
Buildbot triggers retries based strictly on log parsing, so just logging "INFRA-ERROR" on buildbot successfully retries. 

Taskcluster triggers retries based strictly on exit code, which is set to -1 by default when exiting via fatal().

This patch ensures the exit code is set to TBPL_RETRY when an INFRA-ERROR causes an exit via fatal(). demonstrates retries with a forced INFRA-ERROR.
Attachment #8813922 - Flags: review?(kmoir)
Attachment #8813922 - Flags: review?(kmoir) → review+
Pushed by
Trigger job retry on Android infra error; r=kmoir
Closed: 3 years ago
Resolution: --- → FIXED
Target Milestone: --- → mozilla53
You need to log in before you can comment on or make changes to this bug.