1173882 - Autophone - should use updated attempts when inserting a new job due to a retry

Status: RESOLVED FIXED

(Testing Graveyard :: Autophone, defect)

Description

[Bob Clary [:bc] (inactive)]

Last night I noticed a job which continually submitted retries:

https://treeherder.mozilla.org/#/jobs?repo=try&exclusion_profile=false&filter-searchStr=autophone&revision=6e083a56ff88

I saw this again locally when testing --override-build-dir where incompatible api builds were being served to builds. The job would be retried and the newly inserted job would have attempts initialized to 0 which meant that retried jobs would not be purged after exceeding the maximum number of attempts.

see https://github.com/mozilla/autophone/blob/master/worker.py#L629

The fix is to change Jobs.new_job to take an attempts keyword argument with default 0 and to use that to increment the attempts for retried jobs.

Comment 1

[Bob Clary [:bc] (inactive)]

Attachment: bug-1173882-update-retry-attempts.patch

Comment 2

[Bob Clary [:bc] (inactive)]

https://github.com/mozilla/autophone/commit/9170a302a14593156f4936bf018cbf1be4fcde43

Comment 3

[Bob Clary [:bc] (inactive)]

I shouldn't have incremented the attempts in the new retried job. Just passing the existing attempts would have been correct.

Comment 4

[Bob Clary [:bc] (inactive)]

Attachment: bug-1173882-followup.patch

Comment 5

[Geoff Brown [:gbrown]]

Comment on attachment 8621729 [details] [diff] [review]
bug-1173882-followup.patch

Review of attachment 8621729 [details] [diff] [review]:
-----------------------------------------------------------------

Sorry, I should have flagged that.

Comment 6

[Bob Clary [:bc] (inactive)]

https://github.com/mozilla/autophone/commit/b0ce085535ffefe0edd7bc3f9e5c93eea86051d9

Comment 7

[Bob Clary [:bc] (inactive)]

fwiw, this was a regression from bug 1165403. Prior to that check in, the new job generated from the retry would be attached to the existing row in the jobs table. After the check in, a new row would be generated in the jobs table for the build if duplicates were allowed. This new row would have had attempts 0 initially which was the cause of our problem.