1434166 - autoland's pingbacks to reviewboard are stuck in a 404 loop

Status: RESOLVED FIXED

(Conduit Graveyard :: Transplant, defect)

Description

[:glob ✱]

> trying to post mozreview update to: https://reviewboard.mozilla.org/api/extensions/mozreview.extension.MozReviewExtension/autoland-request-updates/ for request: 45441
> Starting new HTTPS connection (1): reviewboard.mozilla.org
> "POST /api/extensions/mozreview.extension.MozReviewExtension/autoland-request-updates/ HTTP/1.1" 404 70
> failed: 404 - {"stat": "fail", "err": {"msg": "Object does not exist", "code": 100}}

i will investigate why this pingback is 404'ing

currently autoland stops processing the list of pending pingbacks at the first failure, and it treats 404 as a transient failure.  the end result is this 404 is being retried infinitely.  autoland should treat a 404 as a fatal error for that pingback, and move on to the next request.

Comment 1

[:glob ✱]

here's the request:
> {"landed": true, "trysyntax": "try: -b do -p all -u all -t none", "result": "af6a04127bfb", "request_id": 45441, "rev": "5105ffc915ea5dfc7ec55479e9d99e34b9217ff7", "destination": "try", "tree": "gecko", "error_msg": ""}

it's missing from rb's mozreview_autolandrequest table, triggering the 404.  i don't see anything in the logs that indicate why this row is missing.

Comment 2

[:glob ✱]

i've removed the row from autoland's MozreviewUpdate table, and the large pile of pingbacks stuck behind it were all successfully sent.  patch to rework 404 handling soon.

Comment 3

[:glob ✱]

Attachment: autoland: treat a 404 from a pingback as a fatal error (bug 1434166)

A 404 from a pingback means the other system no longer has the record for the
autoland request; treat it as a fatal error instead of entering into an
infinite loop of transient retries.

Review commit: https://reviewboard.mozilla.org/r/216562/diff/#index_header
See other reviews: https://reviewboard.mozilla.org/r/216562/

Comment 4

[Gregory Szorc [:gps]]

Comment on attachment 8946544 [details]
autoland: treat a 404 from a pingback as a fatal error (bug 1434166)

https://reviewboard.mozilla.org/r/216562/#review222248

Comment 5

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/hgcustom/version-control-tools/rev/6f75d61627ad
autoland: treat a 404 from a pingback as a fatal error r=gps

Comment 6

[:glob ✱]

deployed to prod.

Comment 7

[Steven MacLeod [:smacleod]]

This same thing came up 4 months ago in Bug 1405126. No idea how these are failing to be saved in MozReview's DB...