1155647 - Don't generate log parser exceptions if the log URL 404s

Status: RESOLVED FIXED

(Tree Management :: Treeherder: Data Ingestion, defect, P2)

Description

[Ed Morley [:emorley]]

In bug 1154248, we're seeing many log URL 404s during log parsing, since for multiple reasons, the logs were never uploaded. This is unfortunate, but not something that is Treeherder's fault, so we should just quietly mark the parse_status as failed and not generate an exception.

I'm on the fence as to whether we should retry a 404 or not (eg in case the log hadn't quite finished uploading), however the way the celery retries work makes it more of a pain to filter out the exceptions in those cases, so I say for now we should just not retry at all, in the case of 404s.

Comment 1

[Ed Morley [:emorley]]

Attachment: Don't generate log parser exceptions if the log URL 404s

Comment 2

[Cameron Dawson [:camd]]

Comment on attachment 8593955 [details] [review]
Don't generate log parser exceptions if the log URL 404s

looks good to me.  :)

Comment 3

[Treeherder GitHub Bugbot]

Commits pushed to master at https://github.com/mozilla/treeherder

https://github.com/mozilla/treeherder/commit/11be596d602bb6c2f75c1e44b13acd3c17390b9c
Bug 1155647 - Refactor out the POST to update_parse_status

https://github.com/mozilla/treeherder/commit/25ffbd9190a11f798a9d6e3d6fb8edf8c2efb032
Bug 1155647 - Don't generate log parser exceptions if the log URL 404s

If attempting to fetch the log from the provided URL results in a 404,
mark the parse_status as failed but do not generate an exception. Whilst
missing logs are not ideal, they are not an app error, nor something is
likely to change over time (vs say an HTTP 500 from ftp.mozilla.org).