Closed Bug 1277506 Opened 6 years ago Closed 6 years ago

store-error-lines should not retry/raise for HTTP 403 and 404 responses


(Tree Management :: Treeherder: Data Ingestion, defect, P2)



(Not tracked)



(Reporter: emorley, Assigned: jgraham)


(Blocks 1 open bug)



(1 file)

For taskcluster log URLs, it's unfortunately expected that they can sometimes 404 or 403 - see bug 1154248.

As such, the log parser special-cases these, so:
(a) it doesn't keep retrying them (if the log is 404 it's not going to be there on the next retry)
(b) it doesn't raise, reducing noise on New Relic

See bug 1155647, bug 1159934.

However store-error-lines doesn't do this, so is currently generating a lot of retries for 403 cases:

See the log parser implementation:

Note also that it avoids the retry (but still raises) for non-HTTPError exceptions (eg a parsing error), since we still want the New Relic entry, but there's no point continually trying to parse the same log content, if it causes an exception. 

Perhaps we should use a common log downloader function between the log parser and store-error-lines? It seems like store-error-lines should also be streaming the content, like the log parser does? (Note the log parser currently uses urllib2, but bug 1165356 is wanting to switch it to requests.)
Flags: needinfo?(james)
Flags: needinfo?(james)
Attachment #8759200 - Flags: review?(emorley)
Attachment #8759200 - Flags: review?(emorley) → review+
Assignee: nobody → james
Commits pushed to master at
Bug 1277506 - Don't raise an error when storing failure lines generates 403/404.

This matches the behaviour of fetching the full logs and represents a
belief that 404 errors will not magically correct themselves when we
Merge pull request #1554 from mozilla/store_error_summary_ignore_404

Bug 1277506 - Don't raise an error when storing failure lines generates 403/404.
Closed: 6 years ago
Resolution: --- → FIXED
You need to log in before you can comment on or make changes to this bug.