Add manual New Relic exception reporting for errors during pushlog ingestion

RESOLVED FIXED

Status

Tree Management
Treeherder: Data Ingestion
P2
normal
RESOLVED FIXED
2 years ago
2 years ago

People

(Reporter: emorley, Assigned: emorley)

Tracking

Details

Attachments

(1 attachment)

(Assignee)

Description

2 years ago
Prior to bug 1280913, if there was an exception whilst ingestion one resultset, an aggregated CollectionNotLoaded exception would be generated on the celery task side (which wasn't that helpful), and also a specific exception for the web API side (which was useful).

Now we just get an aggregated CollectionNotStored exception, and no individual exceptions, since we intentionally want to ensure we still try to insert the remaining resultsets, even if one in the middle fails. (I forgot about this whilst reviewing bug 1280913 )

We can use the New Relic `.record_exception()` feature to manually record these individual exceptions to maintain visibility.
Created attachment 8764065 [details] [review]
[treeherder] mozilla:pushlog-record-exception > mozilla:master
(Assignee)

Updated

2 years ago
Attachment #8764065 - Flags: review?(james)
Attachment #8764065 - Flags: review?(james) → review+

Comment 2

2 years ago
Commit pushed to master at https://github.com/mozilla/treeherder

https://github.com/mozilla/treeherder/commit/e2486506632bf8a07c3ae145cf5b274e83cd5621
Bug 1281309 - Report individual pushlog exceptions to New Relic

Since the `CollectionNotStoredException` exception raised later will
contain a combination of all exceptions and is harder to follow in the
New Relic UI.
(Assignee)

Updated

2 years ago
Status: ASSIGNED → RESOLVED
Last Resolved: 2 years ago
Resolution: --- → FIXED
You need to log in before you can comment on or make changes to this bug.