1281309 - Add manual New Relic exception reporting for errors during pushlog ingestion

Status: RESOLVED FIXED

(Tree Management :: Treeherder: Data Ingestion, defect, P2)

Description

[Ed Morley [:emorley]]

Prior to bug 1280913, if there was an exception whilst ingestion one resultset, an aggregated CollectionNotLoaded exception would be generated on the celery task side (which wasn't that helpful), and also a specific exception for the web API side (which was useful).

Now we just get an aggregated CollectionNotStored exception, and no individual exceptions, since we intentionally want to ensure we still try to insert the remaining resultsets, even if one in the middle fails. (I forgot about this whilst reviewing bug 1280913 )

We can use the New Relic `.record_exception()` feature to manually record these individual exceptions to maintain visibility.

Comment 1

[GitHub Autolander Bot]

Attachment: [treeherder] mozilla:pushlog-record-exception > mozilla:master

Comment 2

[Treeherder GitHub Bugbot]

Commit pushed to master at https://github.com/mozilla/treeherder

https://github.com/mozilla/treeherder/commit/e2486506632bf8a07c3ae145cf5b274e83cd5621
Bug 1281309 - Report individual pushlog exceptions to New Relic

Since the `CollectionNotStoredException` exception raised later will
contain a combination of all exceptions and is harder to follow in the
New Relic UI.