1380450 - MissingPushException messages in New Relic are noise until we actually drop a job

Status: RESOLVED FIXED

(Tree Management :: Treeherder: Data Ingestion, enhancement)

Description

[Cameron Dawson [:camd]]

We should avoid reporting MissingPushExceptions when a job comes in from Pulse for which we have not yet ingested the push until max retries have been exceeded.  It is expected we will hit these retries from time to time (currently about every 10 minutes or so) but after trying once or twice, they get ingested and all is well.

If the retries are exceeded, then we will actually drop a job on the floor, and that's when the exception matters.  So let's keep the noise down until it's an actual problem.

Comment 1

[GitHub Autolander Bot]

Attachment: [treeherder] mozilla:missing-push-exception > mozilla:master

Comment 2

[Cameron Dawson [:camd]]

Comment on attachment 8885954 [details] [review]
[treeherder] mozilla:missing-push-exception > mozilla:master

Hey James and Ed-- Would you both take a quick look at this?  Thanks!!

Comment 3

[Ed Morley [:emorley]]

Comment on attachment 8885954 [details] [review]
[treeherder] mozilla:missing-push-exception > mozilla:master

Many thanks!

Comment 4

[Treeherder GitHub Bugbot]

Commit pushed to master at https://github.com/mozilla/treeherder

https://github.com/mozilla/treeherder/commit/eb6a8f904477562e7b181ba324b655ad9786a301
Bug 1380450 - Quiet the MissingPushException till retries exceeded (#2631)