1457492 - FailuresByBug and FailureCount use differing criteria to determine orange count

Status: RESOLVED FIXED

(Tree Management :: Treeherder, enhancement, P1)

Description

[Ed Morley [:emorley]]

In bug 1453759 it was noticed that the Intermittent counts in the Treeherder IFV UI were different from what the Treeherder /api/failurecount/ API was returning, resulting in:
https://github.com/mozilla-bteam/bmo/pull/559#issuecomment-384804970

I think this is happening because:
* The intermittent failures view UI uses the stats from /api/failuresbybug/ to determine the overall count in the UI
* Those stats come from here:
  https://github.com/mozilla/treeherder/blob/6fe73c39c6c86ace5293d7e4a6cee32658a006a2/treeherder/webapp/api/intermittents_view.py#L37-L77
* Whereas the /api/failurecount/ (which is the right one for them to be using for their graph - and is presumably what the IFV uses for it's own graph too) comes from here:
  https://github.com/mozilla/treeherder/blob/6fe73c39c6c86ace5293d7e4a6cee32658a006a2/treeherder/webapp/api/intermittents_view.py#L80-L125
* ...and the latter has an additional filter on `job__failure_classification__id=4`

Having a different filter on each endpoint seems unintentional? Or if it's required, we should add a code comment explaining why.

We should probably also refactor intermittents_view.py to reduce duplication, which would help avoid deviations between filter criteria in the future.

Comment 1

[Sarah Clements [:sclements]]

The `job__failure_classification__id=4` on the query for failurecount (for bug_id) is indeed a mistake; removing it shows the correct results (with minor variations in a few days' counts due to IFV using UTC, not localizing aggregations). I'll refactor and submit a patch.

Comment 2

[Sarah Clements [:sclements]]

Attachment: patch

Comment 3

[Ed Morley [:emorley]]

Comment on attachment 8972066 [details] [review]
patch

George, don't suppose you could take a look at this? You're probably more familiar with Django model managers than I am at this point (has been a while).

Comment 4

[George Hickman]

This looks great, it's nice having those longer queries wrapped up in a manager too.

Comment 5

[Treeherder GitHub Bugbot]

Commit pushed to master at https://github.com/mozilla/treeherder

https://github.com/mozilla/treeherder/commit/adf9a2083b666cfa96d0d676a9c7654c1ead4c54
Bug 1457492 - Fix FailureCount endpoint (#3495)

* remove `job__failure_classification__id=4` in `FailureCount` & `Failures` endpoint
* create `QuerySet` to remove duplicate code for filters