1329992 - resultset status sometimes returns negatives and extra fields

Status: RESOLVED FIXED

(Tree Management :: Treeherder: API, defect)

Description

[Mark Côté [:mcote]]

Running this command several times

    curl 'https://treeherder.mozilla.org/api/project/try/resultset/157869/status/'

randomly returns two different answers:

    {"testfailed":4,"success":114,"busted":1}

and

    {"testfailed":-4,"success":-114,"busted":-1,"coalesced":119}

Interestingly, it seems that the "coalesced" field is only present when the other fields have negative values.

Comment 2

[Ed Morley [:emorley]]

Huh strange. I see that too:

$ curl https://treeherder.mozilla.org/api/project/try/resultset/157869/status/
{"success":114,"testfailed":4,"busted":1}

$ curl https://treeherder.mozilla.org/api/project/try/resultset/157869/status/
{"testfailed":-4,"success":-114,"busted":-1,"coalesced":119}

$ curl https://treeherder.mozilla.org/api/project/try/resultset/157869/status/
{"testfailed":-4,"success":-114,"busted":-1,"coalesced":119}

$ curl https://treeherder.mozilla.org/api/project/try/resultset/157869/status/
{"testfailed":4,"success":114,"busted":1}

That endpoint is here:
https://github.com/mozilla/treeherder/blob/78f66bc7c5f831acd675ef0f7224ac07f3ec8889/treeherder/webapp/api/resultset.py#L284-L294

Which uses get_status() from here:
https://github.com/mozilla/treeherder/blob/78f66bc7c5f831acd675ef0f7224ac07f3ec8889/treeherder/model/models.py#L143-L173

Judging by:
* the `coalesced` field sometimes being returned and sometimes not (which depends on `total_num_coalesced`, which depends on `num_coalesced)
* the other results sometimes being negative (which presumably comes from `num_coalesced` being greater than `total`)

...I'm presuming there's something up with:

    .annotate(
        total=Count('result'),
        num_coalesced=Count(Case(When(
            coalesced_to_guid__isnull=False, then=1)))

Comment 4

[GitHub Autolander Bot]

Attachment: [treeherder] wlach:1329992 > mozilla:master

Comment 5

[William Lachance (:wlach)]

Comment on attachment 8825906 [details] [review]
[treeherder] wlach:1329992 > mozilla:master

It seems like calling annotate with a set of keyvalues resulted in a non-deterministic ordering when we call `values_list`. I guess this isn't really surprising. Calling annotate() twice seems to fix the issue.

I'm not sure how to write a unit test to reproduce the non-deterministic behaviour, so I think we'll have to just rely on the fact that this (hopefully) doesn't break anything in the current set of tests as validation. :)

Comment 6

[Treeherder GitHub Bugbot]

Commit pushed to master at https://github.com/mozilla/treeherder

https://github.com/mozilla/treeherder/commit/624132cf73c917a6d98fc385145161e5d27698aa
Bug 1329992 - Fix resultset status endpoint (#2079)

The way we were using Django's "annotate" function with values_list
resulted in non-deterministic output.

Comment 7

[William Lachance (:wlach)]

(fix should be deployed to production soon)