1163496 - Unclassified job count is too high, due to jobs incorrectly being marked as coalesced

Status: RESOLVED FIXED

(Tree Management :: Treeherder: Data Ingestion, defect, P1)

Description

[Carsten Book [:Tomcat]]

like shows 16 unclassified while the real number is 3 and so on across all trees :(

Comment 1

[Ed Morley [:emorley]]

More details needed please.
Which repos, specific links, what filters enabled, does it occur straight away, or after N minutes etc etc.

https://developer.mozilla.org/en-US/docs/Mozilla/QA/Bug_writing_guidelines#str

Comment 2

[Carsten Book [:Tomcat]]

Attachment: Screen Shot 2015-05-11 at 9.08.18.png

see screenshot.

repos as example: mozilla-central shows in treeherder 9 unclassifieds - in reality thats 5 

occurs from loading this repro - also reloading etc does not help

Comment 3

[Ed Morley [:emorley]]

After adding some logging to the UI locally, I found an example of one of the jobs the UI thought was unclassified:

https://treeherder.mozilla.org/api/project/mozilla-central/jobs/1473111/
{
...
  "result": "busted",
  "id": 1473111,
  "job_guid": "7c10d61c7b642e75b50a16c5afa92c1ce9a0038b",
  "ref_data_name": "Ubuntu VM 12.04 mozilla-central debug test mochitest-devtools-chrome-1",
  "state": "completed",
  "failure_classification_id": 1,
  "job_coalesced_to_guid": "7c10d61c7b642e75b50a16c5afa92c1ce9a0038b",
...
}

Note how the job_coalesced_to_guid is the same as job_guid - ie: the job is trying to say it got coalesced into itself.

This causes the UI to mark the job as yellow and exclude it from the views, but yet the "calculate the unclassified count" code doesn't use that, and instead goes off of the "result" - which as you can see above is "busted".

Anyway, the job_coalesced_to_guid being identical to the job_guid is a bug - and I've fixed it with:
https://github.com/mozilla/treeherder/commit/54727c2994d9f937bd6297dadd000580bfcfc891

This issue would have been much easier to spot if we used a job result (or state) of "coalesced", rather than only looking at job_coalesced_to_guid. However we already have a bug on file for that, bug 1132546. I'll also file a bug for asserting in the service that the job_guid != job_coalesced_to_guid.

Comment 4

[Treeherder GitHub Bugbot]

Commit pushed to master at https://github.com/mozilla/treeherder-ui

https://github.com/mozilla/treeherder-ui/commit/c97a4d693939e7f31ac7d6d540c59ef88e06ee60
Bug 1163496 - Handle invalid job_coalesced_to_guid values

If job_coalesced_to_guid matches the guid of the job in question, ignore
it, since the value is invalid.

Comment 5

[Ed Morley [:emorley]]

Deployed to production, refresh the UI to see the unclassified jobs that were previously hiding as coalesced.

Comment 6

[Treeherder GitHub Bugbot]

Commit pushed to master at https://github.com/mozilla/treeherder-ui

https://github.com/mozilla/treeherder-ui/commit/1d5aea1ced9721fe69470afb0af734bf8cb0cf02
Bug 1163496 - Don't check if completed jobs are coalesced

With this change, we return early with the job result, if the job was
completed. This not only optimises for the common case, but also means
we avoid issues when job_coalesced_to_guid has been accidentally set for
completed jobs (eg due to the initial landing of bug 1093743, which has
since been fixed, but some old jobs are still in the DB).

Comment 7

[Treeherder GitHub Bugbot]

Commits pushed to master at https://github.com/mozilla/treeherder

https://github.com/mozilla/treeherder/commit/c42dae048cb2530ce7c0db349258f3035acc8c4e
Bug 1163496 - Handle invalid job_coalesced_to_guid values

If job_coalesced_to_guid matches the guid of the job in question, ignore
it, since the value is invalid.

https://github.com/mozilla/treeherder/commit/1d65b645f582c852b60035943e82d550c691c530
Bug 1163496 - Don't check if completed jobs are coalesced

With this change, we return early with the job result, if the job was
completed. This not only optimises for the common case, but also means
we avoid issues when job_coalesced_to_guid has been accidentally set for
completed jobs (eg due to the initial landing of bug 1093743, which has
since been fixed, but some old jobs are still in the DB).