Closed Bug 1062827 Opened 10 years ago Closed 1 year ago

Change names used for job results (eg "busted", "testfailed") to be closer to those used by TaskCluster than buildbot

Categories

(Tree Management :: Treeherder: Data Ingestion, defect, P3)

defect

Tracking

(Not tracked)

RESOLVED WONTFIX

People

(Reporter: emorley, Unassigned)

References

Details

Attachments

(1 obsolete file)

Currently treeherder uses the same names to describe the result states as TBPL did:

https://github.com/mozilla/treeherder-service/blob/bf60d1908e85f925cdf7a7a9b5bdc52082e64bab/treeherder/etl/buildbot.py#L3
RESULT_DICT = {
    0: "success",
    1: "testfailed",
    2: "busted",
    3: "skipped",
    4: "exception",
    5: "retry",
    6: "usercancel"
}

These names seem to be somewhat arbitrary, and do not even match what buildbot calls them:

http://hg.mozilla.org/build/buildbot/file/4b5a85b140b3/master/buildbot/status/builder.py#l25
    25 SUCCESS, WARNINGS, FAILURE, SKIPPED, EXCEPTION, RETRY, CANCELLED = range(7)
    26 Results = ["success", "warnings", "failure", "skipped", "exception",
    27            "retry", "cancelled"]

Given they seem to have been arbitrarily picked & that names like "testfailed" don't really describe build job types well + "busted" is just not very descriptive - I think we should rename them to something like:

success  ->  <no change> (or "pass"?)
testfailed  ->  warnings
busted  ->  failed
skipped  ->  <no change>
exception  ->  infra-failure
retry  ->  <no change>
usercancel  ->  <no change>

This will require quite a few changes throughout the service but particularly the UI, so will be a bit of a pain - but I think it will be best if we change now, rather than end up being stuck with these names X years down the line when we're not using buildbot and our infra has changed.
Checking with Wes and Ryan to confirm they're happy also with the name-mapping changes above? eg. we might have enough room for the full, 'infrastructure-failure' as a status, dunno.

I'll have a look at treeherder-ui to start.
Flags: needinfo?(ryanvm)
Flags: needinfo?(kwierso)
fine with me
Flags: needinfo?(ryanvm)
I've pushed a wip change to my own remote, 'update-job-result-names' for the UI. I'll hold off on opening a PR for now, pending feedback from Wes also.
Fine with me as well.
Flags: needinfo?(kwierso)
Attached file treeherder-ui-PR#248 (obsolete) —
Opened preliminary PR for the UI side.
See Also: → 1057341
Priority: P4 → P5
Given this change hasn't bubbled up in channel as a high priority since I opened the -ui PR 5 months ago, I'd like to close my PR for now, if that's ok Ed? We can always refer to it, and assuming the code is still relevant, re-open the PR when we get to the bug.

I figure that way it doesn't clutter the Open PR list.
Flags: needinfo?(emorley)
Yeah sounds sensible for now :-)
Flags: needinfo?(emorley)
Cool, done.
See Also: 1057341
Attachment #8511244 - Attachment is obsolete: true
This will be easier after bug 1395254.
Depends on: 1395254
Priority: P5 → P3
Summary: Change names for job results (eg "busted", "testfailed") → Change names used for job results (eg "busted", "testfailed") to be closer to those used by TaskCluster than buildbot
There may also be extra states from TaskCluster that we currently lump into one, that we could instead separate out for improved UX.
See Also: → 1066272
Component: Treeherder → Treeherder: Data Ingestion

cleaning up old bugs

Status: NEW → RESOLVED
Closed: 1 year ago
Resolution: --- → WONTFIX
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Created:
Updated:
Size: