1113322 - Implement Multiple-Tier Job support

Status: RESOLVED FIXED

(Tree Management :: Treeherder, defect, P4)

Description

[Jonathan French (:jfrench)]

Apologies, I searched for this feature work discussed in the mtg today, but I couldn't find it in our open bugs.

I did some back and forth with camd in channel on some possible UI solutions for representing a tier(n) intermixed job row which could conceivably support any number of tiers, in addition to a tier2. I'll attach a couple of screen grabs to spark ideas.

Comment 1

[Jonathan French (:jfrench)]

Attachment: tierN_row_rev1

Possible overstrike treatment, to denote a tier(n) job in a job row.

Comment 2

[Jonathan French (:jfrench)]

Attachment: tierN_hoverSelectState_rev1

Same concept above, showing additional detail on hover/select to denote its tier. A tier 2 job is shown in the example.

Comment 3

[Jonathan French (:jfrench)]

As Ryan just pointed out in channel, we may utilize different visibility profiles and so we may not require different job representations, per above. It's all tbd at the moment.

Comment 4

[Ed Morley [:emorley]]

There's bug 1080731 filed already, however that's more the 'step 2' part that we talked about yesterday - this bug can be simply about making tier 2 look visually different (and excluding them from failure counts), whereas bug 1080731 can be about being more flexible on that front, and not excluding them from failure counts etc, but showing them initially, with the sheriffs then suppressing the failures for N days.

Comment 5

[Mauro Doglio [:mdoglio]]

I created an etherpad to track the requirements ob this bug:
https://etherpad.mozilla.org/Mpd2vI1Cjl

Comment 6

[Mauro Doglio [:mdoglio]]

This bug is about the creation of public visibility profiles, one for each job class (aka "tier"). 

Every job class will have a corresponding public profile that the user can select and make their default choice.

If the user hasn't selected yet a job class, a 'sheriffable jobs' class will be selected by default.

Comment 7

[Jonathan French (:jfrench)]

It looks like there's a 2nd etherpad going on this in addition to Mauro's, so including it below, pending consolidation if needed.
https://etherpad.mozilla.org/tier2-ui-sheriff-requirements
https://etherpad.mozilla.org/Mpd2vI1Cjl (mauro's above)

Comment 8

[Jonathan French (:jfrench)]

Attachment: mockupTierTwoStrikeDesat_rev1

There was discussion in this week's meeting about possibly using a strike to denote a tier two job, so I've included an enlarged mockup with both types of jobs. ie. ones with a fill and border like testfailed, and ones without like retry/restart success, etc.

A strike might also be lightweight for load performance, rather than adding more elements to every job to differentiate it.

The mockup uses a grey which is a shade darker than pending (85% border/text, 92% fill), so the only scenario where we might have ambiguity is a pending success/retry and a same-name tier two job side by side. Where the only differentiation would be the strike, and their hover tooltips.

Some of its quick recognition may also be dependent on where the strike passes through the letter.

Comment 9

[Jonathan French (:jfrench)]

Attachment: mockupTierTwoStrikeDesat_rev1a

Minor tweaks to the wording in the mockup for clarity. No other changes.

Comment 10

[Cameron Dawson [:camd]]

Just discussed with Mark Cote:

Latest minimum viable product is:
* distinguish in data tier of the jobs
* showing tier 2 results in a distinguishing way
* support being hidden with exclusion profiles

possible
* be able to hide them (use filtering for this?)

Comment 11

[Ed Morley [:emorley]]

Please can we de-prioritise this? We have quite a more more important reliability/infra/regression type bugs that IMO we should tackle first.

Comment 12

[Jonathan Griffin (:jgriffin)]

There are several use cases for this that we want to support; I'd like to spend the smallest amount of time here possible (hopefully, <1 week), and then move on.  Further enhancements can wait until post Q2.  If there are higher priority reliability issues, we can defer this minimal work until Q2.

Comment 13

[Cameron Dawson [:camd]]

Latest minimum viable product is:

    * distinguish in data tier of the jobs
    * add a column to the job table for the tier info

        * ask sheeri to alter tables on dev/stage/prod.  
        * I update .sql file in repo

    * check if we have the tier info from the data provider; if it's available use that
    * if we don't have the tier info attached to the incoming job:
    
        * if we have any exclusion profile named "tier-N", fetch its signatures and use those signatures to determine the tier info
        * phase 2 (in addition to phase 1)
            * specify tier in the reference_data_signatures table
            * need editor for that table to make it easier (like the exclusion UI)

    * showing tier 2 results in a distinguishing way
    * support being hidden with exclusion profiles
    * support filtering in the UI by tier (same as any field filter)
    * only use tier1 jobs for unclassified failures

Comment 14

[Cameron Dawson [:camd]]

I'm having doubts about using an exclusion profile to specify which jobs are lower tiers.  If we limit this to only tier-1 and tier-2, then it's ok. But if there are multiple tiers, then we'd need a profile for each tier.  

On data ingestion, this would mean loading in all those profiles and checking each one to see if the current job signature is included.

If we go with setting the tier value in the reference_data_signatures table, we don't have that problem.  But editing that table is now only possible through the django admin.  So you'd have to set jobs as tier-N one by one, which is a pain.

wanted to voice my concern here so I can think more about it when I get back from PTO.

I think I will need to ONLY implement tier-1 and tier-2 at this time.  We can handle more tiers later.

Comment 15

[Jonathan Griffin (:jgriffin)]

I'm for whatever is easiest here.  Do we even need to support this differently with exclusion profiles?  Can we just let these be hidden using the existing profile as appropriate?

Comment 16

[Cameron Dawson [:camd]]

Ok, we're only having tier-1 and tier-2.  So the exclusion profile technique is the quickest and easiest.  Going with that.

Comment 17

[Cameron Dawson [:camd]]

Attachment: tier 2 jobs screenshot

This shows Tier-2 jobs in a separate group as well as having a "halo" border.

Comment 18

[Cameron Dawson [:camd]]

Attachment: Tier 2 service PR

Comment 19

[Cameron Dawson [:camd]]

Attachment: tier 2 ui PR

Comment 20

[Jonathan French (:jfrench)]

(In reply to Cameron Dawson [:camd] from comment #17)
> Created attachment 8584119 [details]
> tier 2 jobs screenshot
> 
> This shows Tier-2 jobs in a separate group as well as having a "halo" border.

Looks great. I wonder if the vertical dashes are even needed. In any case the bonus is no new Help additions are required, as it's just another job group.

Comment 21

[Cameron Dawson [:camd]]

Added bug 1148674 which explores possible extensions to this work.

Comment 22

[Treeherder GitHub Bugbot]

Commit pushed to master at https://github.com/mozilla/treeherder-service

https://github.com/mozilla/treeherder-service/commit/f3ee2a81facdb1b02653dc160f57ff6818e125d7
Bug 1113322 - add support for tier-2 jobs

Comment 23

[Treeherder GitHub Bugbot]

Commit pushed to master at https://github.com/mozilla/treeherder-ui

https://github.com/mozilla/treeherder-ui/commit/5c395ce663b3325ee150b6a9e5e7ecb22fd8c853
Bug 1113322 - tier 2 jobs visual representation

Comment 24

[Cameron Dawson [:camd]]

Merged and pushed to dev.

Comment 25

[Treeherder GitHub Bugbot]

Commit pushed to master at https://github.com/mozilla/treeherder

https://github.com/mozilla/treeherder/commit/96f3d92b04746e58f8cfa96c57ea83d476716072
Bug 1113322 - tier 2 jobs visual representation