1045606 - Make the global filters more discoverable & intuitive

Status: RESOLVED FIXED

(Tree Management :: Treeherder, defect, P4)

Description

[:Paolo Amadini]

The state of the global "unclassified" filter should be reflected by the visual state of the button in the toolbar.

At this point, when the filter is activated, only unclassified jobs should be shown. Pending, running, and success jobs should be considered "classified" for the purposes of this filter.

The jobs shown should be the subset of the jobs that are selected by the current per-row filters that do not match the "classified" definition. This means that success, running, and pending jobs are never shown, and other job types are shown in each row based on the active filters.

Comment 1

[Ed Morley [:emorley]]

I'm sorry I'm not sure I follow the difference in the comment 0 behaviour from the already existing behaviour? By 'global "unclassified" filter' do you mean the checkbox inside the filters menu?

Comment 2

[:Paolo Amadini]

(In reply to Ed Morley [:edmorley] from comment #1)
> I'm sorry I'm not sure I follow the difference in the comment 0 behaviour
> from the already existing behaviour? By 'global "unclassified" filter' do
> you mean the checkbox inside the filters menu?

I've observed yesterday with Mauro that, while fiddling with the per-row filters, some classified jobs (for example those in success state) could re-appear. I've used the button in the global toolbar to filter first.

I believe the toggle behavior should be consistent, that is clicking two times on any filter toggle button should give me the original set. And, as long as the general filter in the toolbar is active, I should only be able to view "unclassified" jobs. If I want to see more jobs, I should remove the limiting filter first.

Comment 3

[Ed Morley [:emorley]]

Yeah that makes sense :-)

Comment 4

[Cameron Dawson [:camd]]

The intention of those toggles was to be an "override" of filtering.  No matter what state the global filters were in, you could "peek" into the success, retry, running, etc, statuses.  Then any change to filters at the global level will wipe out the per-resultset filter overrides.  

Just to be sure I understand this bug, here's a scenario:

1. global filters are set to show only "busted" and "testfailed".  
2. on a resultset status line, you click the "success" count and it will be a no-op.  
3. click the "testfailed" count and it will toggle them on and off.

Do I have that right?

Comment 5

[:Paolo Amadini]

(In reply to Cameron Dawson [:camd] from comment #4)
> The intention of those toggles was to be an "override" of filtering.  No
> matter what state the global filters were in, you could "peek" into the
> success, retry, running, etc, statuses.  Then any change to filters at the
> global level will wipe out the per-resultset filter overrides.  

OK, this looks like a potentially useful functionality as explained, but I think there are two core issues in how it is currently presented:

1) The local filters are more prominent than the global ones. Not only, the global filters are actually hidden by default. This means that I'll mainly use the local filters rather than the global ones at first, and then try to refine using the button in the toolbar.

2) The "unclassified" button is a preselection of the hidden states of global filters. The reason why its behavior seems mysterious is that it toggles several checkboxes in the global filters, in different categories. Once you open the filter panel, the behavior of the button can be better understood.

Turning the filtering panel into an always-visible toolbar (or initially visible, collapsible toolbar) would solve the first issue.

For the second issue, I believe there is a core problem with classification filtering: "classified" and "unclassified" are seen as two complementary sets, but this is not what is really needed. (Hence the need for a workaround through a button that toggles multiple states.)

There should be two different, restrictive, task-oriented global filters instead:
 * "To classify": Shows only jobs that are of the "failed" type and don't have the star.
 * "Classified": Shows only jobs that have the star (no filtering on type).

For clarity, I think at any time these filters should be either both deselected, or one of them should be selected. Having both selected is not particularly useful. Clicking "classified" mode should disable "to classify" mode.

Toggling the classification filters should not change the state of the other filters.

All other global filters are still applied in AND. Fore example, when "to classify" is selected, the "failed" category filters and the field filters will be relevant. Global filters for non-failure categories will obviously be irrelevant, and should probably appear dimmed in the UI.

Comment 6

[Cameron Dawson [:camd]]

Attachment: filter-bar PR

Comment 7

[Ed Morley [:emorley]]

Comment on attachment 8564304 [details] [review]
filter-bar PR

Thank you for digging into this - making the global filters more discoverable is something that we really need :-)

Visually the look of the buttons in compact mode look awesome - and the addition of a compact mode is great!

Where I think we may need to tweak things is that the non-compact mode (and even the compact mode, but to a lesser extent) still looks very busy.

I'm wondering if we need to show all of {testfailed,busted,exception,usercancel} (ie the ones that require classification) separately - and instead could lump them under 'failed'. Similarly, for pending and running, could we combine the toggle for them under 'in-progress' (or 'incomplete').

This could cut down the current list (9 states):
testfailed,busted,exception,success,retry,usercancel,coalesced,pending,running

To 5 states:
success,failed,retry,coalesced,in-progress

Also, as earlier comments in this bug have touched on, I think we need to consider the interaction between the global result filters and the classified/needs classifying filters. On the one hand they are completely separate, but on the other, it's common to want to toggle many of the filters on/off at once (ie pressing 'u' flips lots of filter states) and so making unclassified vs classified a separate layer of filtering would affect that workflow.

Perhaps one of the ways the options could be split up is as follows (not necessarily this order of options though, and I'm only displaying vertically to make them easier to read):
{
[x] success
[x] failed
[x] retry
[ ] coalesced
[x] in-progress
--
[ ] hide failed once classified
}

It's potentially clearer than the current filter menu options, with it's "classified" and "unclassified" checkboxes (which don't make it clear that , albeit limits the permutations (though perhaps not a bad thing).

Anyway, the more I think about this, the more I believe we need to take a step back and figure out what the common workflows are here to know how to best split these up. 

Some questions we need to answer:
1) Would someone ever want to view _just_ the failures that have been classified and not the ones that are unclassified? It's possible with the current treeherder UI, but is perhaps an unnecessary complexity.
2) What are common configurations of the prefs, so we can have some presets with shortcuts? eg failed+in-progress+"hide failed once classified"
3) ...probably some more, but it's late and my brain's fried.

Anyway, we've got a great look/design in the PR - once we've got a firm grasp of what would be the most useful for sheriffs we can make the UX for this awesome :-)

Comment 8

[Ed Morley [:emorley]]

Tweaking summary to reflect the work being done, since it's changed a bit since comment 0 (the main issue there was fixed by the removal of the per push counts/toggles), but there's still some discoverability & strange interaction of options issues with the global filters that we can fix here :-)

Comment 10

[Cameron Dawson [:camd]]

Attachment: filter PR

We mentioned this in the work-week and I thought I'd simplify and resurrect this work.  It seems like it'd be worthwhile for folks.

Comment 11

[Cameron Dawson [:camd]]

(In reply to Ed Morley [:emorley] from comment #7)
> Comment on attachment 8564304 [details] [review]
> filter-bar PR
....
> 
> Some questions we need to answer:
> 1) Would someone ever want to view _just_ the failures that have been
> classified and not the ones that are unclassified? It's possible with the
> current treeherder UI, but is perhaps an unnecessary complexity.
> 2) What are common configurations of the prefs, so we can have some presets
> with shortcuts? eg failed+in-progress+"hide failed once classified"
> 3) ...probably some more, but it's late and my brain's fried.
> 
1. I'd talked to Wes, and he actually DOES do this.  I think to double-check that classifications were accurately done.
2. I think that one you mentioned is probably pretty common.  We could make a button for that, but this UI would make that an easy 2-clicks to get it.

I like your idea of groupings!  I'll take a crack at it.  The hard part is if someone goes into the filter panel and PART of one grouping is enabled/part not.  Maybe I'd just show that an ON.  Or maybe make a 3rd state...  That's starting to get cumbersome, though.  :)

Thanks for the great feedback though!  I have let this PR sleep a loooooong time...

Comment 12

[Cameron Dawson [:camd]]

I'll take a crack at the result aggregation you mentioned.  :)

Comment 13

[Cameron Dawson [:camd]]

Hey Ed: I have a working copy of the aggregated states.  But I need to do some cleanup.  I think it'll work quite well.  But I'll finish it up on Monday.  Removing your for review till I have it ready.  :)

Comment 14

[Cameron Dawson [:camd]]

I fooled around with this over the weekend and this morning and I think it's in the right shape now.  I ended up subsuming a fix to the 'i' shortcut for toggling in progress jobs in bug 1106646.  Wes had a PR for it, but it was somewhat necessary/convenient to fix both at the same time.

Comment 15

[Cameron Dawson [:camd]]

Ed: yeah, that sounds good.  Let's remove from the filters panel once we get a little feedback.  Thanks for the review!

Comment 16

[Treeherder GitHub Bugbot]

Commit pushed to master at https://github.com/mozilla/treeherder

https://github.com/mozilla/treeherder/commit/0db65a244701d6bd097ecea8a5a7cb4037940d4c
Bug 1045606 - make status filters more discoverable

Adds buttons (chicklets) to the nav bar for quick toggling of job filters
by result/state.
- busted,testfailed,exception are grouped as "failures"
- pending,running are grouped as "in progress"

Comment 17

[Cameron Dawson [:camd]]

The new filter bar is merged and now on the staging server.  I hope this addresses the main issue in the bug.  If not, please reopen with comment or open a new one.

Comment 18

[William Lachance (:wlach)]

The tooltips for the filter bar could be more descriptive. It only gives the name of the filter.

I think it would be better to say something like "Hide all failures" or "Show all failures" instead of just "failures"