1059375 - Harder to grok if a bug suggestion matches the log failure compared to TBPL

Status: RESOLVED FIXED

(Tree Management :: Treeherder, defect, P2)

Description

[Ed Morley [:emorley]]

With the current text highlighting/indentation it is harder to tell from a bug summary if it is actually the right bug for the failure log line shown.

This makes classifying failures a few seconds slower for each failure, which given hundreds of failures a day makes a significant difference.

Comment 1

[Jonathan Eads ( :jeads )]

Could you give some specific suggestions on how to improve this? Not sure how to proceed to fix it.

Comment 2

[Jonathan French (:jfrench)]

A screen grab mock up would be helpful, and I will try to match it exactly. Noting here for reference, the earlier work on fixing summary readability was bug 1043785.

Comment 3

[Ed Morley [:emorley]]

Ran out of time to try and come up with some ideas here today, will take a look tomorrow :-)

Comment 4

[Ed Morley [:emorley]]

Bug 1057364 will help slightly with this - once that's landed I'll see if this still blocks and if so create a mockup. I should hopefully get some time over the weekend for this.

Comment 5

[Jonathan French (:jfrench)]

I think I see tighter Failure Summary lines on production. Does it seem sufficient we can close this bug Ed? If not, I will watch for the mockup.

Comment 6

[Jonathan French (:jfrench)]

I was playing around a bit for interest. I have a change that groups with even/odd background colors the failure log line and the related bug(s), and does so pretty clearly. Very similar to what we did with mozmill-dashboard with the recent updates. I'll attach a screen grab with different job examples, in case it meets the visual need.

Comment 7

[Jonathan French (:jfrench)]

Attachment: loglineGroupingByColor

This groups the failure log line and related bug(s) by color, the bonus being it is independent of any unrelated UI layout changes later, or the user's browser settings (font size, family, etc). This screen grab shows several job examples.

It also seems to help when there are no bugs at all, just by visually breaking up what otherwise appears as a large block of text.

Comment 8

[Ed Morley [:emorley]]

One of the things that is currently making the the panel hard to use, is definitely the difficulty in telling a failure line from the log line. TBPL makes use of indentation (albeit too much indentation IMO as it increases wrap) + colour/bolding, with treeherder they all seem to merge into one (particularly when the lines wrap). Anything that can distinguish between a bug suggestion and a failure line would help greatly. 

Maybe foreground text colour is the way to go here? :-)

Comment 9

[Jonathan French (:jfrench)]

By having shrunk the line spacing last week with d194b8e7, personally I think it has exacerbated the problem of seeing the log/bugs as discrete pairs. The entire content in the failure summary looks like one very long block. I've tried to do alternate colors this evening (backgrounds) for the failure line and the bug summary in each 'pair' (like TPBL) - but it just makes the appearance worse. Everything looks completely disconnected at that point.

That above attachment was the best so far I was able to do within the restriction of the new reduced line spacing.

As far as I can tell, TBPL never faced this UI problem because it only ever presented one bug and one log line for a given job failure(?). At least the failures I'm checking out right now on TBPL. Caveat I am a newbie to all the functionality.

If you want to make a mockup of what you want, that may be the fastest way to a solution.

Comment 10

[Ed Morley [:emorley]]

(In reply to Jonathan French (:jfrench) from comment #9)
> By having shrunk the line spacing last week with d194b8e7, personally I
> think it has exacerbated the problem of seeing the log/bugs as discrete
> pairs.

This was already an issue for me before this change, and reducing line spacing was important, since the number of viewable lines is sadly much worse in treeherder compared to TBPL (see screenshot).

> As far as I can tell, TBPL never faced this UI problem because it only ever
> presented one bug and one log line for a given job failure(?). At least the
> failures I'm checking out right now on TBPL. Caveat I am a newbie to all the
> functionality.

This isn't the case in TBPL (see screenshot) - note that once failures are classified in TBPL the suggestions disappear, so it's best to use https://tbpl-dev.allizom.org to see the original view, since the failures aren't classified by the sheriffs there.
 
> If you want to make a mockup of what you want, that may be the fastest way
> to a solution.

I've attached a comparison on TBPL to treeherder. When glancing at each, imagine that you have about 1.5-2 seconds between each failure classification (this is roughly the speed that's possible with TBPL) - in that time you have pressed the j/k key, spotted whether the failure matches the bug suggestion, chosen the bug suggestion, submitted it & moved onto the next.

Looking at the screenshot, the TBPL layout:
1) Shows more log lines at once (so no need to scroll; though in treeherder this is a limitation of having the job metadata in the left panel, so I accept this is harder).
2) Has fewer wrapped lines (which are harder to visually compare to the bug suggestion by scanning up/down) [ditto limitations of the changed layout from #1].
3) More clearly differentiates bug suggestion from failure line by line colour.

Now unfortunately I'm not a UX person & apart from "try other colours" I don't have much else to suggest right away - but perhaps this is something I can chat with the other sheriffs about next week when we meet up?

Comment 11

[Ed Morley [:emorley]]

Attachment: TBPL & treeherder failure summary comparison

Comment 12

[Jonathan French (:jfrench)]

(In reply to Ed Morley [:edmorley] from comment #10)
> perhaps this is something I can chat with the other sheriffs about next week when we meet up?

I think this is a good solution. I assume we want to improve its readability beyond both TBPL and existing Treeherder, to make it the best it can be. ie. potentially better and faster to grok than both of them. I'll un-assign myself for now, and potentially even one of the folks in the work week can pick up the bug and iterate with the sheriffs on it, in the absence of a mockup.

If it doesn't get in the fix/queue during the work week, I'll pick it up after.

Comment 13

[Jonathan French (:jfrench)]

I'm poking around on this during the work week. I'm looking at leveraging the existing colors of the structured log failure-line (grey w/red text), in the failure summary panel. So we have a consistency of user experience in both places, for the same line - and to cue users visually it is the same thing. Whether that will beenough differentiation from the bug lines is tbd. Also some other layout improvements.

No worries if someone takes this bug during the work week.

Comment 14

[Jonathan French (:jfrench)]

Attachment: grokFailuresWip_rev1

Iterating a bit. In this rev we:

1) remove some unnecessary padding
2) align the show/hide with the bug list so it's not ragged
3) leverage the color/style of the structured log line failure
4) reduce the size of the pin icon (gives us 1 extra line in the panel)

The net change also gives us 3-4 characters extra before line wrap occurs.

This provides a consistent appearance across several parts of the platform for what is the same thing (a log failure line here, and in the structured log). We'd like to enhance the show/hide further to help it stand out, but potentially not have it consume a full line of space.

Comment 15

[Jonathan French (:jfrench)]

It seems we're getting close so re-assigning myself.

Comment 16

[Jonathan French (:jfrench)]

Attachment: treeherder-ui-PR#164

Please see the above PR for review and status.

Comment 17

[Jonathan French (:jfrench)]

Landed and merged in master
https://github.com/mozilla/treeherder-ui/commit/13332c58fbfd8a1050920c95c0cef995008c79af

I will marked resolved once it's pushed to dev/stage, and I've had a chance to check it live.

Comment 18

[Jonathan French (:jfrench)]

Appears to be looking fine and working correctly on dev. Marking fixed prior to push to stage/prod.