1247997 - Not immediately obvious whether a result in perfherder compare view is a regression/improvement without being able to see the colour

Status: RESOLVED FIXED

(Tree Management :: Perfherder, defect, P3)

Description

[William Lachance (:wlach)]

This is a problem for two people:

* Red/Green colourblind
* Those relying on a screen reader

I think this should be relatively easy to fix.

Comment 1

[William Lachance (:wlach)]

Experimenting with a change that adds a tooltip to the less than / greater than sign for each compare row indicating whether the result was a regression or improvement:

http://wlach.github.io/treeherder/ui/perf.html#/comparesubtest?originalProject=mozilla-inbound&newProject=mozilla-inbound&originalRevision=af8247c22b74&newRevision=432ef38dab95&originalSignature=df8939dc6e77c3a3c208294042ab6b50013d3966&newSignature=df8939dc6e77c3a3c208294042ab6b50013d3966

Trevor, is this interface usable for you?

Comment 2

[Trevor Saunders (:tbsaunde)]

(In reply to William Lachance (:wlach) from comment #1)
> Experimenting with a change that adds a tooltip to the less than / greater
> than sign for each compare row indicating whether the result was a
> regression or improvement:

hm, so most screen readers actually allow inspecting the color of text so going to the </> sign and seeing the tooltip isn't much easier than going there and seeing the color, but it probably is more discoverable.

> Trevor, is this interface usable for you?

I'm not sure why, but I'm not seeingthe tooltip, I guess the text of </> overrides it especially since its not a link?

Comment 3

[William Lachance (:wlach)]

(In reply to Trevor Saunders (:tbsaunde) from comment #2)
> (In reply to William Lachance (:wlach) from comment #1)

> I'm not sure why, but I'm not seeingthe tooltip, I guess the text of </>
> overrides it especially since its not a link?

It's nested inside a span with the property "uib-tooltip". Should I use an aria-label in addition? https://developer.mozilla.org/en-US/docs/Web/Accessibility/ARIA/ARIA_Techniques/Using_the_aria-label_attribute

Comment 4

[Trevor Saunders (:tbsaunde)]

(In reply to William Lachance (:wlach) from comment #3)
> (In reply to Trevor Saunders (:tbsaunde) from comment #2)
> > (In reply to William Lachance (:wlach) from comment #1)
> 
> > I'm not sure why, but I'm not seeingthe tooltip, I guess the text of </>
> > overrides it especially since its not a link?
> 
> It's nested inside a span with the property "uib-tooltip". Should I use an
> aria-label in addition?

I really can't help you here.

Comment 5

[Marco Zehe (:MarcoZ)]

OK, several questions to this:

What are the < and > symbols supposed to convey exactly? Why is it required to mouse over these tiny symbols to actually see an explanation? Aside from just conveying information by color, conveying information by cryptic symbols whose meaning isn't clear excludes people who don't work with this every day, e. g. a lot of community members. And not everybody might get the idea that they can actually mouse over these non-linked characters to get something additional.

As Trevor also already noted, tool tips are mostly oblivious to screen readers. The title attribute is sometimes picked up, on focused elements, but on pure text representations, it is basically ignored. The only exception is the abbr and acronym elements.

In my opinion, this should be greatly simplified by just being explicit about the results, e. g. tell users in plain text if something got improved or worse, or whatever else the tool tips may show. This is in a data table structure already anyway, so putting real text in the columns will help a lot of people and save extra steps for everyone, including mouse users.

Comment 6

[Trevor Saunders (:tbsaunde)]

(In reply to Marco Zehe (:MarcoZ) from comment #5)
> OK, several questions to this:
> 
> What are the < and > symbols supposed to convey exactly?

so the context is something like

test-foo     5.0 < 10.0 ...

so the < just means what it normally does.

> Why is it required
> to mouse over these tiny symbols to actually see an explanation?

Those lines are already pretty long,  and if you see the color its pretty obvious what everything means.

> Aside from
> just conveying information by color, conveying information by cryptic
> symbols whose meaning isn't clear

I think people usually learn what < and > mean in third grade math if not earlier? so I wouldn't say its cryptic.

> excludes people who don't work with this
> every day, e. g. a lot of community members.

No, it just means they need to ask or go look for docs.  And honestly I expect if you see the colors its pretty obvious what it means and very few people will have trouble figuring it out.

besides which this isn't the sort of site you are just going to stumble across so optimizing for people who know what they are doing seems pretty reasonable.

> As Trevor also already noted, tool tips are mostly oblivious to screen
> readers. The title attribute is sometimes picked up, on focused elements,
> but on pure text representations, it is basically ignored. The only
> exception is the abbr and acronym elements.

true, though think the real issue is this.

If you can see color what you do with this page generally is just look down the column seeing what color things are.  I expect that's really fast you don't need to read or anything just scan for red colored things.

I can't really think of a great alternative that doesn't use color.

> In my opinion, this should be greatly simplified by just being explicit
> about the results, e. g. tell users in plain text if something got improved
> or worse, or whatever else the tool tips may show. This is in a data table

I think that text will just get in the way for most people who will just use the color anyway because its faster.

What I think might work is to group the rows by there status so you'd have a group of rows for regressions then a group of rows for neutral and then a group for improvements.  That might actually help since you know you only need to read to the neutral part if you only care about regressions, but on the other hand it would mean the location of each test would move around which might not be great.

Comment 7

[William Lachance (:wlach)]

I'm not actively working on this right now. It sounds like this is still a problem, but I'm not sure if there's any easy solution.

Comment 8

[William Lachance (:wlach)]

Bug 1352207 added a tooltip indicating whether result was better/worse -- in that case to get around the fact that if we weren't confident, we wouldn't show a color at all (which confused even those able to distinguish red/green visually). That may mitigate this somewhat, still open to better solutions.

Comment 9

[Sarah Clements [:sclements]]

I think this has been resolved by the use of icons (with titles for use by screen readers)?

Comment 10

[Dave Hunt [:davehunt] [he/him] ⌚BST]

(In reply to Sarah Clements [:sclements] from comment #9)

I think this has been resolved by the use of icons (with titles for use by screen readers)?

I agree, let's close this.