1127873 - (compat-mdn-tables) [Compat Tables] Circulate Draft #3 for feedback

Status: RESOLVED FIXED

(developer.mozilla.org Graveyard :: General, defect)

Description

[Stephanie Hobson [:shobson]]

Moar feedback, moar user testing.

Comment 1

[Stephanie Hobson [:shobson]]

The draft is here: stephaniehobson.github.io/browsercompat/3/

Comment 2

[Stephanie Hobson [:shobson]]

Attachment: 5 second usability test brand first vs medium first

I've attached an amalgamated screen shot of my 5 second usability test on brand vs medium organization for the browsers. 

The instructions were "Click on the table cell that indicates whether Internet Explorer on a mobile phone supports "type=date"." Which is a use case that I would expect to favour the brand first organization and it did. But, hey, it's always useful to test our assumptions.

The 5 second tests are administered on a wide demographic so non technical people were taking the test which might account for what I consider to be an abysmal rate of accuracy.

On the browser first organization 65% of people were accurate in an average of 19 seconds.

On the brand first organization 65% were accurate but it took them an average of 38 seconds to find the answer.

Comment 3

[Stephanie Hobson [:shobson]]

I also did 5 second tests on the icons I was planning to use for protected, alternate name, and prefixed.

- Protected = shield = good results.
- Alternate name = random = bad results. (no surprise, I think we're going to have to design our own for this)
- Prefixed = -x- = bad results. (bit of a surprise, might have to design our own here too, but possibly a problem due to non-technical audience)

Comment 4

[Stephanie Hobson [:shobson]]

Considering making the legend visible by default to help with this but with the colour swatches as part of the legend now it takes up a lot of space. (load page with javascript disabled to see)

Comment 5

[Stephanie Hobson [:shobson]]

The results of the test I made to test the other use case for brand vs medium organization are in! 

I asked users to decide if a certain property was safe to use on mobile only or on desktop only. I ran two tests, one where it was safe and one where it wasn't.

Brand first
- accuracy: 26/40
- speed: 38.5s

Medium first: 
- accuracy: 29/40 
- speed 42.5s

I am very surprised that the speed for this test was faster on the brand first organization. That is not what I expected. Accuracy on medium first was higher and that is what I expected. 

I'm not sure the results are statistically significant.

Comment 6

[Stephanie Hobson [:shobson]]

It's also been suggested I can make the display order an option in the preferences as part of phase II so mostly I'm just trying to decide on a starting default rather than ruling one of these out permanently.

Comment 7

[Stephanie Hobson [:shobson]]

Icon usability test updates:

Footnotes: switched to [] which got slightly more clicks
Prefix: the -x- I'm already using got slightly more clicks
Alternate Name: everyone hated the options, I'd ask for a custom icon to be designed but I'm not sure we can improve on this. There just isn't an icon that means alternate name :P I'll implement the mask icon I created (not part of font-awesome) because it got the most clicks. 

I'm also going to make the legend visible by default to help with icon recognition.

Comment 8

[Stephanie Hobson [:shobson]]

I set up a "preference" test with the light and dark themes. Results tend towards the dark theme but are not statistically significant. I'm going to stick with the dark theme because I think it goes with our design better. I would not be sad to have input from actual designers here though.

Comment 9

[Stephanie Hobson [:shobson]]

New plan for the alt name icon! The alt symbol! ⌥ My gut says this will work best but I think my twitter feed is stick of testing :P