947522 - Add peerIdentity variant for gUM icons

Status: NEW

(Core :: WebRTC: Audio/Video, defect, P5)

Description

[Martin Thomson [:mt:]]

Currently, the doorhanger icon for gUM shows in grey when the sources are not connected (aka webRTC-shareDevices-notification-icon)  or active and green when they are (webRTC-sharingDevices-notification-icon).  For authenticated WebRTC calls, we will need a third state for this icon (webRTC-sharingDevicesSecure-notification-icon) to provide users an indication that they are in a "secure" call.

To begin with, it's probably sufficient to use colour to distinguish the states.  I'm thinking:
 - grey: inactive
 - orange: site can access your media
 - green: only being send to authenticated peers

I know most of what I need to do to get the plumbing done (it's a work in progress), but as long as this is OK, I'd like to get some UX feedback on whether this is the right thing to do, and - if so - how to proceed.

Comment 1

[Dão Gottwald [:dao]]

I'm not the right person to provide UX feedback.

Comment 2

[Florian Quèze [:florian]]

(In reply to Martin Thomson [:mt] from comment #0)

> To begin with, it's probably sufficient to use colour to distinguish the
> states.

Please keep in mind that a significant proportion of the population (7 to 10% of males according to wikipedia) is color blind.

Comment 3

[Martin Thomson [:mt:]]

(In reply to Florian Quèze [:florian] [:flo] from comment #2)
> Please keep in mind that a significant proportion of the population (7 to
> 10% of males according to wikipedia) is color blind.

I'm well aware of that.  The standard solution - which isn't completely perfect, but covers the largest proportion of that population - is to have the green actually tinted slightly blue.  That's what is done for traffic lights.  I haven't looked into it, but I assumed that when Firefox uses green, that's what is done there as well.

Nevertheless, the "to begin with" was a nod to that problem.  I'm still in two minds about whether to include additional cues on the icon, which has the downside of visual clutter, or to rely on colour alone.  The downside is that some (hopefully very small) proportion of people will be inable to distinguish the peerIdentity and non-peerIdentity states at a glance, but that information would still be available from the doorhanger anyway.

Comment 4

[Florian Quèze [:florian]]

(In reply to Martin Thomson [:mt] from comment #3)

> I'm well aware of that.

Perfect, thanks!

Comment 5

[Maire Reavy [:mreavy]]

Note: should be folded into ongoing UX redesign so we don't have to go back and add it later.

Comment 6

[Florian Quèze [:florian]]

Is this still something we care about? If so the relevant UX people to needinfo are Bryan Bell and Aislinn Grigas. If not (given bug 942382 comment 3), can we close wontfix?

Comment 7

[Maire Reavy [:mreavy]]

Martin -- What are you current thoughts about this? Is this worth pursuing?  (We have folks available to look at this now.)

Comment 8

[Martin Thomson [:mt:]]

Re comment 6, this is different to bug 942382.  The point of this is to show what is using your camera/mic/screen.  When peerIdentity is in use, you aren't sharing your camera/mic/screen with the site, you are sharing it with a specific person.

The latest mockups (https://mozilla.invisionapp.com/share/AF71R266U#/screens/142999150) show a red icon when the site is using the camera.  See above regarding some suggestions for the icon, with both colour and outline changes.  You could maybe overlay a negative of the classic person icon to distinguish it somehow.  If you excuse my hasty pictures, this is what I'm thinking: https://docs.google.com/drawings/d/1VliZm5SOCjTvLuBQubCoLYVcqSE5PFa0MBCHSFeZkcQ/edit?usp=sharing  I don't know if that translates well into the smaller form.

If there are any non-peerIdentity shares, we should just show the other UX, this doesn't need to be a special overlay.

The biggest part of this is the doorhanger.  In a world where we have infinite time, we would show the contact card of the peer in the prompt and use different icons and text.  I think that we should instead settle for showing the different iconography and use different text.  Maybe: "Would you like to use your camera for a private call with **[e-mail]**?"

Comment 9

[Florian Quèze [:florian]]

Adding needinfos to get this on the UX radar.

(In reply to Martin Thomson [:mt:] from comment #8)

Martin, could you tell me more about the current state of this feature? More specifically:
- is the backend implementation finished?
- are all the APIs in place so that the front-end can access this information? (if so, do you have pointers to them?)
- are there known examples of sites using this?

Comment 10

[Martin Thomson [:mt:]]

> - is the backend implementation finished?

Yes.

> - are all the APIs in place so that the front-end can access this information? (if so, do you have pointers to them?)

You can access MediaStreamTrackSource::GetPeerIdentity() http://searchfox.org/mozilla-central/search?q=symbol:_ZNK7mozilla3dom22MediaStreamTrackSource15GetPeerIdentityEv&redirect=false

I suspect that this won't be available to script though, we probably need a simple [ChromeOnly] hook added to something, the easiest being MediaStreamTrack, though that might not suit you.  The goal was to roll this into getConstraints(), but our implementation isn't there yet.

> - are there known examples of sites using this?

I don't have a URL that I can share; I know that there are several people working on implementing something.

Comment 11

[Florian Quèze [:florian]]

(In reply to Martin Thomson [:mt:] from comment #10)

> I suspect that this won't be available to script though, we probably need a
> simple [ChromeOnly] hook added to something, the easiest being
> MediaStreamTrack, though that might not suit you.  The goal was to roll this
> into getConstraints(), but our implementation isn't there yet.

Is there a bug on file I could cc myself on?

> > - are there known examples of sites using this?
> 
> I don't have a URL that I can share; I know that there are several people
> working on implementing something.

If there's no public website yet, a test page would already be useful to test any UI work we may do.

Comment 12

[Jan-Ivar Bruaroey [:jib] (needinfo? me)]

> The goal was to roll this into getConstraints(), but our implementation isn't there yet.

FWIW Bug 1213517 is about ready to land for cameras (waiting on a review from jesup), but yeah we haven't begun to add the peerIdentity parameter (doesn't really fit the constrainable model, is one problem the spec [1] hasn't addressed yet).

[1] http://w3c.github.io/webrtc-pc/#isolated-media-streams

Comment 13

[Martin Thomson [:mt:]]

Jan-Ivar is right; peerIdentity doesn't fit into the getConstraints model.  But that shouldn't matter here, we can add a [ChromeOnly] property for UX needs, I've opened bug 1286415.

Comment 14

[Ash Grigas]

(In reply to Martin Thomson [:mt:] from comment #0)
> Currently, the doorhanger icon for gUM shows in grey when the sources are
> not connected (aka webRTC-shareDevices-notification-icon)  or active and
> green when they are (webRTC-sharingDevices-notification-icon).  For
> authenticated WebRTC calls, we will need a third state for this icon
> (webRTC-sharingDevicesSecure-notification-icon) to provide users an
> indication that they are in a "secure" call.
> 
> To begin with, it's probably sufficient to use colour to distinguish the
> states.  I'm thinking:
>  - grey: inactive
>  - orange: site can access your media
>  - green: only being send to authenticated peers
> 
> I know most of what I need to do to get the plumbing done (it's a work in
> progress), but as long as this is OK, I'd like to get some UX feedback on
> whether this is the right thing to do, and - if so - how to proceed.

Can you provide screenshots or a test page to the current states we have? Its hard to get what you're describing without seeing something. 

Second, if they are not securely connected to authenticated peers why would we let the site access their media? If this is all over https why would we need to warn the user with an orange state. 

I think we can simplify to two states especially given most users dont even notice what changes in the url bar and it has been a goal with any url bar changes to simplify and reduce as much as possible and let the user dig deeper in the control center if they have concerns.

Comment 15

[Eric Rescorla (:ekr)]

(In reply to agrigas from comment #14)
> (In reply to Martin Thomson [:mt:] from comment #0)
> > Currently, the doorhanger icon for gUM shows in grey when the sources are
> > not connected (aka webRTC-shareDevices-notification-icon)  or active and
> > green when they are (webRTC-sharingDevices-notification-icon).  For
> > authenticated WebRTC calls, we will need a third state for this icon
> > (webRTC-sharingDevicesSecure-notification-icon) to provide users an
> > indication that they are in a "secure" call.
> > 
> > To begin with, it's probably sufficient to use colour to distinguish the
> > states.  I'm thinking:
> >  - grey: inactive
> >  - orange: site can access your media
> >  - green: only being send to authenticated peers
> > 
> > I know most of what I need to do to get the plumbing done (it's a work in
> > progress), but as long as this is OK, I'd like to get some UX feedback on
> > whether this is the right thing to do, and - if so - how to proceed.
> 
> Can you provide screenshots or a test page to the current states we have?
> Its hard to get what you're describing without seeing something. 
> 
> Second, if they are not securely connected to authenticated peers why would
> we let the site access their media? If this is all over https why would we
> need to warn the user with an orange state. 

You're conflating two different forms of authentication for the media:

- Authentication of the site that is coordinating the media.
- End-to-end authentication of the peer that the media is being sent to


> I think we can simplify to two states especially given most users dont even
> notice what changes in the url bar and it has been a goal with any url bar
> changes to simplify and reduce as much as possible and let the user dig
> deeper in the control center if they have concerns.

Comment 16

[Bulk Bug Changes for mreavy's org]

Mass change P4->P5 to align with new Mozilla triage process.