[UX] Redesign content fullscreen experience to be less intrusive and jarring when possible

RESOLVED FIXED

Status

()

Firefox
General
RESOLVED FIXED
3 years ago
2 years ago

People

(Reporter: Benjamin Smedberg, Assigned: verdi)

Tracking

(Depends on: 2 bugs, Blocks: 1 bug)

unspecified
x86
Windows 7
Points:
8
Dependency tree / graph
Bug Flags:
firefox-backlog +
qe-verify -

Firefox Tracking Flags

(Not tracked)

Details

(Whiteboard: [ux])

Attachments

(1 attachment, 1 obsolete attachment)

(Reporter)

Description

3 years ago
Especially now that we're using non-Flash video on youtube, more users are going to see our content full-screen experience. The general impression seems to be that our current full-screen dialog is too invasive. We'd like to try and make it smaller or auto-disable, and explore options similar to Chrome and Flash full-screen modes.

See bug 1111099 for a bunch of discussion.
Flags: firefox-backlog+
(Reporter)

Comment 1

3 years ago
To be clear, there are also issues with the full-screen transition, but that is a platform issue that is filed separately and isn't part of this bug.

Updated

3 years ago
Flags: qe-verify-
Whiteboard: [ux]
See Also: → bug 1111099

Updated

3 years ago
Points: --- → 8

Updated

2 years ago
Assignee: nobody → mverdi
Status: NEW → ASSIGNED
Iteration: --- → 40.1 - 13 Apr

Updated

2 years ago
Iteration: 40.1 - 13 Apr → 40.2 - 27 Apr

Updated

2 years ago
Iteration: 40.2 - 27 Apr → 40.3 - 11 May
(Assignee)

Comment 2

2 years ago
Created attachment 8598930 [details]
fullscreen-notice.jpg

More information about this proposal can be found here - http://people.mozilla.org/~mverdi/projects/fullscreen

When an element goes full screen, Firefox animates the transition and displays a modal permission dialog to protect the user from a phishing attack. We've got a number of issues with this interaction that span three main use-cases: video, games and VR. The main things being: 

* On Windows there isn't much of a transition at all.
* The dialog seems a bit scary, especially for common use-cases like watching a video in full screen.
* The dialog is modal, preventing interaction with the content until the user takes action.
* You can deny full screen permission and remember it, effectively breaking the feature on that website. 
* When a VR headset is used, the user can't see or interact with this modal dialog. 
* Games often request multiple permissions (pointer lock, storage, full screen, camera & mic) resulting in stacked or conflicting dialogs.

Security Considerations
The main concern with full screen is that the browser UI is overwritten and if the user is unaware that this has happened then all manner of phishing attacks are possible. Additionally, if you can't distinguish between content going full screen and the whole window going full screen, it makes it easier for content to impersonate browser chrome.

Proposal
The main strategy to mitigate the security concerns is to make it obvious to the user that they are in full screen. This is generally accomplished though a transition and a message. 

Transition
We'll use a 1 second fade-though-black transition (14 frames to fade to black, 2 frames of black, 14 frames to fade back). This provides an obvious reaction to the user's click or key press, indicating that something special is happening. The fade-though-black transition follows a cinematic convention and works well in the video, VR and game contexts. In addition, it is distinct from the Mac application full screen transition (this is a recommended security feature).

Initial Notification
As the full screen content fades in we'll animate a dialog in from above the view port (see attachment). It will:
* Indicate the connection status.
* Display the URL of the content that's being displayed in full screen.
* Include a clickable/tappable exit button.

The notification is displayed for a total of 3 seconds (including 20 frames to animate it in and 20 frames to animate it out).

Accessible status and exit
Once in full screen the user can always access the full screen status notification and exit button by:
* Mousing to the top of the screen. We should use a half second delay to keep from interfering with interacting with content near the top edge of the screen.
* Swiping down from the top edge of the screen on a tablet.
* Using button 16 on a standard gamepad.

Using one of these options will animate in the status message docked to the top of the view port. Of course, full screen can be exited immediately by using the ESC key if available.

View a video mockup of this interaction: 
http://people.mozilla.org/~mverdi/projects/fullscreen/fs-v4.m4v

Changes from current interaction
We'll now use a consistent, fade-though-black transition on all platforms. The permission dialog adds the connection status, exit button and active permissions indicators. Making the transition to full screen obvious and giving the user an indication of the connection status and an always available exit button allows us to remove the allow once and always deny permissions.

Notes on treating HTTP and HTTPS differently
The one security recommendation not fully adopted in this proposal is treating HTTP and HTTPS differently by having a persistent UI or no "always allow" permission on HTTP.

These additions would mainly be to guard against going into full screen without the user present. Given that full screen is triggered by user action (key press or click) I think we should treat any scenario where this could happen without the user present (we don't currently have any) as a bug in the full screen implementation. This proposal also includes displaying the connection status, including a specifically insecure http icon. Therefore, I don't believe that these additional measures would strike the right balance between ease of use and security.

VR
http://people.mozilla.org/~mverdi/projects/fullscreen/#vr-interactions
When a VR headset is used, the user can't see or interact with our modal dialog. In this case, because user interaction and a head mounted display (HMD) is required to initiate full screen, we can simply display a notification that VR mode has been enabled (rendered stereoscopically so it's visible in the headset and on the computer) and do away with granting or denying permission.

Games
http://people.mozilla.org/~mverdi/projects/fullscreen/#game-interactions
There seem to be two main issues with games.
* Asking for multiple permissions results in multiple doorhangers (best case) and sometimes conflicting/missing doorhangers.
* Asking for permissions while in full screen mode results in doorhangers attached to a hidden interface.

If we continue to treat the full screen permission separately, we can group the rest, let you allow them all by default and still have the option to make individual choices. Because this work will have to coordinate with the work being done on control center and because it can be make to work in the framework proposed here, I think we should move ahead with the video use case and continue game interactions in a followup bug.
Attachment #8598930 - Flags: ui-review?(philipp)
Blocks: 1160014
Blocks: 1160023
Comment on attachment 8598930 [details]
fullscreen-notice.jpg

Yep! That works.
The contrast between the highlighted domain and the rest of the text is quite low at the moment. Which font weights are you using here, Michael?
(Assignee)

Comment 4

2 years ago
(In reply to Philipp Sackl [:phlsa] please use needinfo from comment #3)
> Comment on attachment 8598930 [details]
> fullscreen-notice.jpg
> 
> Yep! That works.
> The contrast between the highlighted domain and the rest of the text is
> quite low at the moment. Which font weights are you using here, Michael?

I used open sans regular for the domain and open sans light for the rest.
verdi, is the current proposal of fade-through-black transition stable enough to be implemented?
Flags: needinfo?(mverdi)
Flagging Richard for review here.
Flags: needinfo?(rlb)
Comment on attachment 8598930 [details]
fullscreen-notice.jpg

I don't think we have open sans available in the browser, so we should use platform fonts like Segoe, Tahoma, Helvetica.

Michael, could you please change the mockup accordingly (just to make sure that the contrast and sizes work?

Otherwise this is excellent :)
Attachment #8598930 - Flags: ui-review?(philipp) → ui-review+

Comment 8

2 years ago
Should "esc" be lowercase on Mac to match the keyboard?
(Assignee)

Comment 9

2 years ago
Created attachment 8601584 [details]
fullscreen-fonts.png

Here's a new version with Helvetica Neue, Tahoma and Segoe.
Attachment #8598930 - Attachment is obsolete: true
Flags: needinfo?(mverdi)

Updated

2 years ago
Summary: [UX] Redesign content full-screen experience to be less intrusive and jarring when possible → [UX] Redesign content fullscreen experience to be less intrusive and jarring when possible

Comment 10

2 years ago
> Once in full screen the user can always access the full screen status notification and exit button by:
> * Mousing to the top of the screen. We should use a half second delay to keep from interfering with
> interacting with content near the top edge of the screen.

This might cause problems for overhead strategy games, where moving the cursor to the edges of the screen is also used to scroll the view. Would those games have to request pointer lock?

Updated

2 years ago
Iteration: 40.3 - 11 May → 41.1 - May 25

Updated

2 years ago
Blocks: 1154742
If I remember correctly, the pending reviews for this bug have already happened in some email threads. Since everyone seemed happy with the current solution there, I'm closing the bug.

Feel free to either re-open or file a follow-up if you see any blocking shortcomings of the current design.
Status: ASSIGNED → RESOLVED
Last Resolved: 2 years ago
Flags: needinfo?(rlb)
Resolution: --- → FIXED
(In reply to Verdi [:verdi] from comment #2)
> Transition
> We'll use a 1 second fade-though-black transition (14 frames to fade to
> black, 2 frames of black, 14 frames to fade back). This provides an obvious
> reaction to the user's click or key press, indicating that something special
> is happening. The fade-though-black transition follows a cinematic
> convention and works well in the video, VR and game contexts. In addition,
> it is distinct from the Mac application full screen transition (this is a
> recommended security feature).

The transition to fullscreen should not interrupt the video. Doing so would be a regression compared to what we currently have. With video you start them in windowed mode and switch to full screen when they are playing. This means that you're watching the video during the transition and trying your best to ignore everything else around it. Contrast this with games where you start in full screen with pointer lock.

Verdi - what do I need to do to get a transition that doesn't interrupt video?
Status: RESOLVED → REOPENED
Flags: needinfo?(mverdi)
Resolution: FIXED → ---

Comment 13

2 years ago
(In reply to Anthony Jones (:kentuckyfriedtakahe, :k17e) from comment #12)
> The transition to fullscreen should not interrupt the video. Doing so would
> be a regression compared to what we currently have. With video you start
> them in windowed mode and switch to full screen when they are playing. This
> means that you're watching the video during the transition and trying your
> best to ignore everything else around it. Contrast this with games where you
> start in full screen with pointer lock.
> 
> Verdi - what do I need to do to get a transition that doesn't interrupt
> video?

Let's not reopen this UX bug--this bikeshed's been painted. Please file a separate bug _IF_ the fade-to-black design needs perf optimization as that's a Rendering bug for us to fix. I don't see why we shouldn't have a smooth fade with a moving video.
Status: REOPENED → RESOLVED
Last Resolved: 2 years ago2 years ago
Flags: needinfo?(mverdi)
Resolution: --- → FIXED
(In reply to Jet Villegas (:jet) from comment #13)
> Let's not reopen this UX bug--this bikeshed's been painted. Please file a
> separate bug _IF_ the fade-to-black design needs perf optimization as that's
> a Rendering bug for us to fix. I don't see why we shouldn't have a smooth
> fade with a moving video.

Anthony is not blaming the performance problem of the fade effect. He doesn't like the fact that fade-through-black would make several moving frames invisible during the transition. He thinks if that happens frequently (like watching many different music videos on Youtube), that would be annoying.
Blocks: 1170902
Duplicate of this bug: 1173629
Blocks: 1182982

Comment 16

2 years ago
It seems though that the current (from 1160014) fade through black has a longer period of blackness that the video by Verdi. One feels a bit left in the dark before the video reappears.
Depends on: 1187769
(In reply to Alfred Kayser from comment #16)
> It seems though that the current (from 1160014) fade through black has a
> longer period of blackness that the video by Verdi. One feels a bit left in
> the dark before the video reappears.

Added bug 1187769.
Blocks: 1190229
Depends on: 1196588
You need to log in before you can comment on or make changes to this bug.