1665251 - Picture-in-picture (PiP) button can be tabbed to but not activated when focused for HTML5 videos

Status: VERIFIED FIXED

(Toolkit :: Picture-in-Picture, defect, P3)

Description

[Asa Dotzler [:asa]]

If you open an HTML5 video and tab around, the first tab lands on the video frame. The second tab lands on the closed captions button. The third tab lands on the full-screen button. The next tab moves out of the video frame to what ever's next in the page. None of those tabs land you on the PiP button.

Steps to reproduce:

1. Load the video at http://pauljadam.com/demos/html5-video-a11y.html
2. Tab to focus the video.
3. Space to play the video.
4. Tab again to focus the cc button.
5. Tab again to focus the full-screen button.
6. Tab again and focus leaves the video.

Results: none of the tabbing landed on the PiP button.

Expected results: After focusing the video, the next tab should focus (and make visible) the PiP button. Then pressing space or enter should activate PiP.

Tested with today's Nightly on Windows 10.

Comment 1

[Asa Dotzler [:asa]]

I'm wrong. The pip button does show up in the tab order but I can't seem to activate it.

Comment 2

[James Teh [:Jamie]]

Mike, I looked into this a bit (including your excellent in-tree documentation) and realised this is going to be a bit tricky to fix. So, I thought it might be useful to discuss the implementation before diving in.

First, I realise there's a keyboard shortcut for PIP and this is great. However, this is less discoverable than having the button in the tab order, so we also want this to work.

Because the PiP button doesn't have a simple click listener, the default keyboard handling for the button doesn't work. A few solutions come to mind:

1. Tweak PictureInPictureChild so that a click event alone works without other pointer events.
2. Add a click event handler to the PiP button in videocontrols.js (keeping the existing capturing listeners in PictureInPicture).
3. Add a keypress/keydown listener to the PiP button in videocontrols.js. While we have to use a capturing listener on the root window for pointer events to prevent pages from doing things like skipping ads, pages bothering to do things like this for keyboard events is probably less likely.
4. Add a capturing keypress/keydown listener in PictureInPictureChild.

Beyond keyboard activation, we also need to handle activation via a11y APIs. The a11y engine handles activation by simulating touchStart, mouseDown, touchEnd and mouseUp events:
https://searchfox.org/mozilla-central/rev/819be4899a92213abf121b449779ced662f2ce13/accessible/generic/Accessible.cpp#1869
Just to make things more complicated, the controls can be visually hidden (but still semantically present) as per bug 1622995. This means a11y activation needs to work even while the PiP button is visually hidden. That's probably going to cause problems for the PictureInPictureChild code which checks for occlusion. A11y does target the event directly at the button, so that might help us deal with this. Adding a click event listener directly to the PiP button (option 2 above) would fix this, but I guess pages could still capture the event and cause problems.

I'd be grateful for any guidance you can offer.

Comment 4

[BugBot [:suhaib / :marco/ :calixte]]

The severity field for this bug is set to S3. However, the accessibility severity is higher, [access-s2].
:jaws, could you consider increasing the severity?

For more information, please visit auto_nag documentation.

Comment 5

[:Gijs (he/him)]

Molly said she'll get some eyes on this.

Comment 6

[Neil Deakin]

Attachment: Bug 1665251, handle activating the pip toggle with the keyboard, r=mconley

Comment 7

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/cafe8f49eb3f
handle activating the pip toggle with the keyboard, r=mconley

Comment 8

[Natalia Csoregi [:nataliaCs]]

https://hg.mozilla.org/mozilla-central/rev/cafe8f49eb3f

Comment 9

[Ardelean Oana, Desktop QA [:oardelean]]

Reproducible on Firefox 99.0(20220330194208) on macOS 11. Confirmed as fixed on Firefox 100.0b5(20220412185818) and Nightly 101.0a1(20220414092955) on macOS 11, Win10 64-bit and Ubuntu 20.04.