Closed Bug 1556264 Opened 5 years ago Closed 3 years ago

Use OS-level PiP framework where possible

Categories

(Toolkit :: Picture-in-Picture, defect, P5)

defect

Tracking

()

VERIFIED WONTFIX

People

(Reporter: mark, Unassigned)

References

(Blocks 1 open bug)

Details

User Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_14_5) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/12.1.1 Safari/605.1.15

Steps to reproduce:

  1. Navigate to a page containing HTML5 video
  2. Toggle PIP using extension or site functionality

Actual results:

Firefox uses its own implementation of Picture-in-Picture, regardless of OS-level support.

Expected results:

On macOS, the video should be presented through the native PIPViewController. Similarly, the Windows 10 CompactOverlay API should be leveraged.

macOS's windowing system handles picture-in-picture differently from other windows, as can be observed when using PIP in Safari or other apps that implement PIPViewController, such as IINA. Firefox instead implements its own PIP windowing. macOS windowing tools, such as Mission Control, interfere with Firefox's PIP window. This provides an inconsistent experience for users who watch video in different apps on Mac.

Notice how Firefox's PIP window differs from Safari's:

  • On activating PIP, there is no transitional animation.
  • The PIP window doesn't snap to the edge of the screen, but instead freely floats anywhere, making it a bit of a pain to quickly position.
  • The PIP window does not allow resizing.
  • You cannot move the PIP window off to the edge of the screen and temporarily "hide" it. PIPViewController honors this, blurs the window, and shows a chevron.
  • The PIP window is tied to a Space in macOS windowing. Thus, it doesn't stay on screen when entering fullscreen or switching between spaces.
  • Mission Control shows the PIP video as a separate window, even though it is conceptually an extension of the page in Firefox.

Just to note: PIPViewController on macOS is a private API for the moment, but can easily be implemented in a third party Cocoa app (as it is in IINA) and might soon become public, as it is public on iOS and a lot of iOS API surface is supposedly coming to macOS this year. However, the Windows 10 CompactOverlay API is public.

Blocks: 1527926
Component: Untriaged → Video/Audio Controls
Product: Firefox → Toolkit

From what I recall, CompactOverlay is only available for UWP apps, is it not?

Flags: needinfo?(malstrommark)
Priority: -- → P5

(In reply to Mike Conley (:mconley) (:⚙️) from comment #3)

From what I recall, CompactOverlay is only available for UWP apps, is it not?

It's possible. I can't exactly tell from the documentation.

Flags: needinfo?(malstrommark)

FWIW, this API (now called AVPictureInPictureController) is a public API in macOS 10.15 Catalina.

Blocks: videopip
No longer blocks: 1527926
Blocks: 1532675
No longer blocks: videopip
Blocks: videopip
No longer blocks: 1532675

AVPictureInPictureController is tightly tied to AVFoundation and I would guess that it's probably too limited for Firefox's use case unless it's possible to route all video content through AVPlayerLayer. macOS's PIP framework is private API, but its headers are currently exposed as part of WebKit and support presenting arbitrary NSViewControllers, which is significantly more flexible. IINA has been using the API for the last three years; it is relatively straightforwards to use and hasn't changed much since then. One caveat is that both APIs run out-of-process and there is little support to add additional controls, so the volume control in particular would not work; however, in return you'd get all the benefits that Mark mentioned above.

(Full disclosure: I wrote the PIP implementation for IINA mentioned above. I'd be glad to help write one for Firefox too if this is something the team would be interested in.)

Component: Video/Audio Controls → Picture-in-Picture
Version: 67 Branch → Trunk

As a user, I very much would prefer macOS' built in PIP viewer, despite losing any specific functionality that Firefox may add (see reasons listed by Mark).

Does Firefox have any plans to add support for this? It is one of the soft blocking reasons I prefer Safari over Firefox.

(In reply to alex from comment #7)

As a user, I very much would prefer macOS' built in PIP viewer, despite losing any specific functionality that Firefox may add (see reasons listed by Mark).

Does Firefox have any plans to add support for this? It is one of the soft blocking reasons I prefer Safari over Firefox.

I'm in the same boat. I see a performance impact as well as issues with fullscreen apps (using the green button in windows), in which case the firefox PiP isn't visible, while native PiP is.

Does Firefox have any plans to add support for this? It is one of the soft blocking reasons I prefer Safari over Firefox.

No, we do not have plans to support this at this time. We're actually moving further away, looking into investing in ways to introduce more controls and captions/subtitles to the player window itself, which would not be possible using the OS-level player window.

Status: UNCONFIRMED → RESOLVED
Closed: 3 years ago
Resolution: --- → WONTFIX

(In reply to Mike Conley (:mconley) (:⚙️) from comment #9)

Does Firefox have any plans to add support for this? It is one of the soft blocking reasons I prefer Safari over Firefox.

No, we do not have plans to support this at this time. We're actually moving further away, looking into investing in ways to introduce more controls and captions/subtitles to the player window itself, which would not be possible using the OS-level player window.

Would it not be possible to have an option to disable the custom version and use the native version instead for users that prefer a native experience over additional features?

(In reply to Mike Conley (:mconley) (:⚙️) from comment #9)

Does Firefox have any plans to add support for this? It is one of the soft blocking reasons I prefer Safari over Firefox.

No, we do not have plans to support this at this time. We're actually moving further away, looking into investing in ways to introduce more controls and captions/subtitles to the player window itself, which would not be possible using the OS-level player window.

Sorry to add another comment, but it is possible to add subtitles to a native PiP window, at least in Safari 15.2 and YouTube. (example: https://i.imgur.com/GK2XjmT.png) I can't speak for other platforms, however.

The Firefox-specific implementation of PiP is just not very good on macOS and it is keeping me from switching to Firefox full time.

I would also love to see OS level PiP API used over a custom PiP window. It has all the necessary things and works properly over fullscreen apps. Please, reconsider supporting OS API instead. This is one of the reasons I have to switch to Safari for videos: custom PiP and lack of TouchID over FIDO.

We continue to not have any plans of using the OS-provided Picture-in-Picture API at this time. Instead, we plan on investing in our own custom player window to allow us to introduce captions, subtitles, and additional controls. Individual deficiencies of the player window on macOS (or any other platform) should be filed as separate bugs. Thank you!

Status: RESOLVED → VERIFIED

I don’t really want to drag this conversation on too long given the current decision, but I’m curious what the policy will be for features that cannot be replicated by Firefox without deep SPI, such as dodging notifications and drawing across screens. Would you still like bug reports for these, even if they are unlikely to ever be implemented with a custom solution?

Hi Saagar,

You're welcome to file enhancements requests via https://mozilla.crowdicity.com/ - this is where other users can see and vote on them. The product team grooms top-voted ideas and considers them for Firefox teams' backlogs.

We try to keep a separation between Bugzilla for tracking tasks and defects and Ideas@Mozilla for everything else, so enhancement requests filed as bugs here will get closed.

You need to log in before you can comment on or make changes to this bug.