1868435 - Option to use native macOS PIP API

Status: NEW

(Toolkit :: Picture-in-Picture, enhancement, P3)

Description

[Tanner Davies]

Steps to reproduce:

Currently, Firefox Picture-in-Picture (PIP) uses a custom implementation rather than using macOS’s AVKit PIP functionality (see https://developer.apple.com/documentation/avkit/adopting_picture_in_picture_in_a_custom_player).

There are good reasons for this. Firefox’s PIP is more fully-featured and supports actions such as jumping back or forward in 5-second increments. However, there are some advantages to using Apple’s implementation as well. The two main benefits, from my view, are these:

1. macOS’s PIP allows the PIP window to float across macOS full-screen “spaces”. Currently, Firefox’s PIP does not hover over full-screen apps, unless the full-screen app is Firefox itself.
2. macOS’s PIP allows users to hide PIP windows (while they are still playing) by swiping them into the side of the screen, after which a tab is displayed that allows the user to bring the PIP window back into view.

Some, including myself, use the “spaces” feature of macOS often. The non-native feel of Firefox’s PIP windows is a pain point that often results in me using Safari for watching videos rather than Firefox. That said, Firefox has the superior implementation and the increased flexibility of Firefox’s PIP shouldn’t be thrown out.

There ought to be a setting that allows Firefox users to switch between the macOS-native and non-native versions of PIP.

To reproduce:

1. Open a macOS app full-screen (not Firefox)
2. Open youtube.com in Firefox and play a video
3. Open the PIP viewer for the video
4. Use Ctrl+arrow or a three-finger swipe to move to the full-screen app

Actual results:

The Firefox PIP window is not visible in the new space

Expected results:

With a setting applied, the Firefox PIP window should still be visible in the new space

Comment 1

[Tanner Davies]

I could probably tackle this. I am not familiar with Firefox’s codebase so it might take me a while and I’d ask several questions, but I have the capability to do this

Comment 2

[Tanner Davies]

Attachment: Native PIP (Quicktime) vs Firefox PIP

I've attached a short screen recording that highlights the difference between native PIP and Firefox PIP

Comment 3

[BugBot [:suhaib / :marco/ :calixte]]

The Bugbug bot thinks this bug should belong to the 'Toolkit::Picture-in-Picture' component, and is moving the bug to that component. Please correct in case you think the bot is wrong.

Comment 4

[Tanner Davies]

Hm, this may be a duplicate of bug #1724402

Comment 5

[Tanner Davies]

Molly, thanks for taking a look at this issue. Could I have this assigned to me?

I have a few questions for you (answer at your own leisure; I don't want to monopolize your time with something that isn't high priority for the Mozilla team):

1. I took a cursory look through some of the code. It appears that PIP is actually a browser extension for Firefox that comes bundled. Obviously, for security reasons, an extension couldn't directly access the system APIs such as the one macOS provides for native PIP. To implement this, one would need to add to one of the web extension APIs a method of doing this (which would involve adding the capability to the backend through a combination of Objective-C and Rust or C++ code and then add a wrapper in the extension API that exists for macOS only--carefully so not to pose a security risk by exposing too much via the API). Is there anyone in particular that I would need to involve that needs to give a thumbs up to make changes to one of the APIs available to extensions?
2. I assume there is some sort of process for adding a setting to about:preferences that would involve a product manager. I suspect that a flag could be added to about:config more easily (i.e. without needed to be deliberated about by a product manager). Is there anyone I would need to talk to in order to be allowed to add a flag to about:config? Should I talk to someone to kick off a deliberation process about adding a setting to about:preferences?
3. Are you, as the triage owner, the correct person to be asking things like this to? As I mentioned in another comment, is not a project I've worked extensively on before so I don't know how project leadership is structured.

Thanks for your time and efforts, Molly!

Comment 6

[Molly Howell (she/her) (mostly inactive)]

1. It's a little more complicated than that, there are parts of the PIP feature spread across multiple layers. The extension is the highest layer, where the site-specific code lives; you're not interested in that for this bug, you'll need to be in the lower levels. The highest of those is here where the command to create the PIP window originates. As you'd guess, that call eventually ends up down in the Cocoa platform layer, where a native NSWindow is instantiated. I'm not all that familiar with this code, my specialty is more MS Windows, so any questions about this stuff will probably have to go to #macdev on chat.mozilla.org. You'll also find the people who would need to review changes to that code there.

2. Your guess is pretty much correct. There's really no process or approvals needed at all to add an item to about:config. To make the item appear in about:config with a default value, just add an entry for it here, next the existing ones for PIP. Reading the value would then work the same as any of the others. Now, to create a toggle in about:preferences to associate with that, there would indeed be some more process involved, but we'll cross that bridge when we get to it.

3. Firefox is a complex project with multiple different governance systems in play, so having questions about it is totally fair. I'm one of the module owners for picture-in-picture (somewhat confusingly, that's distinct from "triage lead", but the two often overlap), so I'm a good starting point of contact for technical or engineering questions like this. Probably a better starting point though is our Matrix channel, #picture-in-picture (again on chat.mozilla.org); there's a bunch of people in there who know what they're talking about, not just me. You'll find a number of other channels there relevant to Firefox development as well; #fx-desktop-dev is often a good jumping off point.

I've assigned the bug to you as requested. Thanks so much for being willing to dive into this! Don't hesitate to reach out with questions.

Comment 7

[Tanner Davies]

Thank you, Molly! That is incredibly helpful! It may be a couple weeks before I get to this, but I look forward to completing this issue.

Comment 8

[BugBot [:suhaib / :marco/ :calixte]]

The bug assignee is inactive on Bugzilla, so the assignee is being reset.