1524091 - GeckoView API to prompt the user when a site wants to play EME content

Status: RESOLVED DUPLICATE of bug 1620102

(GeckoView :: General, enhancement, P1)

Description

[Chris Peterson [:cpeterson]]

GV will need an API to notify the app when a site wants to play EME, so the app can asynchronously prompt the user for permission. The app will need to call back into GV to either allow or deny the EME content playback.

TBD whether the GV team will need help from the Gecko Media Playback team to hook into EME.

This GV API will be used by FFTV's EME UI:
https://github.com/mozilla-mobile/firefox-tv/issues/1698

Comment 1

[David Bolter [:davidb] (NeedInfo me for attention)]

Andreas, what is the urgency on this one? I think you said we don't need it soon... and I'd like to give it a priority rating.

Comment 2

[athomas]

To provide some extra info, this is needed for Firefox for Fire TV so that we can play paid content.

Comment 3

[Chris Peterson [:cpeterson]]

Like Ashley said, this is a requirement for GV in FFTV, but is explicitly out of scope for Fenix MVP and Fennec. We will need different UX/UI for Fenix and we will never ship EME on Fennec.

Comment 4

[Chris Peterson [:cpeterson]]

The EME permission request API could build on the site permissions bug 1524489.

Comment 5

[Chris Peterson [:cpeterson]]

Site permissions bug 1524489 was WONTFIX'd we can probably use GV's existing Permissions Delegate API to ask the app to display the EME UI when necessary.

Comment 6

[Andreas Bovens [:abovens]]

I'd like to see a bit of UX investigation here, so we know that providing this GV API is sufficient; perhaps more / something else is needed?

Comment 7

[Chris Peterson [:cpeterson]]

(In reply to Andreas Bovens [:abovens] from comment #6)

I'd like to see a bit of UX investigation here, so we know that providing this GV API is sufficient; perhaps more / something else is needed?

I think this EME permission API will be adequate. Gecko will use this GV API to ask the app whether it can play EME content on the current site (similar to the autoplay permission). The app is responsible for all the policy decisions, state, and UI like the permission notification or prompt and the "Play DRM-controlled content" setting.

For a detailed breakdown of the UI requirements and design mockups, see this doc:

https://docs.google.com/document/d/1O7mPFSW2AO2MxWM1t3CwHrwPEJ7THxHjgXWUG74JBAc/edit#heading=h.8wv8zg2ybxre

Comment 8

[Chris Peterson [:cpeterson]]

Randall said he will look into the GV API for EME after he completes GeckoVRManager bug 1529101.

Gecko's EME code triggers an address bar icon or door hanger on Firefox desktop. GV will need to use something like our permission delegate API to ask the FFTV front end for user permission. I don't know how tightly coupled Gecko's EME code is to the desktop address bar code. We might need to abstract the desktop and mobile EME notifications.

At this time, FFTV is the primary target. Fenix support can come later (bug 1340169). We won't need to support EME on Fennec..

Long doc on Android EME requirements and UX flow:
https://docs.google.com/document/d/1O7mPFSW2AO2MxWM1t3CwHrwPEJ7THxHjgXWUG74JBAc/edit

Shorter summary with UI design mockups:
https://docs.google.com/document/d/1rTSB8Aj5HPqSzrXtYMiOFaQvLOiuyxGRI_eszyak-wY/edit

Comment 9

[Randall Barker [:rbarker]]

@cpearce I've started investigating adding API support for EME into GeckoView. My understanding is that the android implementation needs to support asking for user permission before playing a video due to the fact that on Android, a unique user identifiable piece of data must be shared with the server. Is this actually the case? I started looking at dom/media/eme/MediaKeySystemAccessManager.cpp but did not see anything that supported this use case.

Comment 10

[Chris Pearce [:cpearce (Not reading bugmail)]]

@rbarker: Our understanding is that each device has a factory provisioned unique id baked into it, which is shared by the CDM with the license server via an encrypted message which passes through the EME JS APIs. So yes, we believe it's necessary to warn the user that they're effectively disclosing a super-cookie when they enable DRM.

Comment 11

[David Bolter [:davidb] (NeedInfo me for attention)]

My understanding is the next step here is to have a mechanism for GeckoView to be notified of EME content (every time), and a way for permission to be denied (via return value or something).

GV will incorporate this into our permission delegate and communicate with the embedding app.

Nils could your team handle the first bit?

Comment 12

[Nils Ohlmeier [:drno]]

Bryce, could you start to have a look into this?

Comment 13

[Bryce Seager van Dyk [:bryce] (he/him) - Not reading bugmail]

Moving this and associated work to top of me queue.

Comment 14

[Chris Peterson [:cpeterson]]

I'm editing a bunch of GeckoView bugs. If you'd like to filter all this bugmail, search and destroy emails containing this UUID:

e88a5094-0fc0-4b7c-b7c5-aef00a11dbc9

Comment 15

[Philip Lamb]

This is P2 for Firefox Reality in our current milestone (which is aligned with GV70 milestones).

Comment 16

[Philip Lamb]

Bryce, do you have a bug tracking the work required in Gecko to add the necessary hooks and notifications? As well as hooks for Geckoview, we need similar hooks to disable EME in certain desktop VR sessions and I'd like to link to the right bug.

Comment 17

[Bryce Seager van Dyk [:bryce] (he/him) - Not reading bugmail]

(In reply to Philip Lamb from comment #16)

Bryce, do you have a bug tracking the work required in Gecko to add the necessary hooks and notifications? As well as hooks for Geckoview, we need similar hooks to disable EME in certain desktop VR sessions and I'd like to link to the right bug.

I don't think bugs tracking those have been created yet.

Making sure everyone is on the same page: my priority has shifted away from this since comment 13 was made, and it's not something I'm working on at the moment. I could return to this, and it so, it may be useful to have someone help me get up to speed with regards to where the gecko surface is contacting the other components that need these changes.

Comment 18

[Philip Lamb]

The need here is for some hook or delegate so that when a media element is loaded which requires EME, playback does not proceed until the hook returns some PLAYBACK_ALLOWED result. A second part would be a set of notifications that EME playback is starting or has stopped; these would be useful for UI (e.g. for showing some badge or other UI during EME playback).

I'm not overly familiar with implementation details, but looking at the current Geckoview implementation, I can see https://dxr.mozilla.org/mozilla-central/source/mobile/android/chrome/geckoview/GeckoViewMediaChild.js#174 which is called on the UAWidgetSetupOrChange message and which then checks whether the widget is media, and which then sets up messages for the Geckoview client.

@Randall, do you have any additional comments?

We have a similar need for a hook to allow/deny EME playback in the Firefox Reality for Windows project, but of course the listener on the hook would be different (i.e. not Geckview). @Kip can provide more detail on that.

Comment 19

[Randall Barker [:rbarker]]

I think going through the PermissionDelegate might be the better route. GV would need some sort of asynchronous API that allows the embedder to query the user for permission then relay the results back to the media layer.

Comment 20

[:kip (Kearwood Gilbert)]

For the FxR Desktop side, we wouldn't need to break out such a hook in GV, but would effectively set a global flag (Accessible from the VRManager singleton in the GPU process) indicating if EME should be blocked. This flag would reflect the status of the protected media pathway, including any VR hardware components that may be involved. Checking for this flag could be a synchronous operation, unlike the possibly more advanced needs for the GV interface.

Comment 21

[Chris Peterson [:cpeterson]]

(In reply to Philip Lamb from comment #15)

This is P2 for Firefox Reality in our current milestone (which is aligned with GV70 milestones).

In that case, setting status flag firefox70=affected and firefox69=wontfix for this GV API.

The need here is for some hook or delegate so that when a media element is loaded which requires EME, playback does not proceed until the hook returns some PLAYBACK_ALLOWED result. A second part would be a set of notifications that EME playback is starting or has stopped; these would be useful for UI (e.g. for showing some badge or other UI during EME playback).

This might be a disruptive core change if the Gecko EME code was written assuming desktop Firefox's EME playback check was just a synchronous pref check of "media.eme.enabled". The Android EME project plan has more details on some of the GV EME APIs that may be needed:

https://docs.google.com/document/d/1O7mPFSW2AO2MxWM1t3CwHrwPEJ7THxHjgXWUG74JBAc/edit#heading=h.u0d82h9snun6

Comment 22

[Chris Peterson [:cpeterson]]

We want to work on this in October. Randall is talking with the Gecko Media team about the changes needed to Gecko EME code to support asynchronous permission prompts for GV.

Comment 23

[Chris Peterson [:cpeterson]]

Don't roll over to November sprint because we're blocked waiting for some Gecko EME changes, but keep EME a high priority P2.

Comment 24

[David Bolter [:davidb] (NeedInfo me for attention)]

I think this is a priority for fenix GA.

Comment 25

[Emily Toop (:fluffyemily)]

We are waiting for the Media team to land bug 1587522 before we can work on this issue. As soon as that bug lands then we can prioritise this in the GV team.

Comment 26

[Randall Barker [:rbarker]]

Oops, don't know why I couldn't find this when I looked. I created a new issue and already have a patch posted and reviewed so I'll close this one as a duplicate.