1704278 - InvalidStateError: "getDisplayMedia must be called from a user gesture handler" error in cross-origin iframe with allow="camera; microphone; display-capture"

Status: RESOLVED FIXED

(Core :: WebRTC: Audio/Video, defect, P1)

Description

[C.Bartelmus]

User Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:87.0) Gecko/20100101 Firefox/87.0

Steps to reproduce:

This will happen when api.executeCommand('toggleShareScreen'); is called in Jitsi meet IFrame API (https://jitsi.github.io/handbook/docs/dev-guide/dev-guide-iframe#togglesharescreen). The IFrame Jitsi is using has allow="camera; microphone; display-capture; autoplay; clipboard-write" set.
The API call is triggered during a click event.
Clicking inside the IFrame first and then calling the API does work.
Internally obtainScreenFromGetDisplayMedia is called (cf. https://github.com/jitsi/lib-jitsi-meet/blob/2a2405e93614efd43d4db62f51399f5a870a1615/modules/RTC/ScreenObtainer.js)

Actual results:

InvalidStateError: "getDisplayMedia must be called from a user gesture handler"

Expected results:

Call to getDisplayMedia is allowed because call is made following a click event and the iframe is using allow="display-capture".

Comment 1

[BugBot [:suhaib / :marco/ :calixte]]

The Bugbug bot thinks this bug should belong to the 'Core::WebRTC: Audio/Video' component, and is moving the bug to that component. Please revert this change in case you think the bot is wrong.

Comment 2

[Paul Adenot (:padenot)]

Jan-Ivar, sounds like we're not propagating the fact that this is a click handler.

Reporter, do you happen to have a test-case to repro this, or maybe an existing site where we can repro ?

Comment 3

[C.Bartelmus]

(In reply to Paul Adenot (:padenot) from comment #2)

Jan-Ivar, sounds like we're not propagating the fact that this is a click handler.

Reporter, do you happen to have a test-case to repro this, or maybe an existing site where we can repro ?

Sorry, I can't provide you a link at this time.
Just in case it didn't become obvious by the initial issue descripiton: the data flow is "click handler in main document" -> "iframe.contentWindow.postMessage()" -> "getDisplayMedia inside iframe"

Comment 4

[Jan-Ivar Bruaroey [:jib] (needinfo? me)]

Jan-Ivar, sounds like we're not propagating the fact that this is a click handler.

Note that starting in 86, gDM requires transient activation as per spec (bug 1679735), which is time-based, though that should still work.

the data flow is "click handler in main document" -> "iframe.contentWindow.postMessage()" -> "getDisplayMedia inside iframe"

Hi Bartelmus, thanks for reporting and clarifying! — I've tried to replicate this situation in https://jsfiddle.net/jib1/yu04p8x1/ but it WFM.

It also appears to work from a button inside the iframe https://jan-ivar.github.io/dummy/iframe_gdm.html

If you're able to modify either of these fiddles to repro the problem, that would be helpful, as I don't have the Jitsi meet IFrame API to play with. Thanks!

Comment 5

[C.Bartelmus]

Hi!
The difference to your example is that the main document and the iframe are on different servers. I've just verified that everything is working as expected when all is on the same server.

Comment 6

[Jan-Ivar Bruaroey [:jib] (needinfo? me)]

Thanks Bartelmus for that info!

Karl, do you happen to know whether transient activation is defined to work across origins, or whom to ask that question?

Comment 7

[Karl Tomlinson (:karlt)]

"When a user interaction in a browsing context B causes firing of an activation triggering input event in B's active document D", the only "descendant browsing contexts of D" that get "last activation timestamp" set are those "that have active documents from the same origin as that of D."

So transient activation only crosses origins into ancestor browsing contexts.
For getDisplayMedia() in a cross-origin iframe, the spec requires user interaction in that iframe or one of its descendant browsing contexts.

Comment 8

[C.Bartelmus]

I guess this leaves us in a situation where this can't be fixed?
I can't submit this as a bug in the Jitsi framework because as far as I understand it, they have no way to solve this.
And even allow="display-capture" won't help? A gap in the spec?
I'm guess this will break some existing implementations.
Anyway, thanks for your time!

Comment 9

[Tom S [:evilpie]]

We should definitely submit a bug to Jitsi so they are at least aware of this problem.

Comment 10

[Karl Tomlinson (:karlt)]

(In reply to [e-mail] from comment #8)

I'm guess this will break some existing implementations.

Was getDisplayMedia() allowed in your situation prior to Firefox 86?
IIUC earlier versions would have still required a user interaction in the cross-origin iframe, but it would have been sticky.

I guess this leaves us in a situation where this can't be fixed?

I haven't tested, but if the different servers are subdomains of the same site, then I expect Document.domain could be considered as a workaround, but that would also have other security implications.

Comment 11

[C.Bartelmus]

(In reply to Karl Tomlinson (:karlt) from comment #10)

(In reply to [e-mail] from comment #8)

I'm guess this will break some existing implementations.

Was getDisplayMedia() allowed in your situation prior to Firefox 86?

Yes, I've tested with Firefox 83 Linux: it's working.

IIUC earlier versions would have still required a user interaction in the cross-origin iframe, but it would have been sticky.

I guess this leaves us in a situation where this can't be fixed?

I haven't tested, but if the different servers are subdomains of the same site, then I expect Document.domain could be considered as a workaround, but that would also have other security implications.

Unfortunately not an option for me.

Comment 12

[Edgar Chen [:edgar] (Parental leave until Oct. 09.)]

(In reply to [e-mail] from comment #3)

Sorry, I can't provide you a link at this time.
Just in case it didn't become obvious by the initial issue descripiton: the data flow is "click handler in main document" -> "iframe.contentWindow.postMessage()" -> "getDisplayMedia inside iframe"

There is a discussion about "transferring activation to another frame", https://github.com/whatwg/html/issues/4364.
I guess Chrome has implemented a similar thing, but it's not clear yet at the spec level.

Comment 13

[Karl Tomlinson (:karlt)]

Attachment: Bug 1704278 revert from transient to sticky activation requirement for getDisplayMedia() r?jib

Interim measure for the sake of postMessage() to cross-origin iframe with
allow="display-capture".

This reverts the behavioral change of
https://hg.mozilla.org/mozilla-central/rev/3faff9d75a5d#l1.35

Comment 14

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/7a50bf09d726
revert from transient to sticky activation requirement for getDisplayMedia() r=jib

Comment 15

[Jan-Ivar Bruaroey [:jib] (needinfo? me)]

[Tracking Requested - why for this release]: Breaks screen-sharing in the Jitsi meet API, a popular open source video conferencing solution.

Workaround: "Clicking inside the IFrame first and then calling the API does work." (from comment 0)

Comment 16

[Andreea Pavel [:apavel]]

https://hg.mozilla.org/mozilla-central/rev/7a50bf09d726

Comment 17

[Ryan VanderMeulen [:RyanVM]]

Not a new issue and doesn't seem to meet the criteria for a dot release. Unless you feel strongly otherwise, let's let this fix ride Fx89.

Comment 18

[Daniel Bodea [:danibodea]]

I would like to verify the fix of this issue, but I don't see a clear way to reproduce it. Karl, can you help me?

Comment 19

[Karl Tomlinson (:karlt)]

Reproduction would require access to servers that we don't have and would require some set up.
The automated test does this, and so manual verification is not required thanks.

Comment 20

[C.Bartelmus]

(In reply to Karl Tomlinson (:karlt) from comment #19)

Reproduction would require access to servers that we don't have and would require some set up.
The automated test does this, and so manual verification is not required thanks.

Issue is still present in Firefox 89.0. Did the change make it into this release?

Comment 21

[Karl Tomlinson (:karlt)]

Yes, the change shipped with the 89.0 release.

A user activation (e.g. click) is still required in the iframe, but this now need not be immediately prior to the getDisplayMedia() call.
This is again the same behavior as was expected in 83.0.

Comment 22

[C.Bartelmus]

(In reply to Karl Tomlinson (:karlt) from comment #21)

Yes, the change shipped with the 89.0 release.

A user activation (e.g. click) is still required in the iframe, but this now need not be immediately prior to the getDisplayMedia() call.
This is again the same behavior as was expected in 83.0.

From user perspective nothing has changed, because the error is still thrown. Status RESOLVED is misleading.

Comment 23

[Karl Tomlinson (:karlt)]

Comment 11 said "I've tested with Firefox 83 Linux: it's working."
Is there a remaining difference in behavior from Firefox 83?

Comment 24

[C.Bartelmus]

(In reply to Karl Tomlinson (:karlt) from comment #23)

Comment 11 said "I've tested with Firefox 83 Linux: it's working."
Is there a remaining difference in behavior from Firefox 83?

I've tested it again with Firefox 83. I also get the InvalidStateError: "getDisplayMedia must be called from a user gesture handler". It's not working in Firefox 83 despite what I said before. I'm really not sure why it was working in my previous test. Sorry for the confusion.