1679817 - Gamepad API not counted as User Event, causing Video playing muted, marked as AutoPlay.

Status: ASSIGNED

(Core :: DOM: Device Interfaces, defect, P3)

Description

[w1bi]

User Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:84.0) Gecko/20100101 Firefox/84.0

Steps to reproduce:

I made a game that use gamepad api from the menu until the game itself, so mouse and keyboard interaction is not needed after the user open our website. But the problem happens when the game running a video.

The video will be muted because Gamepad Button (or gamepadconnected event) not counted as User Interaction, so the browser thinks it's an AutoPlay Video causing it muted on start.

The workaround is by clicking the browser first before the video plays, but we don't really like forcing user to move from controller to mouse, or telling to not blocking the autoplay for our website.

Weirdly, this works well on Chrome, the video plays normally and not muted without any clicking or pressing keyboard. It did not work on any version of Firefox. I used 83 and Developer Edition 84.

Actual results:

Video is muted because browser thinks there is no user interaction and counted as an autoplay video .

Expected results:

Video will play normally and not muted.

Comment 1

[BugBot [:suhaib / :marco/ :calixte]]

Bugbug thinks this bug should belong to this component, but please revert this change in case of error.

Comment 2

[Bryce Seager van Dyk [:bryce] (he/him) - Not reading bugmail]

Do you have a page we could visit to test this on?

Alastor, do you have insight into this? Is this something that media should fix, or will we need to deal with this elsewhere (like in the gamepad API)?

Comment 3

[w1bi]

Hi, yes you can test it on https://demo.rabbitsky.io/, when the menu load, press Button 0 (X for ps4 controller) on Join The Sky. Currently its using twitch embed video.

Well, technically it's not a game, but I called it that way to be simpler. Also, this is kinda unreleased project and the gamepad support still in beta. But it works ok right now.

Comment 4

[Alastor Wu [:alwu]]

When browsing GamePad API paga on MDN, I saw this paragraph,

In Firefox, gamepads are only exposed to a page when the user interacts with one with the page visible. This helps prevent gamepads being used for fingerprinting the user. Once one gamepad has been interacted with, other gamepads that are connected will automatically be visible.

Our blocking autoplay mechanism relies on the user activation, but from above description, it seems we don't want to use gamepad as an activation gesture because of fingerprinting issue?

---

Hi, edgar,
Do you have any idea that if we should take GamePad event as an user activation gesture?
Thank you.

Comment 5

[Edgar Chen [:edgar]]

I think it makes sense that treats GamePad input as a user gesture, spec seems to try to define it, https://github.com/w3c/gamepad/pull/107 and https://github.com/w3c/gamepad/issues/124.
I did some quick research on how Chrome handles this, found https://source.chromium.org/chromium/chromium/src/+/master:third_party/blink/renderer/modules/gamepad/navigator_gamepad.cc;l=184;drc=828932702391775d9b4696c8a5f67f749690fd39, but it a bit weird that update user activation state in navigator.getGamepads(), I think it might make more sense that update based on gamepadconnected event, and maybe gamepadbuttondown event which is is a firefox-only event.

(In reply to Alastor Wu [:alwu] from comment #4)

Our blocking autoplay mechanism relies on the user activation, but from above description, it seems we don't want to use gamepad as an activation gesture because of fingerprinting issue?

I think that is fine we don't expose GamePad state or fire event at all when RFP is enabled, we just need to ensure we won't update user activation state when RFP is enabled.