Closed Bug 1452536 Opened 2 years ago Closed 2 years ago

Don't gesture activate documents upon interaction with the browser/OS

Categories

(Core :: DOM: Events, enhancement, P2)

Product:
Core
Component:
DOM: Events
Type:
enhancement

Tracking

()

Status:
RESOLVED FIXED
Milestone:
mozilla61
Tracking Flags:
Tracking Status
firefox61 --- fixed

People

(Reporter: cpearce, Assigned: cpearce, NeedInfo)

References

(Blocks 1 open bug)

Details

Attachments

(3 files)

Bug 1452536 - Don't gesture activate documents on non-printable key events.
59 bytes, text/x-review-board-request
masayuki
: review+
Details
Bug 1452536 - Test that key events for non-printable keys and interaction with editable elements don't unblock autoplay.
59 bytes, text/x-review-board-request
masayuki
: review+
Details
Bug 1452536 - Don't gesture activate for events targetting editable elements.
59 bytes, text/x-review-board-request
masayuki
: review+
Details 
Currently we gesture activate a document whenever there's a key up event:
https://searchfox.org/mozilla-central/rev/7ccb618f45a1398e31a086a009f87c8fd3a790b6/dom/events/EventStateManager.cpp#826

This results in an unpleasantly surprising user experience, as keyboard shortcuts that the user intended to direct at the browser or operating system gesture activate the document.

For example, keyboard shortcuts like Ctrl+Tab, or Alt+Tab, Ctrl+R, etc and scrolling keys like PageUp/Down, and cursor Up/Down, should not be considered interaction with the page, and so shouldn't gesture activate the document.

So I think we need to restrict gesture activation via key events to key press events for printable characters.
Did you know actual problem which caused by your explanation?

Anyway, I don't understand why we need to mark the document active with keyup events. keydown/keypress/keyup events are fired on focused document *when* PresShell dispatches them into a DOM tree. So, if focus is changed while dispatching keydown/keypress events, following keyup event may not be intended by the user.

If we need to activate document at keyup rather than keydown, how about to check if last keydown event which is not marked as auto-repeated is fired in the document? It's possible to check easily with storing last keydown event fired ESM with a static variable.
Flags: needinfo?(cpearce)
(In reply to Masayuki Nakano [:masayuki] (JST, +0900) from comment #4)
> Did you know actual problem which caused by your explanation?

I tried to explain in the first patch's commit message. The current block-autoplay strategy of unblocking autoplay upon keyboard interaction with the document is not good, because if the user uses a keyboard shortcut, they'll unexpectedly unblock autoplay. For example, of you press CTRL+Tab to switch tab, you activate the document as you leave. If you scroll with ArrowUp/Down, you activate the document. This is annoying, and so we want to restrict gesture activation to keypresses that are not likely to be intended as interaction with the browser or OS.


> Anyway, I don't understand why we need to mark the document active with
> keyup events. keydown/keypress/keyup events are fired on focused document
> *when* PresShell dispatches them into a DOM tree. So, if focus is changed
> while dispatching keydown/keypress events, following keyup event may not be
> intended by the user.
> 
> If we need to activate document at keyup rather than keydown[...]

I took over this work from alwu after the Taipei layoffs. I don't know why alwu chose keyup over keydown. Activating on keydown seems fine to me. What about activating on keypress?
Flags: needinfo?(cpearce)
Flags: needinfo?(masayuki)
(In reply to Chris Pearce (:cpearce) from comment #5)
> (In reply to Masayuki Nakano [:masayuki] (JST, +0900) from comment #4)
> > Did you know actual problem which caused by your explanation?
> 
> I tried to explain in the first patch's commit message. The current
> block-autoplay strategy of unblocking autoplay upon keyboard interaction
> with the document is not good, because if the user uses a keyboard shortcut,
> they'll unexpectedly unblock autoplay. For example, of you press CTRL+Tab to
> switch tab, you activate the document as you leave. If you scroll with
> ArrowUp/Down, you activate the document. This is annoying, and so we want to
> restrict gesture activation to keypresses that are not likely to be intended
> as interaction with the browser or OS.

Okay, I got it.

> > Anyway, I don't understand why we need to mark the document active with
> > keyup events. keydown/keypress/keyup events are fired on focused document
> > *when* PresShell dispatches them into a DOM tree. So, if focus is changed
> > while dispatching keydown/keypress events, following keyup event may not be
> > intended by the user.
> > 
> > If we need to activate document at keyup rather than keydown[...]
> 
> I took over this work from alwu after the Taipei layoffs. I don't know why
> alwu chose keyup over keydown.

I see... It *might* be that if keydown activates the document, shortcut key or something which is handled before keyup event might be broken.

> Activating on keydown seems fine to me. What
> about activating on keypress?

I don't think that we should active document with keypress since if keydown changes focus, keypress may also be fired on unexpected document.

However, if you want to minimize risk of this change, I recommend to active the document at keyup but add check if preceding keydown event is fired on same document. Then, we can avoid the whitelist from our code.
Flags: needinfo?(masayuki) → needinfo?(cpearce)
(In reply to Masayuki Nakano [:masayuki] (JST, +0900) from comment #6)
> However, if you want to minimize risk of this change, I recommend to active
> the document at keyup but add check if preceding keydown event is fired on
> same document.

Ok. That makes sense. I will add a changeset to the push to check that the preceding keydown was fired on the same document. Thanks!

> Then, we can avoid the whitelist from our code.

I want to clarify this statement. Are you saying we can remove the blacklist? Irrespective of what key event we activate documents on, I think we still need a blacklist, as we don't want keys intended to be interaction with the browser to activate the document.
Flags: needinfo?(cpearce)
(In reply to Chris Pearce (:cpearce) from comment #7)
> (In reply to Masayuki Nakano [:masayuki] (JST, +0900) from comment #6)
> > Then, we can avoid the whitelist from our code.
> 
> I want to clarify this statement. Are you saying we can remove the
> blacklist? Irrespective of what key event we activate documents on, I think
> we still need a blacklist, as we don't want keys intended to be interaction
> with the browser to activate the document.

Hmm, I don't think that we should check specific key combinations for such purpose since key bindings depend on *current* implementation of both Firefox and OSes. Although, it's okay to ban a couple of key combinations with blacklist, but current your patch is too big and banning Shift + something which is used a lot, for example, typing upper case letters, scrolling with Shift + Space, etc.
(In reply to Masayuki Nakano [:masayuki] (JST, +0900) from comment #8)
> (In reply to Chris Pearce (:cpearce) from comment #7)
> > (In reply to Masayuki Nakano [:masayuki] (JST, +0900) from comment #6)
> > > Then, we can avoid the whitelist from our code.
> > 
> > I want to clarify this statement. Are you saying we can remove the
> > blacklist? Irrespective of what key event we activate documents on, I think
> > we still need a blacklist, as we don't want keys intended to be interaction
> > with the browser to activate the document.
> 
> Hmm, I don't think that we should check specific key combinations for such
> purpose since key bindings depend on *current* implementation of both
> Firefox and OSes.

The Block Autoplay UX spec that has been developed by the team working on this calls for keypresses that are intended as interaction with the browser or OS to not unblock autoplay:
https://mozilla.invisionapp.com/share/N8F2K0I93#/screens/287176969

If we don't blacklist key combinations, then how do you propose we achieve the goal of preventing key combinations that are intended as interaction with the browser and OS from activating a document?

Is your complaint here that we're embedding Firefox behaviour inside Gecko? Would you prefer us to store the list of blacklisted key combinations inside some sort of bundled file which is part of the Firefox chrome code?
(In reply to Masayuki Nakano [:masayuki] (JST, +0900) from comment #6)
> > I took over this work from alwu after the Taipei layoffs. I don't know why
> > alwu chose keyup over keydown.
> 
> I see... It *might* be that if keydown activates the document, shortcut key
> or something which is handled before keyup event might be broken.
> 
> > Activating on keydown seems fine to me. What
> > about activating on keypress?
> 
> I don't think that we should active document with keypress since if keydown
> changes focus, keypress may also be fired on unexpected document.
> 
> However, if you want to minimize risk of this change, I recommend to active
> the document at keyup but add check if preceding keydown event is fired on
> same document. Then, we can avoid the whitelist from our code.

Having looked into this bit further, by the time the keydown event handler runs in Chrome, the document is activated. So to be compatible with Chrome we need to have activated the document before the keydown handler runs, we can't wait until the keyup event.

For example, the following JS code:
(live: http://pearce.org.nz/video/autoplay-play-in-key-events.html )

  ["keypress", "keydown", "keyup"].forEach(
      function(e) {
        window.addEventListener(e, async function(event){
          let played;
          try {
            let x = await video.play();
            played = true;
          } catch (e) {
            played = false;
          }
          logEvent(event.type + " " + event.key + " played=" + played);
        }, false);
      });

When run in Chrome Unstable (which has autoplay blocked by default), if you press the 'p' key, this logs:

keydown p played=true
keypress p played=true
keyup p played=true

So Chrome is activating before their keydown handler runs.

Whereas because we're activating in the keyup event, we'll refuse to play in the keydown handler, so we'll log:

keydown p played=false
keypress p played=false
keyup p played=true


So I think we should change the behaviour here to activate on keydown, and not on keyup, otherwise we'll not be compatible with Chrome.

Masayuki: does that make sense to you? Thanks!
Flags: needinfo?(masayuki)
A similar test for mouse events shows that Chrome is also activating before dispatching mousedown, not mouseup like we do. So we should also change our gesture activation from mouseup to mousedown.
(In reply to Chris Pearce (:cpearce) from comment #9)
> (In reply to Masayuki Nakano [:masayuki] (JST, +0900) from comment #8)
> > (In reply to Chris Pearce (:cpearce) from comment #7)
> > > (In reply to Masayuki Nakano [:masayuki] (JST, +0900) from comment #6)
> > > > Then, we can avoid the whitelist from our code.
> > > 
> > > I want to clarify this statement. Are you saying we can remove the
> > > blacklist? Irrespective of what key event we activate documents on, I think
> > > we still need a blacklist, as we don't want keys intended to be interaction
> > > with the browser to activate the document.
> > 
> > Hmm, I don't think that we should check specific key combinations for such
> > purpose since key bindings depend on *current* implementation of both
> > Firefox and OSes.
> 
> The Block Autoplay UX spec that has been developed by the team working on
> this calls for keypresses that are intended as interaction with the browser
> or OS to not unblock autoplay:
> https://mozilla.invisionapp.com/share/N8F2K0I93#/screens/287176969
> 
> If we don't blacklist key combinations, then how do you propose we achieve
> the goal of preventing key combinations that are intended as interaction
> with the browser and OS from activating a document?

Hmm, okay, for conforming to the document, it doesn't make sense to listen keyup events since we cannot distinguish if the key operation is for inputting some characters or not. Currently, we can know that only with keypress event.  If keypress event's all of ctrlKey, altKey and metaKey are false and charCode is not 0, the key operation is for inputting characters.

Note that following key events could occur, in this case, your patch doesn't work as expected:

1. Pressing Ctrl key:  keydown for Control.
2. Pressing R key:     keydown for R and keypress for R whose ctrlKey is true.
3. Releasing Ctrl key: keyup for Control.
4. Releasing R key:    keyup for R whose ctrlKey is false.

> Is your complaint here that we're embedding Firefox behaviour inside Gecko?
> Would you prefer us to store the list of blacklisted key combinations inside
> some sort of bundled file which is part of the Firefox chrome code?

Not so. I just want to avoid including blacklist/whitelist which need to maintain.

(In reply to Chris Pearce (:cpearce) from comment #10)
> (In reply to Masayuki Nakano [:masayuki] (JST, +0900) from comment #6)
> > > I took over this work from alwu after the Taipei layoffs. I don't know why
> > > alwu chose keyup over keydown.
> > 
> > I see... It *might* be that if keydown activates the document, shortcut key
> > or something which is handled before keyup event might be broken.
> > 
> > > Activating on keydown seems fine to me. What
> > > about activating on keypress?
> > 
> > I don't think that we should active document with keypress since if keydown
> > changes focus, keypress may also be fired on unexpected document.
> > 
> > However, if you want to minimize risk of this change, I recommend to active
> > the document at keyup but add check if preceding keydown event is fired on
> > same document. Then, we can avoid the whitelist from our code.
> 
> Having looked into this bit further, by the time the keydown event handler
> runs in Chrome, the document is activated. So to be compatible with Chrome
> we need to have activated the document before the keydown handler runs, we
> can't wait until the keyup event.
>
> ...
> 
> So I think we should change the behaviour here to activate on keydown, and
> not on keyup, otherwise we'll not be compatible with Chrome.
> 
> Masayuki: does that make sense to you? Thanks!

Yeah, it must be better to behave exactly same as Chrome. On the other hand, we don't have any flag whether keydown event is followed by keypress event which will cause inputting text. However, perhaps, ignoring only a few special cases, e.g., cases to input a character with Ctrl + foo, we can test as following simple code:

if (!keyEvent ||
    !keyEvent->PseudoCharCode() ||
    (keyEvent->IsControl() && !keyEvent->IsAltGraph()) ||
    (keyEvent->IsAlt() && !keyEvent->IsAltGraph()) ||
    keyEvent->IsMeta() ||
    keyEvent->IsOS()) {
  return;
}
// do something.

If you need to allow to keydown event which will cause compositionstart, you need to allow |key->mKayNameIndex == KEY_NAME_INDEX_Process| even if any other members are true. "Process" key means that the key combination is consumed by IME.

(In reply to Chris Pearce (:cpearce) from comment #11)
> A similar test for mouse events shows that Chrome is also activating before
> dispatching mousedown, not mouseup like we do. So we should also change our
> gesture activation from mouseup to mousedown.

Yeah, we should change it in another bug.
Flags: needinfo?(masayuki)
(In reply to Masayuki Nakano [:masayuki] (JST, +0900) from comment #12)
> we can test
> as following simple code:
> 
> if (!keyEvent ||
>     !keyEvent->PseudoCharCode() ||
>     (keyEvent->IsControl() && !keyEvent->IsAltGraph()) ||
>     (keyEvent->IsAlt() && !keyEvent->IsAltGraph()) ||
>     keyEvent->IsMeta() ||
>     keyEvent->IsOS()) {
>   return;
> }
> // do something.

This seems to give the behaviour we want, and is much simpler! Thank you!


> If you need to allow to keydown event which will cause compositionstart, you
> need to allow |key->mKayNameIndex == KEY_NAME_INDEX_Process| even if any
> other members are true. "Process" key means that the key combination is
> consumed by IME.

Just checking my understanding, we need to make the above test:

  WidgetKeyboardEvent* keyEvent = aEvent->AsKeyboardEvent();
  if (keyEvent && (!keyEvent->PseudoCharCode() ||
                   (keyEvent->IsControl() && !keyEvent->IsAltGraph()) ||
                   (keyEvent->IsAlt() && !keyEvent->IsAltGraph()) ||
                   keyEvent->IsMeta() || keyEvent->IsOS() ||
                   keyEvent->mKeyNameIndex == KEY_NAME_INDEX_Process)) {
    return;
  }



> (In reply to Chris Pearce (:cpearce) from comment #11)
> > A similar test for mouse events shows that Chrome is also activating before
> > dispatching mousedown, not mouseup like we do. So we should also change our
> > gesture activation from mouseup to mousedown.
> 
> Yeah, we should change it in another bug.

OK, thanks!
Flags: needinfo?(masayuki)
(In reply to Chris Pearce (:cpearce) from comment #13)
> (In reply to Masayuki Nakano [:masayuki] (JST, +0900) from comment #12)
> > If you need to allow to keydown event which will cause compositionstart, you
> > need to allow |key->mKayNameIndex == KEY_NAME_INDEX_Process| even if any
> > other members are true. "Process" key means that the key combination is
> > consumed by IME.
> 
> Just checking my understanding, we need to make the above test:
> 
>   WidgetKeyboardEvent* keyEvent = aEvent->AsKeyboardEvent();
>   if (keyEvent && (!keyEvent->PseudoCharCode() ||
>                    (keyEvent->IsControl() && !keyEvent->IsAltGraph()) ||
>                    (keyEvent->IsAlt() && !keyEvent->IsAltGraph()) ||
>                    keyEvent->IsMeta() || keyEvent->IsOS() ||
>                    keyEvent->mKeyNameIndex == KEY_NAME_INDEX_Process)) {
>     return;
>   }

Ah, no. If we need to allow to activate document with first key press even though it's consumed by IME, it should be:

if (!keyEvent ||
    (keyEvent->mKeyNameIndex != KEY_NAME_INDEX_Process &&
     (!keyEvent->PseudoCharChode() ||
      ...))) {
  return; // not activate.
}

I don't know whether Google Chrome allows such keydown events too since such keydown event is dispatched only in <input type="text">, <textarea> or <foo contenteditable>. It might be better to active document with compositionstart event instead if web apps won't try to start to play video at preceding keydown event of compositionstart.
Flags: needinfo?(masayuki)
(In reply to Masayuki Nakano [:masayuki] (JST, +0900) from comment #12)
> (In reply to Chris Pearce (:cpearce) from comment #11)
> > A similar test for mouse events shows that Chrome is also activating before
> > dispatching mousedown, not mouseup like we do. So we should also change our
> > gesture activation from mouseup to mousedown.
> 
> Yeah, we should change it in another bug.

Will change to gesture activating on key/mouse down in bug 1456037.
(In reply to Masayuki Nakano [:masayuki] (JST, +0900) from comment #14)
> (In reply to Chris Pearce (:cpearce) from comment #13)
> > (In reply to Masayuki Nakano [:masayuki] (JST, +0900) from comment #12)
> > > If you need to allow to keydown event which will cause compositionstart, you
> > > need to allow |key->mKayNameIndex == KEY_NAME_INDEX_Process| even if any
> > > other members are true. "Process" key means that the key combination is
> > > consumed by IME.
> > 
> > Just checking my understanding, we need to make the above test:
> > 
> >   WidgetKeyboardEvent* keyEvent = aEvent->AsKeyboardEvent();
> >   if (keyEvent && (!keyEvent->PseudoCharCode() ||
> >                    (keyEvent->IsControl() && !keyEvent->IsAltGraph()) ||
> >                    (keyEvent->IsAlt() && !keyEvent->IsAltGraph()) ||
> >                    keyEvent->IsMeta() || keyEvent->IsOS() ||
> >                    keyEvent->mKeyNameIndex == KEY_NAME_INDEX_Process)) {
> >     return;
> >   }
> 
> Ah, no. If we need to allow to activate document with first key press even
> though it's consumed by IME, it should be:
> 
> if (!keyEvent ||
>     (keyEvent->mKeyNameIndex != KEY_NAME_INDEX_Process &&
>      (!keyEvent->PseudoCharChode() ||
>       ...))) {
>   return; // not activate.
> }
> 
> I don't know whether Google Chrome allows such keydown events too since such
> keydown event is dispatched only in <input type="text">, <textarea> or <foo
> contenteditable>. It might be better to active document with
> compositionstart event instead if web apps won't try to start to play video
> at preceding keydown event of compositionstart.

Based discussions elsewhere, we want to not gesture activate for input events directed at editable elements. This would prevent typing using IME or in Latin/English character sets in editable inputs from gesture activating documents. This would also mean we don't need to check for KEY_NAME_INDEX_Process, as it should only be dispatched to editable target content, and we will be refusing to gesture activate on these anyway.
Comment on attachment 8967271 [details]
Bug 1452536 - Don't gesture activate documents on non-printable key events.

I think that you're updating the patches for this bug.
Attachment #8967271 - Flags: review?(masayuki)
Comment on attachment 8967271 [details]
Bug 1452536 - Don't gesture activate documents on non-printable key events.

https://reviewboard.mozilla.org/r/235958/#review246030
Attachment #8967271 - Flags: review?(masayuki) → review+
Comment on attachment 8971457 [details]
Bug 1452536 - Don't gesture activate for events targetting editable elements.

https://reviewboard.mozilla.org/r/240208/#review246038

Giving r+ since Japan starts holiday week and anyway the pointed issues are trivial.

::: dom/events/EventStateManager.cpp:911
(Diff revision 1)
> +IsTextInput(nsIContent* aContent)
> +{
> +  MOZ_ASSERT(aContent);
> +  nsCOMPtr<nsITextControlElement> textCtrl = do_QueryInterface(aContent);
> +  return textCtrl != nullptr;

I think that this returns true for any types of <input> element. Perhaps, you want to do:

if (!aContent->IsElement()) {
  return false;
}
return !!aContent->AsElement()->GetTextEditorInternal();

Although, this could return true for readonly <input>/<textarea> elements. If you want to ban readonly editors, then, 

TextEditor* textEditor =
  aContent->AsElement()->GetTextEditorInternal();
return textEditor && !textEditor()->IsReadOnly();

::: dom/events/EventStateManager.cpp:945
(Diff revision 1)
>  
> +  // Don't activate if the target content of the event is contentEditable or
> +  // is inside an editable document, or is a text input control. Activating
> +  // due to typing/clicking on a text input would be surprising user experience.
> +  if (aTargetContent->IsEditable() ||
> +      doc->IsEditable() ||

Must not be necessary to check this since nsINode::IsEditable() checks it:
https://searchfox.org/mozilla-central/rev/78dbe34925f04975f16cb9a5d4938be714d41897/dom/base/nsINode.cpp#208,215,218
Attachment #8971457 - Flags: review?(masayuki) → review+
Comment on attachment 8967272 [details]
Bug 1452536 - Test that key events for non-printable keys and interaction with editable elements don't unblock autoplay.

https://reviewboard.mozilla.org/r/235960/#review246040

::: dom/media/test/file_autoplay_policy_key_blacklist.html:62
(Diff revision 2)
> +          "Shift",
> +          "Escape",
> +        ];
> +
> +        let modifiedKeys = [
> +          { key: "C", modifiers: { ctrlKey: true } },

nit: "c" since shiftKey is not true.

::: dom/media/test/file_autoplay_policy_key_blacklist.html:64
(Diff revision 2)
> +        ];
> +
> +        let modifiedKeys = [
> +          { key: "C", modifiers: { ctrlKey: true } },
> +          { key: "V", modifiers: { ctrlKey: true, shiftKey: true } },
> +          { key: "A", modifiers: { altKey: true } },

nit: "a" since shiftKey is not true.

::: dom/media/test/file_autoplay_policy_key_blacklist.html:74
(Diff revision 2)
> +          synthesizeComposition({ type: "compositioncommit", data: "\u30E9\u30FC\u30E1\u30F3", key: { key: "KEY_Enter" } });
> +          played = await element.play().then(() => true, () => false);
> +          ok(!played, "Entering text to " + name + " via IME should not activate document and should not unblock play");

I think that you should check synthesizeCompositionChange() before "compositioncommit".

  synthesizeCompositionChange({
    composition: {
      string: "\u30E9\u30FC\u30E1\u30F3",
      clauses: [
        { length: 4, attr: COMPOSITION_ATTR_RAW_CLAUSE }
      ]
    },
    caret: { start: 4, length: 0 }
  });

Then, you tested usual IME sequence (i.e., keydown -> compositionstart -> compositionupdate (-> text) -> keyup -> keydown (-> text) -> compositionend -> keyup

::: dom/media/test/file_autoplay_policy_key_blacklist.html:78
(Diff revision 2)
> +          for (let ch of "some ascii text for " + name) {
> +            synthesizeKey(ch);
> +          }
> +          played = await element.play().then(() => true, () => false);
> +          ok(!played, "Entering ASCII text into " + name + " should not activate document and should not unblock play");

Why don't you check if usual character input in editor won't cause allowing autoplay?

E.g., sendString("a");
Attachment #8967272 - Flags: review?(masayuki) → review+
Thanks for the review and your thoughtful comments Masayuki!
Pushed by cpearce@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/006f976d7963
Don't gesture activate documents on non-printable key events. r=masayuki
https://hg.mozilla.org/integration/autoland/rev/c454505cc025
Don't gesture activate for events targetting editable elements. r=masayuki
https://hg.mozilla.org/integration/autoland/rev/1259c5bc20a7
Test that key events for non-printable keys and interaction with editable elements don't unblock autoplay. r=masayuki
Status: RESOLVED → REOPENED
status-firefox61: fixed → ---
Resolution: FIXED → ---
Target Milestone: mozilla61 → ---
As per bug 1458166 comment 14 this is not the cause of the failure reported in bug 1458166. So I'm going to re-land this.
Flags: needinfo?(cpearce)
Pushed by cpearce@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/46471bf78ee8
Don't gesture activate documents on non-printable key events. r=masayuki
https://hg.mozilla.org/integration/autoland/rev/2f2a7a27a90f
Don't gesture activate for events targetting editable elements. r=masayuki
https://hg.mozilla.org/integration/autoland/rev/9c2deb188bdf
Test that key events for non-printable keys and interaction with editable elements don't unblock autoplay. r=masayuki
https://hg.mozilla.org/mozilla-central/rev/46471bf78ee8
https://hg.mozilla.org/mozilla-central/rev/2f2a7a27a90f
https://hg.mozilla.org/mozilla-central/rev/9c2deb188bdf
Status: REOPENED → RESOLVED
Closed: 2 years ago → 2 years ago
status-firefox61: --- → fixed
Resolution: --- → FIXED
Target Milestone: --- → mozilla61

Based discussions elsewhere, we want to not gesture activate for input events directed at editable elements.

Chris, could you point me to those discussions? This is rather counter-intuitive, as the act of typing into an input seems to qualify as "interacting with the web page", and the current rules motivate the developer to avoid native inputs in order to get the page gesture-activated, see the use-case described here: https://stackoverflow.com/questions/57504122/browser-denying-javascript-play

Flags: needinfo?(chris)

I tested the examlple in comment37 and another one reported in bug1572939, I found that Chrome and Safari didn't block autoplay when haiving user input on editable content. I can imagine some scenario like simulating a typing machine by playing a short sound while user is typing [1].

It seems that we don't really need to restrict that user inputs, which can activate document, should only occur on non-editable content. Even if they occur on non-editable content, it might still have a chance to annoy user, it's totally depending on websites' design.

I will go to remove this constraint in bug1572939.

[1] https://alastor0325.github.io/htmltests/autoplay_tests/playSoundOnEditableInput.html

You need to log in before you can comment on or make changes to this bug.