Closed Bug 1319771 Opened 8 years ago Closed 7 years ago

If the pinned tab is the first tab in the tab list, the media won't be blocked after browser restarts

Categories

(Core :: Audio/Video: Playback, defect)

Product:
Core
Component:
Audio/Video: Playback
Version:
52 Branch
Type:
defect
Priority:
Not set
Severity:
normal

Tracking

()

Status:
VERIFIED FIXED
Milestone:
mozilla54
Tracking Flags:
Tracking Status
firefox50 --- unaffected
firefox51 --- affected
firefox52 --- affected
firefox53 --- affected
firefox54 --- verified

People

(Reporter: sbadau, Assigned: alwu)

References

Details

Attachments

(3 files)

block media.mp4
3.29 MB, video/mp4
Details
Bug 1319771 - part1 : only resume the window when there has active media components.
59 bytes, text/x-review-board-request
baku
: review+
Details
Bug 1319771 - part2 : resume foreground window if it was still be blocked.
59 bytes, text/x-review-board-request
baku
: review+
Details
Attached video block media.mp4 — 
Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:53.0) Gecko/20100101 Firefox/53.0
Build ID: 20161122030216

[Affected versions]:
Nightly 53.0a1, Aurora 52.0a2

[Affected platforms]:
All

[Steps to reproduce]:
1. Launch Nightly
2. Navigate to YouTube and start playing any video
3. In the tab bars - right click the YouTube tab and pin it
4. Pin other 3 tabs (different than YouTube)
5. Open a new tab and restart Nightly (Shift+F2+ restart)

[Expected result]:
- After restart, the YouTube's media content should be blocked. 

[Actual result]:
- The media content is not blocked after restart. Please see the screen cast for more details.

[Regression range]:
- not a regression

[Additional notes]:
- If I rearrange the pinned tabs and YouTube is the second one, then the YouTube video is blocked.
- If I close Firefox (YouTube being opened in the first pinned tab), media is blocked when opening Firefox.
Blocks: delay-autoplay
Hi, Boris,
I think this bug is caused by the document's wrong visibility state, do you know what's happened?
The following is the short description about this issue.
Thanks!

---

By default, the tab would block its autoplay media until that tab becomes visible in the first time. When the nsDocument becomes visible, it would activate the media component [1], so that the tab can resume its blocked media. 

However, in this case, even the tab is still invisible (on the background), but the document's visibility state has already become visible.

I think the document's visibility state shouldn't become visible, unless it becomes the foreground tab.

Note, this issue would only happen when the pin tab is the FIRST tab.

[1] http://searchfox.org/mozilla-central/rev/b4e6fa3527436bdbcd9dd2e1982382fe423431f3/dom/base/nsDocument.cpp#11826
Flags: needinfo?(bzbarsky)
> do you know what's happened?

Breakpoint on when the relevant document's visibility state becomes visible and see why that happens? 

In general, the visibility of tabs is maintained by the frontend code which modifies the "active" boolean on the docshell and that propagates through to the document.  Chances are, during startup the pinned tab is created first (and it's in the foreground at this point), then the other tabs are created and the pinned tab should be set inactive.  It's possible that's not happening.
Flags: needinfo?(bzbarsky)
Summary: Media not blocked in pinned tabs after restart → If the pinned tab is the first tab in the tab list, the media won't be blocked after browser restarts
Hi, Boris,
The first pinned tab would be regard as active tab after browser restarts, this behavior is our intention or the bug? 
Thanks.
Flags: needinfo?(bzbarsky)
The intention is that any tab that is not the selected one is inactive.  If that's not what we're doing, that's a bug.

But I just tried actually testing this directly.  In a clean profile I set my browser to restart with "windows and tabs from last time", loaded this page in one tab:

  <script>
  function logIt(str) {
    console.log(str + " " + document.hidden);
  }
  logIt("loading");
  onload = function() {
    logIt("onload");
    // setTimeout/setInterval are unreliable in background tabs, so do this
    // postMessage hack.
    var lastLog = performance.now();
    window.onmessage = function() {
      if (performance.now() - lastLog > 1000) {
        logIt("message");
        lastLog = performance.now();
      }
      window.postMessage(undefined, "*");
    }
    window.postMessage(undefined, "*");
  }
  </script>

and mozilla.org in another tab, pinned the tab with my logging page, made sure the mozilla.org tab is selected, and quit the browser.  Then I restarted the browser; it loaded the two pages, with the mozilla.org tab selected.  I waited 10s or so, then looked at the console on the pinned tab.  It showed "true" (so visibilityState == "hidden") for the first however many console logs.
Flags: needinfo?(bzbarsky)
Just tried the same with my test page and mozilla.org both pinned.  Same results.
After testing, I found that the mVisibilityState of the first pinned tab would be visible for a short while and then be set to hidden. (after restart the browser) I guess the onload message from comment4 was printed after mVisibilityState became to hidden.

The point is, why the first pinned tab's document would be visible after restart? It seems nsDocument's bug.
> I found that the mVisibilityState of the first pinned tab would be visible for a short while
> and then be set to hidden.

Sure.  It's visible when it's first created and is the only tab.  Then when the next tab gets created it gets set to hidden.

> I guess the onload message from comment4 was printed after mVisibilityState became to hidden.

And the "loading" message?  But then, the "loading" message is quite early on in that page; if the visibilityState is hidden by then, what's the problem.

> It seems nsDocument's bug.

Please see comment 2.
> I found that the mVisibilityState of the first pinned tab would be visible for a short while and then be set to hidden.

Just so we're clear:

1)  What's the URL of the document in question when it gets set to hidden?
2)  What is the ordering of the "set as hidden" bit compared to whatever media stuff is going on?
3)  What are the C++ and JS callstacks when it gets set to hidden?
(In reply to Boris Zbarsky [:bz] (still a bit busy) from comment #7)
> Sure.  It's visible when it's first created and is the only tab.  Then when
> the next tab gets created it gets set to hidden.

So this is the root cause! 

Since the document would activate the media component when it's visible at the first time, and if the first pinned tab is always visible in the beginning, that means we can't block the first pinned tab.

hmm... I don't have any idea about how to solve this issue if we can't change this kind of behavior.
> Since the document would activate the media component when it's visible at the first time

What I don't understand is the timing here.  The way I would _expect_ this would work, in the UI, is this:

1)  Create first tab, start load of page in it.
2)  Create second tab, set first tab hidden, start load of page in second tab.

in very quick sequence.  So I would expect the first tab to be marked hidden before we get anywhere close to loading enough stuff in it to know we have media stuff.

This is why I was asking for the information in comment 8.  I would expect the change to hidden to be happening when we still have the initial "about:blank" in that first tab!  And if that's not the case, I want to see what's causing the change and why it's happening so late.

So _I_ still don't understand the root cause of the problem you're seeing, and hence how to solve it.
(In reply to Boris Zbarsky [:bz] (still a bit busy) from comment #10)
> So I would expect the first tab to be marked hidden
> before we get anywhere close to loading enough stuff in it to know we have
> media stuff.

Our blocking mechanism is like that
1. have a member variable (mMediaSuspend) for media suspend in nsPIDOMWindowOuter and the default value is SUSPENDED_BLOCK (to block every media in the window) [1]

2. when nsDocument goes to foreground first time, set the mMediaSuspend to NONE_SUSPENDED [2]

3. when the media want to start playing, the AudioChannelService would check that variable in the outer window to decide whether the media can start playback or not
- if mMediaSuspend == SUSPENDED_BLOCK, block the media
- if mMediaSuspend == NONE_SUSPENDED, allow it to playback

[1] http://searchfox.org/mozilla-central/rev/51aa673e28802fe6ea89f4793721fc55334a6ac8/dom/base/nsGlobalWindow.cpp#652
[2] http://searchfox.org/mozilla-central/rev/51aa673e28802fe6ea89f4793721fc55334a6ac8/dom/base/nsDocument.cpp#11902

---

Even the first tab was marked as visible just a moment, we have already set the mMediaSuspend to NONE_SUSPENDED, that is why we can't block the media from the first tab.
Flags: needinfo?(bzbarsky)
OK, well, why is the blocking boolean on the _outer_ window, exactly?  It means that media in a tab will never be blocked if I have ever looked at that tab.  Except now with e10s it will somewhat randomly depend on whether all the navigations after I stop looking at it are same-process or not...

That entire model seems somewhat broken to me.  What is the actual goal mMediaSuspend is trying to accomplish?
Flags: needinfo?(bzbarsky)
That is because the audio channel system controls audio by per window, so we put this variable is in the outer window. Audio channel needs to manage the whole window, so that it can achieve the abilities like mute/unmute/unblock all audio in the same tab.

The mMediaSuspend is used to block any autoplay media from the tab user never visits before, and the media would be resumed after the tab become visible at the first time. So user always looks at the tab, that means that tab shouldn't be blocked. You can see more details about this feature in bug1308399 comment1.
OK, well, that model is fundamentally incompatible with how tabs actually get created during session restore, as far as I can tell.
According to comment14, we don't have any good way to solve this issue now.
Since this is not a serious problem, I'll put the bug here until we have better solution (maybe change the whole blocking-media mechanism or waiting the DOM mechanism changes someday)
What DOM mechanism changes?

It really sounds to me like the blocking-media mechanism is just broken by design and needs to be fixed...
Mozilla/5.0 (Windows NT 10.0; WOW64; rv:51.0) Gecko/20100101 Firefox/51.0
Build ID: 20170109165508

Firefox 51 beta 13 is also affected by this issue - since the preference "media.block-autoplay-until-in-foreground" is set to true by default.

Changing status-firefox:51 to affected.
status-firefox51: unaffected → affected
Assignee: nobody → alwu
Comment on attachment 8827722 [details]
Bug 1319771 - part2 : resume foreground window if it was still be blocked.

https://reviewboard.mozilla.org/r/105344/#review106736

::: dom/audiochannel/AudioChannelAgent.cpp:187
(Diff revision 1)
>      mWeakCallback = do_GetWeakReference(aCallback);
>    } else {
>      mCallback = aCallback;
>    }
>  
> +  MaybeActiveMediaComponents();

this doesn't seem to be the right place.
AudioChannelAgent doesn't know anything about media components.
This seems an hack :)
Attachment #8827722 - Flags: review?(amarchesini) → review-
Comment on attachment 8827106 [details]
Bug 1319771 - part1 : only resume the window when there has active media components.

https://reviewboard.mozilla.org/r/100778/#review106740

::: dom/audiochannel/AudioChannelService.cpp:205
(Diff revision 5)
>    if (!gAudioChannelService) {
>      gAudioChannelService = new AudioChannelService();
>    }
>  }
>  
> +/* static */ bool

Write a comment here saying that the service starts when the first AudioChannelAgent is created
Attachment #8827106 - Flags: review?(amarchesini) → review+
Comment on attachment 8827722 [details]
Bug 1319771 - part2 : resume foreground window if it was still be blocked.

I don't know why the reviewboard doesn't set the review flag, so set the flag from here.
Attachment #8827722 - Flags: review?(amarchesini)
Comment on attachment 8827722 [details]
Bug 1319771 - part2 : resume foreground window if it was still be blocked.

https://reviewboard.mozilla.org/r/105344/#review108564

thanks.
Attachment #8827722 - Flags: review?(amarchesini) → review+
Pushed by alwu@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/7302c447d09a
part1 : only resume the window when there has active media components. r=baku
https://hg.mozilla.org/integration/autoland/rev/9d882c051d00
part2 : resume foreground window if it was still be blocked. r=baku
https://hg.mozilla.org/mozilla-central/rev/7302c447d09a
https://hg.mozilla.org/mozilla-central/rev/9d882c051d00
Status: NEW → RESOLVED
Closed: 7 years ago
status-firefox54: --- → fixed
Resolution: --- → FIXED
Target Milestone: --- → mozilla54
Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:54.0) Gecko/20100101 Firefox/54.0
Build ID: 20170206030211

Verified as fixed using the latest Nightly 54.0a1 on Windows 10 x64, Ubuntu 16.04 and Mac OS X 10.11.
status-firefox54: fixed → verified
Status: RESOLVED → VERIFIED
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Creator:
Created:
Updated:
Size: