1462906 - Switching to tabs with data (streams) being received pauses data on inactive tabs

Status: VERIFIED FIXED

(Core :: Networking: HTTP, defect, P2)

Description

[lpy750]

User Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:60.0) Gecko/20100101 Firefox/60.0
Build ID: 20180516032328

Steps to reproduce:

An example could be having two Twitch or YouTube streams open at once in separate tabs.



Actual results:

Once switched to another tab, the inactive tab no longer receives data and will eventually cause the stream to buffer until it is the active tab again.

Basically the current active tab will "steal" the internet from inactive tabs while it is receiving data.

This bug appears to be present in the latest Beta and Nightly but not present in the latest stable release 60.0.1.

Comment 1

[Maire Reavy [:mreavy]]

Chris -- Is this behavior intentional?

Comment 2

[lpy750]

Forgot to post this yesterday... This is a video demonstrating what happens. With 61.0b7, activity in right tab (in this case mashing F5 to keep the activity consistent) prevents the left tab to continue to receive stream data and therefore freezes the stream until the right tab stops loading. With 60.0.1, the right tab doesn't affect data at all in the left tab.

https://www.youtube.com/watch?v=x_eVV56zTV0

I normally use Beta but this recent bug has brought me back to the Stable version.

Comment 3

[Chris Pearce [:cpearce (Not reading bugmail)]]

YouTube/Twitch uses XHR for its networking, so there is not something in the media playback stack's networking that's causing this. If this is a regression, it must be caused by a change in DOM or networking priority/scheduling code.

Looking at the video of the problem, maybe the tab being reloaded is maxing out the limit on the number of concurrent network connections, and thus starving networking in the Twitch tab?

Patrick: Did anything change in 61 which would affect our connection limits? Can you think of anything else that would cause some tabs to starve the networking in other tabs?

Comment 4

[Patrick McManus [:mcmanus]]

nothing like that in 61 afaict

Comment 5

[lpy750]

Found the setting.

network.http.throttle.enable

I set that to false.

I'm not exactly sure what its purpose is or if it's working correctly/incorrectly in 61, but I guess it could give people the impression that Firefox is slower than other browsers.

Comment 6

[[:philipp]]

this issue gets reported on various support channels after the 61 update:
https://support.mozilla.org/en-US/questions/firefox?owner=all&tagged=bug1462906&show=all
https://old.reddit.com/r/firefox/comments/8ws8gm/when_youtube_or_twitch_is_in_another_tab_it_stops/
https://old.reddit.com/r/firefox/comments/8vmbkg/switching_tabs_when_having_twitchyoutube_stream/
https://old.reddit.com/r/firefox/comments/8vj98g/is_there_any_way_to_prevent_inactive_tab/
https://old.reddit.com/r/firefox/comments/8wuhf8/620b6_64bit_video_frames_freezingskipping_on/

due to comment #5 i'm guesstimating this might be related to bug 1444975

Comment 7

[[:philipp]]

many of the problem reports also mention having a dual-monitor setup, in case that matters...

Comment 8

[Honza Bambas (:mayhemer)]

(In reply to [:philipp] from comment #7)
> many of the problem reports also mention having a dual-monitor setup, in
> case that matters...

It very likely makes the problem way more visible.

Apparently, the old throttling version (v1) was nearly non-functional.  The new one actually does something, but is too aggressive at the default settings.  

I'll think of adjustments and mitigations, we may return back to the previous version or simply turn throttling off completely.

Comment 9

[[:philipp]]

perhaps another usecase for a pref flip via normandy as a temporary mitigation?

Comment 10

[Honza Bambas (:mayhemer)]

So, after talking to RyanVM, this will be switched back to version 1 on release using normandy and we will land patches on beta and central for the pref switch in-code.

I will then file a new bug to take a deeper look, referring this bug for STR and symptoms.

Comment 11

[Ryan VanderMeulen [:RyanVM]]

Attachment: make HTTP throttling v2 Nightly-only

Thinking about it a bit more, I'm wondering if we should leave v2 on for Nightly builds to avoid any new regressions from appearing while it's off by default. Feel free to r- if you prefer to unconditionally disable and I'll submit a revised patch.

Comment 12

[Honza Bambas (:mayhemer)]

Comment on attachment 8991085 [details] [diff] [review]
make HTTP throttling v2 Nightly-only

Review of attachment 8991085 [details] [diff] [review]:
-----------------------------------------------------------------

Thanks!

Comment 13

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/mozilla-inbound/rev/88c4d6e6d529
Make HTTP throttling v2 algorithm Nightly-only due to video streaming regressions. r=mayhemer

Comment 14

[Ryan VanderMeulen [:RyanVM]]

Here's the Normandy rule for 61 (requires VPN): https://delivery-console.prod.mozaws.net/recipe/503/
Public API: https://normandy.cdn.mozilla.net/api/v1/recipe/503/

I got r+ from Mythmon on Slack already for this. Plan is to deploy this recipe immediately, then uplift the patch in this bug to Beta/Release for Fx62+ and any future 61.0.x builds.

Comment 15

[Ryan VanderMeulen [:RyanVM]]

Mythmon has approved and deployed the recipe and it should be going out to release users immediately.

Comment 16

[Cosmin Sabou [:CosminS]]

https://hg.mozilla.org/mozilla-central/rev/88c4d6e6d529

Comment 17

[Ryan VanderMeulen [:RyanVM]]

Hi, this patch should be in Nightly builds within the next few hours. Also, we've pushed out a remote pref flip for release users. Can you confirm that this fix has been effective for you?

Comment 18

[lpy750]

I can confirm that the fix is effective when installing the release version.

Something very minor however is on first startup, it appears to be running the aggressive v2. Subsequent restarts are fine.

My guess on this is, the first time Firefox opens, throttle setting is changed from v2 to v1 but requires a restart of Firefox to take effect.

Very minor as I said, and the main thing is Twitch/YouTube/etc function as they should once again.

Comment 19

[Ryan VanderMeulen [:RyanVM]]

Yes, that's expected behavior for this pref (needing a restart to take effect) given how it's being set on release. We'll be uplifting the patch in this bug to affected branches as well so that it's set by default for all newer releases too. Thanks for the verification!

Comment 20

[Ryan VanderMeulen [:RyanVM]]

Comment on attachment 8991085 [details] [diff] [review]
make HTTP throttling v2 Nightly-only

Approval Request Comment
[Feature/Bug causing the regression]: bug 1444975
[User impact if declined]: video playback issues in inactive tabs
[Is this code covered by automated tests?]: yes
[Has the fix been verified in Nightly?]: yes, and on release via Normandy
[Needs manual test from QE? If yes, steps to reproduce]: No
[List of other uplifts needed for the feature/fix]: None
[Is the change risky?]: No
[Why is the change risky/not risky?]: Just reverts us to the pre-61 throttling behavior
[String changes made/needed]: None

Comment 21

[Emma Humphries ☕️🎸🧞‍♀️✨ (she/they) [:emceeaich] (Pacific Time) use needinfo]

NB: still seeing this stalling behavior on Nightly 63.0a1 (2018-07-11) (64-bit, MacOS) after flipping the network.http.throttle.enable back to false.

Comment 22

[Liz Henry (:lizzard) (relman/hg->git project)]

Comment on attachment 8991085 [details] [diff] [review]
make HTTP throttling v2 Nightly-only

I'd like to see how this behaves on beta (aside from verifying in Nightly)
Let's uplift for beta 8 .

Comment 23

[Liz Henry (:lizzard) (relman/hg->git project)]

Asking for verification here, but I think we need to check both Nightly and beta.

Comment 24

[Ryan VanderMeulen [:RyanVM]]

https://hg.mozilla.org/releases/mozilla-beta/rev/9ff4fe6505c2

Comment 25

[Abe - QA (:Abe_LV)]

I tested this on Windows 10, OSX 10.13, and Ubuntu 16.04 with beta 62.0b8 and latest Nightly 63.0a1.
In the latest beta build (62.0b8), 'network.http.throttle.version' is set to 1 and the stalling issue is verified as fixed.
In Nightly, the issue is not reproducible when 'network.http.throttle.version' is manually changed to 1. 

I will also verify this bug on release channel when it gets fixed. 

Test Environment:

Version 	62.0b8
Build ID 	20180712042337
Update Channel 	beta
User Agent 	Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:62.0) Gecko/20100101 Firefox/62.0

Comment 26

[Abe - QA (:Abe_LV)]

I retested this using release 60.0.1 that has the pref fliped via Normandy "network.http.throttle.version; 1". 
The stalling issue is not reproducible anymore. It is verified as fixed on release as well. 

Test Environments:
-----------------
Version 	61.0.1
Build ID 	20180704003137
Update Channel 	release
User Agent 	Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:61.0) Gecko/20100101 Firefox/61.0

Comment 27

[Honza Bambas (:mayhemer)]

(In reply to Emma Humphries, Bugmaster ☕️🎸🧞‍♀️✨ (she/her) [:emceeaich] (UTC-8) needinfo? me from comment #21)
> NB: still seeing this stalling behavior on Nightly 63.0a1 (2018-07-11)
> (64-bit, MacOS) after flipping the network.http.throttle.enable back to
> false.

Emma, is this is question or are stating observations?

Comment 28

[[:philipp]]

the original problem might also affect downloads that are ran in parallel according to the observations of some german users:
https://www.camp-firefox.de/forum/viewtopic.php?f=1&t=125634

Comment 29

[Tom S [:evilpie]]

We still see multiple reports of this on /r/firefox, do we plan on pushing this directly to a point release of 61?

Comment 30

[Julien Cristau [:jcristau]]

Users on 61 should be getting the pref flip from https://normandy.cdn.mozilla.net/api/v1/recipe/503/ (see comments 14/15)

This may also be included in 61.0.2 if and when that happens, but so far isn't driving at dot release on its own.

Comment 31

[Ryan VanderMeulen [:RyanVM]]

Comment on attachment 8991085 [details] [diff] [review]
make HTTP throttling v2 Nightly-only

Approved for 61.0.2.

Comment 32

[Ryan VanderMeulen [:RyanVM]]

https://hg.mozilla.org/releases/mozilla-release/rev/13eedf348fc4

Comment 33

[Iulia Cristescu, QA [:JuliaC] (away, please needinfo? [e-mail])]

I managed to reproduce the initial issue using 61.0.1 (20180704003137). I also can confirm that 61.0.2 build1 (20180807170231) is fixed across platforms (Windows 10 x64, Ubuntu 18.04 x64 and macOS 10.13.4) . Marking it accordingly.

Comment 34

[Michael Cooper [:mythmon]]

Ryan, do we want to keep the Normandy hotfix in place for this?

Comment 35

[Ryan VanderMeulen [:RyanVM]]

I think we can drop it (especially since versions <61.0.2 don't have bug 1477380 which would seemingly limit the effectiveness of this recipe anyway).

Comment 36

[Michael Cooper [:mythmon]]

I have disabled the Normandy hotfix for this.

Comment 38

[Tom Ritter [:tjr]]

I found this while reviewing 2+ year old Nightly differences. IS this something we can ship to all channels or unify behavior on?

Comment 39

[Jan Honza Odvarko [:Honza] (always need-info? me)]

Honza Bambas might have been involved in this (not me), but he isn't with Mozilla anymore. Somebody else from the Necko team could know.
Dragana, perhaps you know more about this?

Comment 40

[Dragana Damjanovic [:dragana]]

(In reply to Tom Ritter [:tjr] from comment #38)

I found this while reviewing 2+ year old Nightly differences. IS this something we can ship to all channels or unify behavior on?

This feature is disabled because it was not working and causing regression. So this is not important.

Comment 41

[Marco Castelluccio [:marco]]

Should we then unify the behavior and just keep network.http.throttle.version at 1 for both Nightly and non-Nightly?

Comment 42

[Dragana Damjanovic [:dragana]]

(In reply to Marco Castelluccio [:marco] from comment #41)

Should we then unify the behavior and just keep network.http.throttle.version at 1 for both Nightly and non-Nightly?

We were thinking about removing the feature completely. I do not mind unifying the behavior.