1285987 - HTML5 player on Twitch.tv freezes after a ~30 seconds

Status: RESOLVED FIXED

(Core :: Audio/Video: Playback, defect, P1)

Description

[btbn]

User Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:48.0) Gecko/20100101 Firefox/48.0
Build ID: 20160706215822

Steps to reproduce:

After Updating from 47 to 48 beta, watching stream on Twitch.tv(Using the native HTML5 player, disable Flash to get it), some of the streams freeze after playing for 10~30 seconds.
I am testing e10s, so i tried disabling it. But even without e10s force-enabled this issue is still present.



Actual results:

Most streams seem to work mostly fine, but basically all of them show occasional stuttering and short freezes that weren't happening on 47 release.
A few of them, for example https://www.twitch.tv/dansgaming, freeze every time after playing for a short while.
When leaving the tab open the stream will play a few segments again after being frozen for a few minutes.
Looking at the Debug-Network-Console, the site constantly keeps loading the m3u8 playlist and video segments, even while the player is frozen.


Expected results:

The streams should play flawlessly like they did on 47.

Comment 1

[Alice0775 White]

[Tracking Requested - why for this release]:  Breaks video playback

I can reproduce the freeze on Nightly50.0a1[1].



[1]
https://hg.mozilla.org/mozilla-central/rev/aac8ff1024c553d9c92b85b8b6ba90f65de2ed08
Mozilla/5.0 (Windows NT 10.0; WOW64; rv:50.0) Gecko/20100101 Firefox/50.0 ID:20160712030234



Regression window:
https://hg.mozilla.org/integration/mozilla-inbound/pushloghtml?fromchange=ecdb77ae56812fff63dc9bd7633bc976b2813980&tochange=71a44348d3b75b475fa14c11720b069393602c38

Regressed by: Bug 1276184

Comment 2

[u480271]

:jya is the expert here.

Comment 3

[Jean-Yves Avenard [:jya]]

FWIW, I get freeze with other browsers too.

Comment 4

[Jean-Yves Avenard [:jya]]

FWIW, the twitch html5 player isn't public yet and the default player given when flash isn't enabled is broken (as per twitch people).

The proper html5 (in beta) would be accessed via http://player.twitch.tv/?volume=0.4&channel=dansgaming&mseDev

I get no freeze there. Can you confirm that it works for you ?

Comment 5

[Alice0775 White]

(In reply to Jean-Yves Avenard [:jya] from comment #4)
> FWIW, the twitch html5 player isn't public yet and the default player given
> when flash isn't enabled is broken (as per twitch people).
> 
> The proper html5 (in beta) would be accessed via
> http://player.twitch.tv/?volume=0.4&channel=dansgaming&mseDev
> 
> I get no freeze there. Can you confirm that it works for you ?

I watched the video about 10 minutes,
Yes, I can confirm that the video works on Nightly50.0a1(2016-07-12) without any problem.

Comment 6

[Jean-Yves Avenard [:jya]]

I can't reproduce the problem with the provided URL. Please provide exact steps to reproduce (STR). including what you did to disable flash.

as for me https://www.twitch.tv/dansgaming doesn't even start if flash isn't installed and using the proper html5 player plays just fine.

Otherwise, I'll close this bug as invalid. thanks

Comment 7

[Alice0775 White]

(In reply to Jean-Yves Avenard [:jya] from comment #6)
> I can't reproduce the problem with the provided URL. Please provide exact
> steps to reproduce (STR). including what you did to disable flash.
> 
> as for me https://www.twitch.tv/dansgaming doesn't even start if flash isn't
> installed and using the proper html5 player plays just fine.
> 
> Otherwise, I'll close this bug as invalid. thanks

It need to disable "Mixed contents block protection" from padlock icon in LocationBar. 
(or setting security.mixed_content.block_active_content = false from about config)

Comment 8

[Jean-Yves Avenard [:jya]]

Then WFM with Nightly 50 with either the default URL (flash disabled) or using http://player.twitch.tv/?volume=0.4&channel=dansgaming&mseDev on either windows 10 or mac 10.11.

debugging shows that at no stage is the code changed by bug 1276184 hit... so I'm not sure how that could have caused to appear in the regression range.

Alice0075 how certain are you for your regression range?

Comment 9

[Alice0775 White]

(In reply to Jean-Yves Avenard [:jya] from comment #8)
> Then WFM with Nightly 50 with either the default URL (flash disabled) or
> using http://player.twitch.tv/?volume=0.4&channel=dansgaming&mseDev on
> either windows 10 or mac 10.11.

Yep, I cannot reproduce the problem using URL of comment#0 on Beta48b7, Nightly50.0a1 and on the bad build too..., it's weird.

Comment 10

[Alice0775 White]

Okey, I remove tracking flag and change status to UNCONFIRMED.

Comment 11

[Jean-Yves Avenard [:jya]]

my guess is that twitch fixed the default html5 player that had the problem following my discussion with them yesterday.
so we're not being served the same player anymore.

There's likely a regression anyway so I will ask them if it's possible to access to the player that was used when this bug was reported.

Comment 12

[btbn]

The player at http://player.twitch.tv/?volume=0.4&channel=dansgaming&mseDev works flawlessly for me.
The one at https://www.twitch.tv/dansgaming still shows the occasional freezes and video artifacts, didn't have a full freeze today yet, but the video is definitely troubled.

Opening the player.twitch.tv one via https complains about missing Flash and plays audio-only.
Which can be workarounded by adding &html5: https://player.twitch.tv/?volume=0.4&channel=dansgaming&mseDev&html5

All I did to get the HTML5 player on the main website was completely uninstalling flash from my PC(Windows 10 x64, running Firefox 64 bit version).
It should be noted that I am logged in and a Turbo Subscriber. Logging out and/or opening the site in a private window results in the player not working with a black screen, also for the player.twitch.tv one.
Can't test if Turbo is required, but they mentioned in a recent blog post that the new HTML5 player will be rolled out for Turbo-Users only(as it can't play ads yet I'd guess): https://blog.twitch.tv/coming-soon-html5-video-player-beta-cdb94b026a8c

Also, for me it's no longer required to disable security.mixed_content.block_active_content, the playlist and segments are all served via https while I'm logged in.

Now the interesting thing:
https://player.twitch.tv/?volume=0.4&channel=dansgaming&mseDev&html5 shows the same freezes and artifacts for me as the player on the main site. Changing https to http makes it play flawlessly.

Comment 13

[Jean-Yves Avenard [:jya]]

(In reply to btbn from comment #12)
> The player at http://player.twitch.tv/?volume=0.4&channel=dansgaming&mseDev
> works flawlessly for me.
> The one at https://www.twitch.tv/dansgaming still shows the occasional
> freezes and video artifacts, didn't have a full freeze today yet, but the
> video is definitely troubled.

that's was to be expected with the old html5 player as the data being fed was incorrect, video artifacts were normal.

> 
> Opening the player.twitch.tv one via https complains about missing Flash and
> plays audio-only.
> Which can be workarounded by adding &html5:
> https://player.twitch.tv/?volume=0.4&channel=dansgaming&mseDev&html5

html5 is the old player and shouldn't be used. You should use mseDev instead

Comment 14

[btbn]

With mseDev the player only works via http as of yet. They probably didn't update it for their now https capable CDN. Strange thing is that it does download the video segments via https.

So it seems like the main website also still uses that older player, which is where the issues come from.

I just confirmed that all of those "bad players" work perfectly fine in Firefox 47 though.
Tested with the following URLs, while being logged in as Turbo-User:
https://www.twitch.tv/dansgaming
https://player.twitch.tv/?volume=0.4&channel=dansgaming&html5
https://player.twitch.tv/?volume=0.4&channel=dansgaming
http://player.twitch.tv/?volume=0.4&channel=dansgaming

In Firefox 47.0.1 they play fine, with no visible artifacts or freezes, even after several minutes.
In Firefox 48.0b7 I almost immediately get artifacts, and after ~30 seconds it starts freezing, until it eventually freezes completely after a few minutes.

The only one that works fine in 48 is http://player.twitch.tv/?volume=0.4&channel=dansgaming&mseDev

So I'm not sure if this is a valid regression in Firefox if it's known that their older HTML5 player provides broken data.
It did work fine in 47 though, and some change in 48 broke it.

Comment 15

[Jean-Yves Avenard [:jya]]

ooooh, I now get it. if you are using the old player, all video frames are incorrectly tagged as keyframes.

Change in bug 1276184 introduces a workaround that if a keyframe is seen, it assumes it's a new block of data and search where to insert that new block. This completely breaks GOP detection (seeing that a GOP is now incorrectly assumed to be a single video frame)

Now that makes sense... would also explain why you're seeing the video corruption as frames as incorrectly re-ordered.

I'm going to write a workaround so that the fix introduced in bug 1276184 is narrowed only to the case that had the original bug (webm container)

Comment 16

[Jean-Yves Avenard [:jya]]

Mind you, the data is buggy anyway. Will work with live stream and only if the user never attempts to seek

Comment 17

[Jean-Yves Avenard [:jya]]

Attachment: Bug 1285987: Narrow the workaround added in bug 1276184 to only be effective with webm.

Some invalid streams incorrectly tag all frames as keyframes, which cause the frames to be inserted in the wrong order in the trackbuffer.

Review commit: https://reviewboard.mozilla.org/r/63924/diff/#index_header
See other reviews: https://reviewboard.mozilla.org/r/63924/

Comment 18

[Gerald Squelart (he/him) (not at Mozilla since 2022-09-15)]

Comment on attachment 8770507 [details]
Bug 1285987: Narrow the workaround added in bug 1276184 to only be effective with webm.

https://reviewboard.mozilla.org/r/63924/#review61118

Comment 19

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/fa68522872c7
Narrow the workaround added in bug 1276184 to only be effective with webm. r=gerald

Comment 20

[Jean-Yves Avenard [:jya]]

Comment on attachment 8770507 [details]
Bug 1285987: Narrow the workaround added in bug 1276184 to only be effective with webm.

Approval Request Comment
[Feature/regressing bug #]:1276184
[User impact if declined]: some streams will stop playing (in particular Amazon live streams and twitch.tv). While invalid, those streams played okayish before. Now they won't play at all.
[Describe test coverage new/current, TreeHerder]: manual tests.
[Risks and why]: can't think of any. It's a partial revert of the regressing bug, limiting the conditions in which the workaround was applied (and will currently be strictly limited to YouTube)
[String/UUID change made/needed]: none

Comment 21

[Liz Henry (:lizzard) (relman/hg->git project)]

Looks like this may also fix bug 1285709.

Comment 22

[Liz Henry (:lizzard) (relman/hg->git project)]

Once we verify this fix on Nightly I do think we should uplift to aurora and beta.

Comment 23

[Carsten Book [:Tomcat]]

https://hg.mozilla.org/mozilla-central/rev/fa68522872c7

Comment 24

[Gerry Chang [:gchang]]

Hi Cornel,
Can you help to find someone to verify this on nightly?

Comment 25

[Jean-Yves Avenard [:jya]]

I don't think there's much to verify. The html5 player in use at the time the bug was reported is no longer in use. The new one doesn't have the bug.

Comment 26

[Gerry Chang [:gchang]]

Comment on attachment 8770507 [details]
Bug 1285987: Narrow the workaround added in bug 1276184 to only be effective with webm.

This patch fixes a regression which how we deal with invalid data provided by player. Take it in 48 beta 9 and aurora.

Comment 27

[Carsten Book [:Tomcat]]

https://hg.mozilla.org/releases/mozilla-aurora/rev/79ec0cb6ba71

Comment 28

[Carsten Book [:Tomcat]]

https://hg.mozilla.org/releases/mozilla-beta/rev/8b6506beee8e

Comment 29

[Andrei Vaida [:avaida]]

Removing the qe+ and ni? flags, per Comment 25.