1469257 - twitch.tv video clip 1080p playback has momentary freezes

Status: VERIFIED FIXED

(Core :: Audio/Video: Playback, defect, P2)

Description

[Kestrel]

First reported here:
https://www.reddit.com/r/firefox/comments/8rq1fr/video_stuttering_issue_with_specific/

Two clips from the same stream at the same time, one has momentary freezes of 1-2 seconds at certain timestamps and the other doesn't.
https://clips.twitch.tv/SolidRichCrabs4Head - freezes
https://clips.twitch.tv/SmoothGracefulDuckPogChamp - no freezes

Freezes only happen at 1080p quality setting.

Tested 60.0.2 and 62.0a1 on Ubuntu 18.04 with new profiles.

No freezes on Chromium 67.

Comment 1

[Jean-Yves Avenard [:jya]]

Please provide a copy of the about:support output...

Comment 2

[Jean-Yves Avenard [:jya]]

And which version of ffmpeg are you using?

Comment 3

[Kestrel]

It also happens on Windows 10 1803 which uses WMF decoder.

MediaInfo (Windows): https://pastebin.mozilla.org/9087940

No hardware decoding used on Linux or Windows.

Chromium 67 with hardware decoding disabled is smooth.

Software decoding uses 50% CPU on Firefox and 30% CPU on Chromium.

CPU usage is steady for the smooth video and fluctuates for the freezing video.

Comment 4

[Jean-Yves Avenard [:jya]]

When HW decoding on Windows is switched off in Chrome, it uses ffmpeg to decode which is likely more optimised than WMF H264 decoder.


Enable HW layout on Ubuntu, it will make play-back much better. In about:config set layers.acceleration.force-enabled to true and restart Firefox

Comment 5

[Jean-Yves Avenard [:jya]]

Is your problem on Ubuntu or Windows?

Comment 6

[Kestrel]

Both. layers.acceleration.force-enabled and gfx.webrender.all make no difference on Linux or Windows, nor does playing the video file directly in a tiny window, it still has the same 1-2 second freezes every few seconds.

Comment 7

[Kestrel]

Found an old build with it working smoothly. First bad build is 20170602, no further build data available for refinement.

Regression window:
https://hg.mozilla.org/mozilla-central/pushloghtml?fromchange=bdb2387396b4a74dfefb7c983733eed3625e906a&tochange=87c745019518b1d6cd782534f2553721e5735657

Comment 8

[Jean-Yves Avenard [:jya]]

can reproduce on mac.

Comment 9

[Liz Henry (:lizzard) (relman/hg->git project)]

Hoping we can fix this in 62 beta, tracking for 62.

Comment 10

[Jean-Yves Avenard [:jya]]

I'm guessing the regression was introduced by bug 1313398. What could be happening is that it believes the SPS/PPS inband is changing and that causes a new decoder to be created, which means frame would be skipped until the next keyframe.

Problem like this are typically due to bad encoding.

Comment 11

[Jean-Yves Avenard [:jya]]

So it is as I expected...

This streams has SPS NAL inband and they change from one keyframe to the next.

The SPS's chroma_loc_info_present_flag change from true to false and vice-versa... Weirder even is that the SPS changes on non-keyframe...

So a new decoder is created, and we have to wait for the next keyframe.

Maybe we could simply ignore SPS change on non-keyframe...

Comment 12

[Phabricator Automation]

Attachment: Bug 1469257 - [H264] Only check for SPS changes on keyframe

Some invalid streams contain SPS changes and those appear to only occur on non-keyframe, this cause all frames to be dropped until the next keyframe is found. This result in apparent freezes.

While it is theoretically possible to have SPS changes inband on non-keyframe those should be very rare (I've never seen one). The content would have been invalid anyway in an non-fragmented mp4.

So we now only check for a SPS change on keyframe. This would cause no affect on either windows, android or ffmpeg as those decoders handle format change fine. The mac decoder could however show garbled frames temporarily.

Comment 13

[Phabricator Automation]

Comment on attachment 8986532 [details]
Bug 1469257 - [H264] Only check for SPS changes on keyframe

Bryce Van Dyk (:bryce) has approved the revision.

https://phabricator.services.mozilla.com/D1733

Comment 14

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/ddde6ad6d9e6
[H264] Only check for SPS changes on keyframe r=bryce

Comment 15

[Cosmin Sabou [:CosminS]]

https://hg.mozilla.org/mozilla-central/rev/ddde6ad6d9e6

Comment 16

[Jean-Yves Avenard [:jya]]

Could you test with today's nightly (build id 20180621013659 and later) to see if the problem is fixed for you?

Thank you

Comment 17

[Jean-Yves Avenard [:jya]]

Comment on attachment 8986532 [details]
Bug 1469257 - [H264] Only check for SPS changes on keyframe

Approval Request Comment
[Feature/Bug causing the regression]: 1313398
[User impact if declined]: stuttering, frozen frames with some videos on popular site.
[Is this code covered by automated tests?]: no
[Has the fix been verified in Nightly?]: not yet, manually checked
[Needs manual test from QE? If yes, steps to reproduce]:  follow instructions in bug description
[List of other uplifts needed for the feature/fix]: no
[Is the change risky?]: no
[Why is the change risky/not risky?]: The commit message explain the risks and downside. Ultimately, checking the changes on stream content is something we never used to do until recently. So we limit the scope at which we check for changes. Bug 1313398 was manually checked to make sure it was still fixed.
[String changes made/needed]: none

[Approval Request Comment]
If this is not a sec:{high,crit} bug, please state case for ESR consideration:
User impact if declined: 
Fix Landed on Version:
Risk to taking this patch (and alternatives if risky): 
String or UUID changes made by this patch: 

See https://wiki.mozilla.org/Release_Management/ESR_Landing_Process for more info.

Comment 18

[Kestrel]

Verified as fixed on Nightly 62.0a1 (20180621013659) on Ubuntu 18.04.

Comment 19

[Ryan VanderMeulen [:RyanVM]]

Comment on attachment 8986532 [details]
Bug 1469257 - [H264] Only check for SPS changes on keyframe

Fx61 is already in the RC stage (shipping Tuesday). It's too late for 61.0 at this point, but I'll leave it on the radar for possibly dot-release consideration.

Comment 20

[Ryan VanderMeulen [:RyanVM]]

Comment on attachment 8986532 [details]
Bug 1469257 - [H264] Only check for SPS changes on keyframe

Approved for 61.0.1.

Comment 21

[Ryan VanderMeulen [:RyanVM]]

https://hg.mozilla.org/releases/mozilla-release/rev/4a7ccfb9d5ed

Comment 22

[Maria Berlinger [:mberlinger], Services QA]

I have managed to reproduce this bug on an affected Release build 60.0.2 (2018-18-06), using the STR from comment 0. 

This is verified fixed on Beta 62.0b5 (20180702164905) and Release 61.0.1 (20180704003137) on the following OSes: Windows 10 x64, Ubuntu 16.04 x64 and macOS 10.12.

Comment 23

[Ryan VanderMeulen [:RyanVM]]

Comment on attachment 8986532 [details]
Bug 1469257 - [H264] Only check for SPS changes on keyframe

Approved for ESR 60.2 also.

Comment 24

[Ryan VanderMeulen [:RyanVM]]

https://hg.mozilla.org/releases/mozilla-esr60/rev/42232ab8fa6d

Comment 25

[Maria Berlinger [:mberlinger], Services QA]

I have also verified this bug on build 60.1.1esr (buildID=20180724124937), using the STR from comment 0. 

This is verified fixed on the following OSes: Windows 10 x64, Ubuntu 16.04 x64 and macOS 10.12.