Closed Bug 1469257 Opened 6 years ago Closed 6 years ago

twitch.tv video clip 1080p playback has momentary freezes

Categories

(Core :: Audio/Video: Playback, defect, P2)

62 Branch
defect

Tracking

()

VERIFIED FIXED
mozilla62
Tracking Status
firefox-esr52 --- unaffected
firefox-esr60 62+ verified
firefox60 --- wontfix
firefox61 --- verified
firefox62 + verified

People

(Reporter: ke5trel, Assigned: jya)

Details

(Keywords: regression)

Attachments

(1 file)

First reported here:
https://www.reddit.com/r/firefox/comments/8rq1fr/video_stuttering_issue_with_specific/

Two clips from the same stream at the same time, one has momentary freezes of 1-2 seconds at certain timestamps and the other doesn't.
https://clips.twitch.tv/SolidRichCrabs4Head - freezes
https://clips.twitch.tv/SmoothGracefulDuckPogChamp - no freezes

Freezes only happen at 1080p quality setting.

Tested 60.0.2 and 62.0a1 on Ubuntu 18.04 with new profiles.

No freezes on Chromium 67.
Please provide a copy of the about:support output...
Flags: needinfo?(ke5trel)
And which version of ffmpeg are you using?
It also happens on Windows 10 1803 which uses WMF decoder.

MediaInfo (Windows): https://pastebin.mozilla.org/9087940

No hardware decoding used on Linux or Windows.

Chromium 67 with hardware decoding disabled is smooth.

Software decoding uses 50% CPU on Firefox and 30% CPU on Chromium.

CPU usage is steady for the smooth video and fluctuates for the freezing video.
Flags: needinfo?(ke5trel)
When HW decoding on Windows is switched off in Chrome, it uses ffmpeg to decode which is likely more optimised than WMF H264 decoder.


Enable HW layout on Ubuntu, it will make play-back much better. In about:config set layers.acceleration.force-enabled to true and restart Firefox
Is your problem on Ubuntu or Windows?
Both. layers.acceleration.force-enabled and gfx.webrender.all make no difference on Linux or Windows, nor does playing the video file directly in a tiny window, it still has the same 1-2 second freezes every few seconds.
Found an old build with it working smoothly. First bad build is 20170602, no further build data available for refinement.

Regression window:
https://hg.mozilla.org/mozilla-central/pushloghtml?fromchange=bdb2387396b4a74dfefb7c983733eed3625e906a&tochange=87c745019518b1d6cd782534f2553721e5735657
Has Regression Range: --- → yes
Whiteboard: regression
can reproduce on mac.
Assignee: nobody → jyavenard
Priority: -- → P2
Keywords: regression
Whiteboard: regression
Hoping we can fix this in 62 beta, tracking for 62.
I'm guessing the regression was introduced by bug 1313398. What could be happening is that it believes the SPS/PPS inband is changing and that causes a new decoder to be created, which means frame would be skipped until the next keyframe.

Problem like this are typically due to bad encoding.
So it is as I expected...

This streams has SPS NAL inband and they change from one keyframe to the next.

The SPS's chroma_loc_info_present_flag change from true to false and vice-versa... Weirder even is that the SPS changes on non-keyframe...

So a new decoder is created, and we have to wait for the next keyframe.

Maybe we could simply ignore SPS change on non-keyframe...
Some invalid streams contain SPS changes and those appear to only occur on non-keyframe, this cause all frames to be dropped until the next keyframe is found. This result in apparent freezes.

While it is theoretically possible to have SPS changes inband on non-keyframe those should be very rare (I've never seen one). The content would have been invalid anyway in an non-fragmented mp4.

So we now only check for a SPS change on keyframe. This would cause no affect on either windows, android or ffmpeg as those decoders handle format change fine. The mac decoder could however show garbled frames temporarily.
Comment on attachment 8986532 [details]
Bug 1469257 - [H264] Only check for SPS changes on keyframe

Bryce Van Dyk (:bryce) has approved the revision.

https://phabricator.services.mozilla.com/D1733
Attachment #8986532 - Flags: review+
Pushed by jyavenard@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/ddde6ad6d9e6
[H264] Only check for SPS changes on keyframe r=bryce
https://hg.mozilla.org/mozilla-central/rev/ddde6ad6d9e6
Status: NEW → RESOLVED
Closed: 6 years ago
Resolution: --- → FIXED
Target Milestone: --- → mozilla62
Could you test with today's nightly (build id 20180621013659 and later) to see if the problem is fixed for you?

Thank you
Flags: needinfo?(ke5trel)
Comment on attachment 8986532 [details]
Bug 1469257 - [H264] Only check for SPS changes on keyframe

Approval Request Comment
[Feature/Bug causing the regression]: 1313398
[User impact if declined]: stuttering, frozen frames with some videos on popular site.
[Is this code covered by automated tests?]: no
[Has the fix been verified in Nightly?]: not yet, manually checked
[Needs manual test from QE? If yes, steps to reproduce]:  follow instructions in bug description
[List of other uplifts needed for the feature/fix]: no
[Is the change risky?]: no
[Why is the change risky/not risky?]: The commit message explain the risks and downside. Ultimately, checking the changes on stream content is something we never used to do until recently. So we limit the scope at which we check for changes. Bug 1313398 was manually checked to make sure it was still fixed.
[String changes made/needed]: none

[Approval Request Comment]
If this is not a sec:{high,crit} bug, please state case for ESR consideration:
User impact if declined: 
Fix Landed on Version:
Risk to taking this patch (and alternatives if risky): 
String or UUID changes made by this patch: 

See https://wiki.mozilla.org/Release_Management/ESR_Landing_Process for more info.
Attachment #8986532 - Flags: approval-mozilla-esr60?
Attachment #8986532 - Flags: approval-mozilla-beta?
Verified as fixed on Nightly 62.0a1 (20180621013659) on Ubuntu 18.04.
Status: RESOLVED → VERIFIED
Flags: needinfo?(ke5trel)
Comment on attachment 8986532 [details]
Bug 1469257 - [H264] Only check for SPS changes on keyframe

Fx61 is already in the RC stage (shipping Tuesday). It's too late for 61.0 at this point, but I'll leave it on the radar for possibly dot-release consideration.
Attachment #8986532 - Flags: approval-mozilla-beta? → approval-mozilla-release?
Comment on attachment 8986532 [details]
Bug 1469257 - [H264] Only check for SPS changes on keyframe

Approved for 61.0.1.
Attachment #8986532 - Flags: approval-mozilla-release? → approval-mozilla-release+
Flags: qe-verify+
I have managed to reproduce this bug on an affected Release build 60.0.2 (2018-18-06), using the STR from comment 0. 

This is verified fixed on Beta 62.0b5 (20180702164905) and Release 61.0.1 (20180704003137) on the following OSes: Windows 10 x64, Ubuntu 16.04 x64 and macOS 10.12.
Comment on attachment 8986532 [details]
Bug 1469257 - [H264] Only check for SPS changes on keyframe

Approved for ESR 60.2 also.
Attachment #8986532 - Flags: approval-mozilla-esr60? → approval-mozilla-esr60+
Flags: qe-verify+
I have also verified this bug on build 60.1.1esr (buildID=20180724124937), using the STR from comment 0. 

This is verified fixed on the following OSes: Windows 10 x64, Ubuntu 16.04 x64 and macOS 10.12.
Flags: qe-verify+
You need to log in before you can comment on or make changes to this bug.