Open Bug 1931328 Opened 13 days ago Updated 15 hours ago

Video with createMediaElementSource becomes permanently sluggish / freezes after switching playbackRate a few times

Categories

(Core :: Audio/Video: Playback, defect)

Firefox 134
defect

Tracking

()

Tracking Status
firefox-esr115 --- wontfix
firefox-esr128 --- affected
firefox132 --- wontfix
firefox133 --- wontfix
firefox134 --- affected
firefox135 --- affected

People

(Reporter: wofwca, Assigned: sotaro)

References

(Blocks 1 open bug, Regression)

Details

(Keywords: nightly-community, regression)

Attachments

(1 file)

Attached video demo.mp4

User Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:132.0) Gecko/20100101 Firefox/132.0

Steps to reproduce:

  1. Download https://download.blender.org/demo/movies/BBB/bbb_sunflower_1080p_30fps_normal.mp4.zip
  2. Open it as a local file through Firefox and start playback
  3. Execute the following code:
ctx = new AudioContext();
v = document.querySelector('video');
src = ctx.createMediaElementSource(v);
src.connect(ctx.destination);

switchesLeft = 20;
intervalId = setInterval(() => {
	v.playbackRate = 8;
 	// v.playbackRate = 8.25; // (a muted speed) also makes the bug appear.

	switchesLeft--;
	if (switchesLeft <= 0) {
		clearInterval(intervalId);
		v.playbackRate = 1;
		console.log('back to normal speed');
		return;
	}

	setTimeout(() => {
		v.playbackRate = 1.25;
	}, 100)
}, 300)

Actual results:

After a few playbackRate changes the video becomes super sluggish (FPS drops to about 0.1).

  • The behavior seems to remain even after we go back to playbackRate = 1.
  • The audio seems to be unaffected and plays normally.
  • Pausing + unpausing the video restores normal FPS.
  • Seeking also restores normal behavior.
  • I tried the same with one YouTube video I was unable to reproduce this.
  • Lower playback rates (below ~6) don't always make this bug appear.
  • The CPU doesn't seem to be overloaded. None of the cores' usage goes above 50%.
  • Sometimes switching playbackRate even once makes this bug show up. But sometimes the normal behavior restores after a while of playing back the video at the rate of 1.

I was able to reproduce this on the following Firefox versions: 134.0a1, 132.0.1, 130.0b9, 128.4.0esr, 107.0.1, 103.0.2, 103.0.
On 92.0b9, 97.0b9, 100.0.2, 101.0.1, 102.9.0esr I was unable to reproduce the bug at all.

So it looks like 103.0 is the first version with this bug.

This bug significantly affects an extension of mine called "Jump Cutter", where switching playback rate is the entire purpose of the extension. See original bug report: https://github.com/WofWca/jumpcutter/issues/186

Expected results:

Video doesn't freeze.
If it's impossible to maintain high FPS on high playbackRate, it at least should go back to normal after the playbackRate restores to normal.

The Bugbug bot thinks this bug should belong to the 'Core::Audio/Video: Playback' component, and is moving the bug to that component. Please correct in case you think the bot is wrong.

Component: Untriaged → Audio/Video: Playback
Product: Firefox → Core

About the profile: I executed my JS snippet at the start. In the last few seconds, after running at low FPS, I paused and unpaused the video.

Profile with media preset logging: https://share.firefox.dev/4eqeRgs

Status: UNCONFIRMED → NEW
Ever confirmed: true

I gave this a try on both Linux and Windows and was unable to replicate. I tried using the 4k/60fps version of Buck Bunny and increased the playback speed/number of iterations as well.

Would you be able to install mozregression to help narrow down the changes that caused this bug? When you start mozregression, click the icon in the top left corner of the window that looks like a pair of scissors. Leave "basic configuration" and "profile selection" as they are, clicking next for each. On the build selection screen, click the "release" options under the dropdowns, and then put 102 as the last known good build and 103 as the first known bad build. Click finish, and then Firefox will launch. Replicate the buggy behavior, close Firefox, and mozregression will ask you whether it was a good or bad build. After several iterations of this, it will narrow down the commit which caused the bug, showing something like the following:

2024-04-02T11:28:04.770000: DEBUG : Found commit message:
Bug 1887305 - Move ValueToIdentifier call before requireDebuggee check. r=iain
ValueToIdentifier can invoke JS code that removes the debuggee, so we have to reorder these operations.
Differential Revision: https://phabricator.services.mozilla.com/D205835
2024-04-02T11:28:04.770000: DEBUG : Did not find a branch, checking all integration branches
2024-04-02T11:28:04.770000: INFO : The bisection is done.
2024-04-02T11:28:04.770000: INFO : Stopped

Please paste that output here. Thank you!

Flags: needinfo?(wofwca)

I can reproduce the issue with STR in Comment#0 on Nightly134.a1 Windows11 (W/ AMD Radeon Graphics).

Regression window:
https://hg.mozilla.org/integration/autoland/pushloghtml?fromchange=5886d166601b6c778a05c9f7028d4bdafd7b829c&tochange=d52c1ab80f0123ce77837924151f04efcab45623

Regressed by: 1899450

Thanks Alice! :sotaro, would you have any thoughts here?

Flags: needinfo?(wofwca) → needinfo?(sotaro.ikeda.g)

Set release status flags based on info from the regressing bug 1899450

Hi wofwca, can you attach about:support to this bug?

Flags: needinfo?(wofwca)
Blocks: 1769643

(In reply to wofwca from comment #2)

I have performed profiling as per https://bugzilla.mozilla.org/show_bug.cgi?id=1807968#c8: https://share.firefox.dev/40M6sR5

The symptom seems similar to Bug 1929389 and Bug 1923697. The video decoding was blocked WMFVideoMFTManager::Output().

Flags: needinfo?(sotaro.ikeda.g)
See Also: → 1929389, 1923697

(In reply to az [:az] from comment #5)

Would you be able to install mozregression

I ran a bisection, thanks for the instructions! However, it's a bit weird because it turned out that the bad build isn't actually 103. Perhaps the release build and dev builds are different in this regard? I bisected between 92 and 104.
The logs got truncated, unfortunately, but the last part is:

2024-11-15T11:45:57.265000: WARNING : Skipping build 6ff2b7d52aa3: Unable to find build info using the taskcluster route 'gecko.v2.mozilla-central.shippable.revision.6ff2b7d52aa3bdee68ed01cfd4b653253f0b4d51.firefox.win64-opt'
2024-11-15T11:46:00.091000: INFO : Stopped

It narrowed it down to https://hg.mozilla.org/mozilla-central/pushloghtml?fromchange=0992c635a8c53621810151649a24429836987fd6&tochange=b8732a1c1fa931fc3cc5369e31a64db459f60c83

Last known good build: https://archive.mozilla.org/pub/firefox/nightly/2022/04/2022-04-06-19-03-21-mozilla-central/firefox-101.0a1.en-US.win64.zip
First known bad build: https://archive.mozilla.org/pub/firefox/nightly/2022/04/2022-04-07-21-29-10-mozilla-central/firefox-101.0a1.en-US.win64.zip

I confirmed the problem with latest nightly with AMD GPU on my Win11 PC. I take the bug.

pref media.wmf.zero-copy-nv12-textures = false addressed the problem for me.

But the following did not address the problem

  • pref gfx.webrender.compositor = false
  • pref gfx.webrender.dcomp-win.enabled = false
Assignee: nobody → sotaro.ikeda.g

pref media.wmf.zero-copy-nv12-textures = false addressed the problem for me.

I confirm this.

Flags: needinfo?(wofwca)

Thank you for the checking!

Zero copy video is enabled until early beta with AMD GPUs. It does not affect to release.

Set release status flags based on info from the regressing bug 1899450

You need to log in before you can comment on or make changes to this bug.

Attachment

General

Creator:
Created:
Updated:
Size: