1800882 - (interop-2024-requestVideoFrameCallback) Implement HTMLVideoElement.requestVideoFrameCallback()

Status: RESOLVED FIXED

(Core :: Audio/Video, enhancement)

Description

[Dennis Schubert [:denschub]]

The Mozilla standards position for that is positive. According to the MDN Browser Compat Data, all other browsers are shipping requestVideoFrameCallback and cancelVideoFrameCallback already.

Comment 1

[Jim Mathies [:jimm]]

no op

Comment 3

[Jim Mathies [:jimm]]

Attachment: rvfc.patch

Comment 4

[Andrew Osmond [:aosmond] (he/him)]

Attachment: Bug 1800882 - Implement HTMLVideoElement.requestVideoFrameCallback.

See https://wicg.github.io/video-rvfc/ for standard details.

Comment 5

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/e2de2a86795f
Implement HTMLVideoElement.requestVideoFrameCallback. r=webidl,media-playback-reviewers,emilio,ErichDonGubler,padenot

Comment 6

[tszentpeteri]

Backed out for causing mochitest failures related to animations on compositor.

• Backout link
• Push with failures
• Failure Log:

[task 2024-07-24T20:15:54.287Z] 20:15:54     INFO - TEST-START | dom/animation/test/chrome/test_animation_performance_warning.html
[task 2024-07-24T20:15:54.288Z] 20:15:54     INFO - GECKO(1306) | [Parent 1306, Main Thread] WARNING: NS_ENSURE_SUCCESS(rv, rv) failed with result 0x80004005 (NS_ERROR_FAILURE): file /builds/worker/checkouts/gecko/chrome/nsChromeRegistry.cpp:182
[task 2024-07-24T20:15:54.288Z] 20:15:54     INFO - GECKO(1306) | [Parent 1306, Main Thread] WARNING: 'NS_FAILED(rv)', file /builds/worker/checkouts/gecko/chrome/nsChromeProtocolHandler.cpp:73
[task 2024-07-24T20:15:54.289Z] 20:15:54     INFO - GECKO(1306) | [Parent 1306, Main Thread] WARNING: Failed to retarget HTML data delivery to the parser thread.: file /builds/worker/checkouts/gecko/parser/html/nsHtml5StreamParser.cpp:1215
[task 2024-07-24T20:15:54.290Z] 20:15:54     INFO - TEST-INFO | started process screentopng
[task 2024-07-24T20:15:54.516Z] 20:15:54     INFO - TEST-INFO | screentopng: exit 0
[task 2024-07-24T20:15:54.516Z] 20:15:54     INFO - TEST-UNEXPECTED-FAIL | dom/animation/test/chrome/test_animation_performance_warning.html | animations on compositor - animations on compositor: assert_equals: runningOnCompositor property should match expected true but got false
[task 2024-07-24T20:15:54.517Z] 20:15:54     INFO - TEST-PASS | dom/animation/test/chrome/test_animation_performance_warning.html | Bug 1196114 - Test metadata related to which animation properties are running on the compositor - Bug 1196114 - Test metadata related to which animation properties are running on the compositor: Elided 67 passes or known failures.

Comment 7

[Emilio Cobos Álvarez (:emilio)]

Attachment: Bug 1800882 - Clean up rvfc a bit to match the spec closer. r=aosmond!

Deal with throttled docs only once upfront, since
ShouldThrottleFrameRequests is kind of expensive (doing a full
IntersectionObserver check).

Add proper spec quotes too, while at it. This also fixes the failure
that got the previous patch backed out.

Comment 8

[Emilio Cobos Álvarez (:emilio)]

Attachment: Bug 1800882 - Keep includes in AnimationFrameProvider.h down. r=aosmond!

This fixes rusttests. Otherwise we include HTMLVideoElement.h from
Document.h, which includes a bunch of media headers, which causes
rusttest failures because
https://searchfox.org/mozilla-central/rev/d353cfa1fbd207e13dc974f30e5f88535a4303ae/dom/media/platforms/EncoderConfig.h#95
hits https://github.com/rust-lang/rust-bindgen/issues/380.

We could hide those types from rust but it seems slightly nicer to keep
Document.h lean, since it's included in a gazillion places.

Comment 9

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/548a1763e689
Implement HTMLVideoElement.requestVideoFrameCallback. r=webidl,media-playback-reviewers,emilio,ErichDonGubler,padenot
https://hg.mozilla.org/integration/autoland/rev/1e76fd470d00
Clean up rvfc a bit to match the spec closer. r=aosmond
https://hg.mozilla.org/integration/autoland/rev/8acda0361e97
Keep includes in AnimationFrameProvider.h down. r=aosmond

Comment 10

[Atila Butkovits]

Backed out for causing webgpu failures.

Backout link: https://hg.mozilla.org/integration/autoland/rev/c8b2755739da05a6986c6d43cdd28718b4ac390d

Push with failures: https://treeherder.mozilla.org/jobs?repo=autoland&resultStatus=testfailed%2Cbusted%2Cexception%2Cretry%2Cusercancel&revision=8acda0361e97220b903120da806455c114180678&selectedTaskRun=FWLxSSnvR0iuB1AXQ51wSA.0

Failure log: https://treeherder.mozilla.org/jobs?repo=autoland&resultStatus=testfailed%2Cbusted%2Cexception%2Cretry%2Cusercancel&revision=8acda0361e97220b903120da806455c114180678&selectedTaskRun=FWLxSSnvR0iuB1AXQ51wSA.0

Comment 11

[Florin Bilt]

(In reply to Atila Butkovits from comment #10)

Backed out for causing webgpu failures.

Backout link: https://hg.mozilla.org/integration/autoland/rev/c8b2755739da05a6986c6d43cdd28718b4ac390d

Push with failures: https://treeherder.mozilla.org/jobs?repo=autoland&resultStatus=testfailed%2Cbusted%2Cexception%2Cretry%2Cusercancel&revision=8acda0361e97220b903120da806455c114180678&selectedTaskRun=FWLxSSnvR0iuB1AXQ51wSA.0

Failure log: https://treeherder.mozilla.org/jobs?repo=autoland&resultStatus=testfailed%2Cbusted%2Cexception%2Cretry%2Cusercancel&revision=8acda0361e97220b903120da806455c114180678&selectedTaskRun=FWLxSSnvR0iuB1AXQ51wSA.0

Perfherder has detected a browsertime performance change from push b81788b1a72473564135c6dd38ffa0e89a21d308.

Regressions:

Ratio	Test	Platform	Options	Absolute values (old vs new)	Performance Profiles
168%	speedometer3 Editor-TipTap/Highlight/Async	macosx1400-64-shippable-qr	fission webrender	0.63 -> 1.69	Before/After
87%	speedometer3 Editor-TipTap/Highlight/Async	linux1804-64-nightlyasrelease-qr	fission webrender	1.83 -> 3.43	Before/After
4%	speedometer3 TodoMVC-Preact-Complex-DOM/CompletingAllItems/Sync	macosx1400-64-shippable-qr	fission webrender	0.84 -> 0.87	Before/After

Improvements:

Ratio	Test	Platform	Options	Absolute values (old vs new)	Performance Profiles
38%	speedometer3 Editor-TipTap/Long/Async	macosx1400-64-shippable-qr	fission webrender	1.30 -> 0.80	Before/After
16%	speedometer3 Editor-TipTap/Long/Async	linux1804-64-nightlyasrelease-qr	fission webrender	4.64 -> 3.88	Before/After
6%	speedometer3 Editor-TipTap/Highlight/Sync	macosx1400-64-shippable-qr	fission webrender	31.75 -> 29.86	Before/After
4%	speedometer3 Editor-TipTap/total	macosx1400-64-shippable-qr	fission webrender	69.56 -> 67.01	Before/After
3%	speedometer3 Editor-TipTap/Highlight/total	macosx1400-64-shippable-qr	fission webrender	32.44 -> 31.37	Before/After
3%	speedometer3 Editor-TipTap/Long/total	macosx1400-64-shippable-qr	fission webrender	36.84 -> 35.65	Before/After

As author of one of the patches included in that push, we need your help to address this regression.
Details of the alert can be found in the alert summary, including links to graphs and comparisons for each of the affected tests. Please follow our guide to handling regression bugs and let us know your plans within 3 business days, or the patch(es) may be backed out in accordance with our regression policy.

If you need the profiling jobs you can trigger them yourself from treeherder job view or ask a sheriff to do that for you.

You can run these tests on try with ./mach try perf --alert 1489

For more information on performance sheriffing please see our FAQ.

Comment 12

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/3ac74b4d6534
Implement HTMLVideoElement.requestVideoFrameCallback. r=webidl,media-playback-reviewers,emilio,ErichDonGubler,padenot
https://hg.mozilla.org/integration/autoland/rev/fa46622c4ec7
Clean up rvfc a bit to match the spec closer. r=aosmond
https://hg.mozilla.org/integration/autoland/rev/fd02fd0205ab
Keep includes in AnimationFrameProvider.h down. r=aosmond

Comment 13

[Sandor Molnar[:smolnar]]

Backed out for causing webcodecs failures

Backout link: https://hg.mozilla.org/integration/autoland/rev/de8daa863ecfe1b5b6f081143389f87ea0189a6a

Push with failures

Failure log -> TEST-UNEXPECTED-TIMEOUT | /webcodecs/videoFrame-canvasImageSource.html

Comment 14

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/40274941ace1
Implement HTMLVideoElement.requestVideoFrameCallback. r=webidl,media-playback-reviewers,emilio,ErichDonGubler,padenot
https://hg.mozilla.org/integration/autoland/rev/6418025eb4ec
Clean up rvfc a bit to match the spec closer. r=aosmond
https://hg.mozilla.org/integration/autoland/rev/65d739fc547d
Keep includes in AnimationFrameProvider.h down. r=aosmond

Comment 15

[Norisz Fay [:noriszfay]]

https://hg.mozilla.org/mozilla-central/rev/40274941ace1
https://hg.mozilla.org/mozilla-central/rev/6418025eb4ec
https://hg.mozilla.org/mozilla-central/rev/65d739fc547d

Comment 16

[Jim Mathies [:jimm]]

Woot! \o/