Implement AnimationFrameProvider for dedicated workers
Categories
(Core :: DOM: Workers, enhancement, P3)
Tracking
()
| Tracking | Status | |
|---|---|---|
| firefox97 | --- | fixed |
People
(Reporter: aosmond, Assigned: aosmond)
References
(Blocks 1 open bug)
Details
(Keywords: dev-doc-needed)
Attachments
(5 files)
https://html.spec.whatwg.org/multipage/imagebitmap-and-animations.html#animation-frames
It is a common pattern that OffscreenCanvas workers use requestAnimationFrame in order to drive the rendering pipeline. We need to implement this in order to ship OffscreenCanvas support.
This will consistent of subscribing to vsync notifications inside a DedicatedWorkerGlobalScope. I needs to be modulated based on the visibility of the owning window.
|
Assignee | |
Comment 1•7 months ago
|
||
This patch splits out AnimationFrameProvider from the Document WebIDL to
allow the workers to implement it. It also splits out a helper class to
manage the requestAnimationFrame callbacks which may be reused on a
worker thread.
|
|
Assignee | |
Comment 2•7 months ago
|
||
requestAnimationFrame callbacks are supposed to be delayed if the
worker's owning document is no longer visible. This patch adds the
ability for a WorkerGlobalScope to listen to changes in the visiblility
state of the BrowserChild which propogates those changes.
|
|
Assignee | |
Comment 3•7 months ago
|
||
VsyncChild is main thread only, and we would like to reuse PVsync on the
worker threads via PBackgroundChild which already implements it. This
patch does the necessary refactoring to have multiple implementations of
PVsyncChild.
|
|
Assignee | |
Comment 4•7 months ago
|
||
requestAnimationFrame callbacks are supposed to be synchronized with the
vsync events into the refresh driver for the window. This patch adds an
IPDL actor implementation for workers to subscribe as necessary to vsync
events.
|
|
Assignee | |
Comment 5•7 months ago
|
||
|
|
Assignee | |
Comment 6•7 months ago
|
||
|
|
Assignee | |
Comment 7•7 months ago
|
||
I've also been testing this with a demo (combined with my OffscreenCanvas patches, not landed):
https://chrisprice.io/offscreen-canvas/?100000
Like Chrome, our implementation doesn't depend on the main thread (aside from the BrowserChild visibility change events which should be relatively infrequent by comparison), so rAF continues to drive the WebGL animation even when the main thread is blocked by the alert box.
|
||
Comment 8•7 months ago
|
||
Hi Andrew, great to see this! Does this implicitly fix also bug 1203382 ? Thank you!
|
|
Assignee | |
Comment 9•7 months ago
|
||
(In reply to Jens Stutte [:jstutte] from comment #8)
Hi Andrew, great to see this! Does this implicitly fix also bug 1203382 ? Thank you!
Oh I missed it and it is blocked against the same meta bug :). Yes, that is a duplicate bug.
|
||
Updated•7 months ago
|
|
||
Updated•7 months ago
|
|
||
Comment 11•6 months ago
|
||
Pushed by aosmond@mozilla.com: https://hg.mozilla.org/integration/autoland/rev/562a1e8e5ac3 Part 1. Split out plumbing for AnimationFrameProvider from Document. r=dom-worker-reviewers,smaug https://hg.mozilla.org/integration/autoland/rev/d9aef3e9797e Part 2. Add WorkerDocumentListener to get visibility events. r=dom-worker-reviewers,gfx-reviewers,jgilbert,smaug https://hg.mozilla.org/integration/autoland/rev/dc4503052cf1 Part 3. Rework VsyncChild to allow multiple implementations. r=gfx-reviewers,jgilbert https://hg.mozilla.org/integration/autoland/rev/894ba6b7b68f Part 4. Add VsyncWorkerChild to get vsync events. r=gfx-reviewers,dom-worker-reviewers,jgilbert,smaug https://hg.mozilla.org/integration/autoland/rev/3dde5ddb65e5 Part 5. Implement AnimationFrameProvider for dedicated workers. r=dom-worker-reviewers,gfx-reviewers,jgilbert,smaug
|
||
Comment 12•6 months ago
|
||
Backed out for bp-hybrid failures and others, along with Bug 1736177
-
backout: https://hg.mozilla.org/integration/autoland/rev/d38a89ced939d330f8f9cd88926297e7c158ce72
-
failure logs:
- chrome/common/ipc_message_utils.h:116:33: error: no member named 'Write' in 'IPC::ParamTraits<mozilla::gfx::IntRectTyped<mozilla::gfx::UnknownUnits>>'
- TEST-UNEXPECTED-FAIL | dom/media/webrtc/tests/mochitests/test_peerConnection_captureStream_canvas_webgl.html | Test timed out. -
- TEST-UNEXPECTED-FAIL | dom/serviceworkers/test/test_serviceworker_interfaces.html | undefined: If this is failing: DANGER, are you sure you want to expose the new interface WebGLQuery to all webpages as a property on self? Do not make a change to this file without a review from a DOM peer for that specific change!!! (or a JS peer for changes to ecmaGlobals)
|
|
||
Comment 13•6 months ago
|
||
Pushed by aosmond@mozilla.com: https://hg.mozilla.org/integration/autoland/rev/85f1ffa0e689 Part 1. Split out plumbing for AnimationFrameProvider from Document. r=dom-worker-reviewers,smaug https://hg.mozilla.org/integration/autoland/rev/33119564bca1 Part 2. Add WorkerDocumentListener to get visibility events. r=dom-worker-reviewers,gfx-reviewers,jgilbert,smaug https://hg.mozilla.org/integration/autoland/rev/f7a9a9186e2b Part 3. Rework VsyncChild to allow multiple implementations. r=gfx-reviewers,jgilbert https://hg.mozilla.org/integration/autoland/rev/4ec1575e57c5 Part 4. Add VsyncWorkerChild to get vsync events. r=gfx-reviewers,dom-worker-reviewers,jgilbert,smaug https://hg.mozilla.org/integration/autoland/rev/e59ba9934ece Part 5. Implement AnimationFrameProvider for dedicated workers. r=dom-worker-reviewers,gfx-reviewers,jgilbert,smaug
|
|
Assignee | |
Updated•6 months ago
|
|
||
Comment 14•6 months ago
|
||
| bugherder | ||
https://hg.mozilla.org/mozilla-central/rev/85f1ffa0e689
https://hg.mozilla.org/mozilla-central/rev/33119564bca1
https://hg.mozilla.org/mozilla-central/rev/f7a9a9186e2b
https://hg.mozilla.org/mozilla-central/rev/4ec1575e57c5
https://hg.mozilla.org/mozilla-central/rev/e59ba9934ece
|
||
Comment 15•5 months ago
•
|
||
This seems to cause build failures with some unified builds:
0:37.94 In file included from UnifiedBindings10.cpp:2:
0:37.94 In file included from /media/external/dev/gecko-dev/obj-b2g-desktop/dom/bindings/HTMLPictureElementBinding.cpp:19:
0:37.94 In file included from /media/external/dev/gecko-dev/obj-b2g-desktop/dist/include/mozilla/dom/BindingUtils.h:30:
0:37.94 In file included from /media/external/dev/gecko-dev/obj-b2g-desktop/dist/include/mozilla/dom/Document.h:48:
0:37.94 In file included from /media/external/dev/gecko-dev/obj-b2g-desktop/dist/include/mozilla/dom/AnimationFrameProvider.h:10:
0:37.94 In file included from /media/external/dev/gecko-dev/obj-b2g-desktop/dist/include/mozilla/dom/AnimationFrameProviderBinding.h:12:
0:37.94 /media/external/dev/gecko-dev/obj-b2g-desktop/dist/include/mozilla/dom/ToJSValue.h:133:10: error: use of undeclared identifier 'MaybeWrapValue'
0:37.94 return MaybeWrapValue(aCx, aValue);
0:37.94 ^
0:37.94 /media/external/dev/gecko-dev/obj-b2g-desktop/dist/include/mozilla/dom/ToJSValue.h:153:21: error: use of undeclared identifier 'NonRefcountedDOMObject'
0:37.94 std::is_base_of<NonRefcountedDOMObject, T>::value, bool>
0:37.94 ^
0:37.94 /media/external/dev/gecko-dev/obj-b2g-desktop/dist/include/mozilla/dom/ToJSValue.h:181:21: error: use of undeclared identifier 'NonRefcountedDOMObject'
0:37.94 std::is_base_of<NonRefcountedDOMObject, T>::value, bool>
0:37.94 ^
0:37.94 /media/external/dev/gecko-dev/obj-b2g-desktop/dist/include/mozilla/dom/ToJSValue.h:253:10: error: use of undeclared identifier 'XPCOMObjectToJsval'
0:37.94 return XPCOMObjectToJsval(aCx, scope, helper, &iid, true, aValue);
0:37.94 ^
0:37.94 /media/external/dev/gecko-dev/obj-b2g-desktop/dist/include/mozilla/dom/ToJSValue.h:290:10: error: use of undeclared identifier 'MaybeWrapValue'
0:37.94 return MaybeWrapValue(aCx, aValue);
0:37.94 ^
0:37.95 /media/external/dev/gecko-dev/obj-b2g-desktop/dist/include/mozilla/dom/ToJSValue.h:296:10: error: use of undeclared identifier 'MaybeWrapValue'
0:37.95 return MaybeWrapValue(aCx, aValue);
0:37.95 ^
0:37.95 /media/external/dev/gecko-dev/obj-b2g-desktop/dist/include/mozilla/dom/ToJSValue.h:305:10: error: use of undeclared identifier 'MaybeWrapValue'
0:37.95 return MaybeWrapValue(aCx, aValue);
0:37.95 ^
0:37.95 /media/external/dev/gecko-dev/obj-b2g-desktop/dist/include/mozilla/dom/ToJSValue.h:313:10: error: use of undeclared identifier 'MaybeWrapValue'
0:37.95 return MaybeWrapValue(aCx, aValue);
0:37.95 ^
0:37.95 /media/external/dev/gecko-dev/obj-b2g-desktop/dist/include/mozilla/dom/ToJSValue.h:322:10: error: use of undeclared identifier 'MaybeWrapObjectOrNullValue'
0:37.95 return MaybeWrapObjectOrNullValue(aCx, aValue);
0:37.95 ^
0:37.95 9 errors generated.
As fas as I can tell, this is because BindingUtils.h includes Document.h, which itself pulls AnimationFrameProviderBinding.h : at this point we need MaybeWrapValue() from BindingUtils.h, but the include guard prevents it from being included again.
|
|
||
Comment 16•5 months ago
|
||
Feel free to borrow the fix I landed on the b2g fork at https://github.com/kaiostech/gecko-b2g/commit/38879cc1f1a393d9db4bdea36537bd37ddeb0097
|
||
Comment 17•5 months ago
|
||
- A reason given for doing this work was to enable generating frames from an OffscreenCanvas.
- The PR for that work was backed out in https://bugzilla.mozilla.org/show_bug.cgi?id=1746110
- Is this a dependency? i.e. is this feature pointless without it? Does this feature require the same preference? I'm trying to work out if this is worth documenting now if the OffscreenCanvas doesn't work yet.
- When this comes along, my understanding of the change is that we will be adding the methods
requestAnimationFrameandcancelAnimationFrameto DedicatedWorkerGlobalScope. Is that right? - These methods are already present and documented for Window - see requestAnimationFrame and cancelAnimationFrame. Would there be any particular difference in the dedicated worker version? My guess is that it should be very similar except that you'd probably do your animation on an offscreen canvas, and presumably then need to transfer the canvas to the worker thread and back similar to https://developer.mozilla.org/en-US/docs/Web/API/OffscreenCanvas#asynchronous_display_of_frames_produced_by_an_offscreencanvas . Do you have any working examples/test code that show how to do this already?
|
|
Assignee | |
Comment 18•4 months ago
|
||
-
I am splitting the pref out in bug 1749323. It could have value to some content authors without OffscreenCanvas, but that is the use case that drove its implementation. Once said bug lands, and we have either complete OffscreenCanvas (including Canvas 2D) or we have partners that want just WebGL (on the domain list in said bug), it will get turned on in that nightly. Hopefully 98.
-
Yes, that is correct.
-
It is driven by the same vsync timer, so it should behave very similarly. There is more machinery on the window version, but I would expect them to functionally happen around the same time, with no guarantee as to which fires first.
An example that uses it (you need to force on OffscreenCanvas today) is at https://chrisprice.io/offscreen-canvas/?100000
https://chrisprice.io/offscreen-canvas/index.js
https://chrisprice.io/offscreen-canvas/worker.js
It doesn't explicitly transfer content between the worker and the main thread, because it relies upon HTMLCanvasElement.transferControlToOffscreen. This creates a binding between the canvas element and the OffscreenCanvas object used on the worker thread, allowing the browser to handle the heavy lifting efficiently for display. I'm sure you could rig up something by doing a post of an ImageBitmap or similar as well.
Description
•