Closed Bug 1738971 Opened 7 months ago Closed 6 months ago

Implement AnimationFrameProvider for dedicated workers

Categories

(Core :: DOM: Workers, enhancement, P3)

enhancement

Tracking

()

RESOLVED FIXED
97 Branch
Tracking Status
firefox97 --- fixed

People

(Reporter: aosmond, Assigned: aosmond)

References

(Blocks 1 open bug)

Details

(Keywords: dev-doc-needed)

Attachments

(5 files)

https://html.spec.whatwg.org/multipage/imagebitmap-and-animations.html#animation-frames

It is a common pattern that OffscreenCanvas workers use requestAnimationFrame in order to drive the rendering pipeline. We need to implement this in order to ship OffscreenCanvas support.

This will consistent of subscribing to vsync notifications inside a DedicatedWorkerGlobalScope. I needs to be modulated based on the visibility of the owning window.

This patch splits out AnimationFrameProvider from the Document WebIDL to
allow the workers to implement it. It also splits out a helper class to
manage the requestAnimationFrame callbacks which may be reused on a
worker thread.

requestAnimationFrame callbacks are supposed to be delayed if the
worker's owning document is no longer visible. This patch adds the
ability for a WorkerGlobalScope to listen to changes in the visiblility
state of the BrowserChild which propogates those changes.

VsyncChild is main thread only, and we would like to reuse PVsync on the
worker threads via PBackgroundChild which already implements it. This
patch does the necessary refactoring to have multiple implementations of
PVsyncChild.

requestAnimationFrame callbacks are supposed to be synchronized with the
vsync events into the refresh driver for the window. This patch adds an
IPDL actor implementation for workers to subscribe as necessary to vsync
events.

I've also been testing this with a demo (combined with my OffscreenCanvas patches, not landed):

https://chrisprice.io/offscreen-canvas/?100000

Like Chrome, our implementation doesn't depend on the main thread (aside from the BrowserChild visibility change events which should be relatively infrequent by comparison), so rAF continues to drive the WebGL animation even when the main thread is blocked by the alert box.

Hi Andrew, great to see this! Does this implicitly fix also bug 1203382 ? Thank you!

Flags: needinfo?(aosmond)

(In reply to Jens Stutte [:jstutte] from comment #8)

Hi Andrew, great to see this! Does this implicitly fix also bug 1203382 ? Thank you!

Oh I missed it and it is blocked against the same meta bug :). Yes, that is a duplicate bug.

Flags: needinfo?(aosmond)
Duplicate of this bug: 1203382
Attachment #9249037 - Attachment description: Bug 1738971 - Part 2. Add WorkerBrowserListener to get visibility events. → Bug 1738971 - Part 2. Add WorkerDocumentListener to get visibility events.
Pushed by aosmond@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/562a1e8e5ac3
Part 1. Split out plumbing for AnimationFrameProvider from Document. r=dom-worker-reviewers,smaug
https://hg.mozilla.org/integration/autoland/rev/d9aef3e9797e
Part 2. Add WorkerDocumentListener to get visibility events. r=dom-worker-reviewers,gfx-reviewers,jgilbert,smaug
https://hg.mozilla.org/integration/autoland/rev/dc4503052cf1
Part 3. Rework VsyncChild to allow multiple implementations. r=gfx-reviewers,jgilbert
https://hg.mozilla.org/integration/autoland/rev/894ba6b7b68f
Part 4. Add VsyncWorkerChild to get vsync events. r=gfx-reviewers,dom-worker-reviewers,jgilbert,smaug
https://hg.mozilla.org/integration/autoland/rev/3dde5ddb65e5
Part 5. Implement AnimationFrameProvider for dedicated workers. r=dom-worker-reviewers,gfx-reviewers,jgilbert,smaug
Pushed by aosmond@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/85f1ffa0e689
Part 1. Split out plumbing for AnimationFrameProvider from Document. r=dom-worker-reviewers,smaug
https://hg.mozilla.org/integration/autoland/rev/33119564bca1
Part 2. Add WorkerDocumentListener to get visibility events. r=dom-worker-reviewers,gfx-reviewers,jgilbert,smaug
https://hg.mozilla.org/integration/autoland/rev/f7a9a9186e2b
Part 3. Rework VsyncChild to allow multiple implementations. r=gfx-reviewers,jgilbert
https://hg.mozilla.org/integration/autoland/rev/4ec1575e57c5
Part 4. Add VsyncWorkerChild to get vsync events. r=gfx-reviewers,dom-worker-reviewers,jgilbert,smaug
https://hg.mozilla.org/integration/autoland/rev/e59ba9934ece
Part 5. Implement AnimationFrameProvider for dedicated workers. r=dom-worker-reviewers,gfx-reviewers,jgilbert,smaug
Flags: needinfo?(aosmond)

This seems to cause build failures with some unified builds:

 0:37.94 In file included from UnifiedBindings10.cpp:2:                                                                                                                                           
 0:37.94 In file included from /media/external/dev/gecko-dev/obj-b2g-desktop/dom/bindings/HTMLPictureElementBinding.cpp:19:                                                                       
 0:37.94 In file included from /media/external/dev/gecko-dev/obj-b2g-desktop/dist/include/mozilla/dom/BindingUtils.h:30:                                                                          
 0:37.94 In file included from /media/external/dev/gecko-dev/obj-b2g-desktop/dist/include/mozilla/dom/Document.h:48:                                                                              
 0:37.94 In file included from /media/external/dev/gecko-dev/obj-b2g-desktop/dist/include/mozilla/dom/AnimationFrameProvider.h:10:                                                                
 0:37.94 In file included from /media/external/dev/gecko-dev/obj-b2g-desktop/dist/include/mozilla/dom/AnimationFrameProviderBinding.h:12:                                                         
 0:37.94 /media/external/dev/gecko-dev/obj-b2g-desktop/dist/include/mozilla/dom/ToJSValue.h:133:10: error: use of undeclared identifier 'MaybeWrapValue'                                          
 0:37.94   return MaybeWrapValue(aCx, aValue);                                                                                                                                                    
 0:37.94          ^                                                                                                                                                                               
 0:37.94 /media/external/dev/gecko-dev/obj-b2g-desktop/dist/include/mozilla/dom/ToJSValue.h:153:21: error: use of undeclared identifier 'NonRefcountedDOMObject'                                  
 0:37.94     std::is_base_of<NonRefcountedDOMObject, T>::value, bool>                                                                                                                             
 0:37.94                     ^                                                                                                                                                                    
 0:37.94 /media/external/dev/gecko-dev/obj-b2g-desktop/dist/include/mozilla/dom/ToJSValue.h:181:21: error: use of undeclared identifier 'NonRefcountedDOMObject'                                  
 0:37.94     std::is_base_of<NonRefcountedDOMObject, T>::value, bool>                                                                                                                             
 0:37.94                     ^                                                                                                                                                                    
 0:37.94 /media/external/dev/gecko-dev/obj-b2g-desktop/dist/include/mozilla/dom/ToJSValue.h:253:10: error: use of undeclared identifier 'XPCOMObjectToJsval'                                      
 0:37.94   return XPCOMObjectToJsval(aCx, scope, helper, &iid, true, aValue);                                                                                                                     
 0:37.94          ^                                                                                                                                                                               
 0:37.94 /media/external/dev/gecko-dev/obj-b2g-desktop/dist/include/mozilla/dom/ToJSValue.h:290:10: error: use of undeclared identifier 'MaybeWrapValue'                                          
 0:37.94   return MaybeWrapValue(aCx, aValue);                                                                                                                                                    
 0:37.94          ^                                                                                                                                                                               
 0:37.95 /media/external/dev/gecko-dev/obj-b2g-desktop/dist/include/mozilla/dom/ToJSValue.h:296:10: error: use of undeclared identifier 'MaybeWrapValue'                                          
 0:37.95   return MaybeWrapValue(aCx, aValue);                                                                                                                                                    
 0:37.95          ^                                                                                                                                                                               
 0:37.95 /media/external/dev/gecko-dev/obj-b2g-desktop/dist/include/mozilla/dom/ToJSValue.h:305:10: error: use of undeclared identifier 'MaybeWrapValue'                                          
 0:37.95   return MaybeWrapValue(aCx, aValue);                                                                                                                                                    
 0:37.95          ^                                                                                                                                                                               
 0:37.95 /media/external/dev/gecko-dev/obj-b2g-desktop/dist/include/mozilla/dom/ToJSValue.h:313:10: error: use of undeclared identifier 'MaybeWrapValue'                                          
 0:37.95   return MaybeWrapValue(aCx, aValue);                                                                                                                                                    
 0:37.95          ^                                                                                                                                                                               
 0:37.95 /media/external/dev/gecko-dev/obj-b2g-desktop/dist/include/mozilla/dom/ToJSValue.h:322:10: error: use of undeclared identifier 'MaybeWrapObjectOrNullValue'                              
 0:37.95   return MaybeWrapObjectOrNullValue(aCx, aValue);                                                                                                                                        
 0:37.95          ^                                                                                                                                                                               
 0:37.95 9 errors generated.

As fas as I can tell, this is because BindingUtils.h includes Document.h, which itself pulls AnimationFrameProviderBinding.h : at this point we need MaybeWrapValue() from BindingUtils.h, but the include guard prevents it from being included again.

  1. A reason given for doing this work was to enable generating frames from an OffscreenCanvas.
    • The PR for that work was backed out in https://bugzilla.mozilla.org/show_bug.cgi?id=1746110
    • Is this a dependency? i.e. is this feature pointless without it? Does this feature require the same preference? I'm trying to work out if this is worth documenting now if the OffscreenCanvas doesn't work yet.
  2. When this comes along, my understanding of the change is that we will be adding the methods requestAnimationFrame and cancelAnimationFrame to DedicatedWorkerGlobalScope. Is that right?
  3. These methods are already present and documented for Window - see requestAnimationFrame and cancelAnimationFrame. Would there be any particular difference in the dedicated worker version? My guess is that it should be very similar except that you'd probably do your animation on an offscreen canvas, and presumably then need to transfer the canvas to the worker thread and back similar to https://developer.mozilla.org/en-US/docs/Web/API/OffscreenCanvas#asynchronous_display_of_frames_produced_by_an_offscreencanvas . Do you have any working examples/test code that show how to do this already?
Flags: needinfo?(aosmond)
  1. I am splitting the pref out in bug 1749323. It could have value to some content authors without OffscreenCanvas, but that is the use case that drove its implementation. Once said bug lands, and we have either complete OffscreenCanvas (including Canvas 2D) or we have partners that want just WebGL (on the domain list in said bug), it will get turned on in that nightly. Hopefully 98.

  2. Yes, that is correct.

  3. It is driven by the same vsync timer, so it should behave very similarly. There is more machinery on the window version, but I would expect them to functionally happen around the same time, with no guarantee as to which fires first.

An example that uses it (you need to force on OffscreenCanvas today) is at https://chrisprice.io/offscreen-canvas/?100000
https://chrisprice.io/offscreen-canvas/index.js
https://chrisprice.io/offscreen-canvas/worker.js

It doesn't explicitly transfer content between the worker and the main thread, because it relies upon HTMLCanvasElement.transferControlToOffscreen. This creates a binding between the canvas element and the OffscreenCanvas object used on the worker thread, allowing the browser to handle the heavy lifting efficiently for display. I'm sure you could rig up something by doing a post of an ImageBitmap or similar as well.

Flags: needinfo?(aosmond)
You need to log in before you can comment on or make changes to this bug.