1596655 - Wake up MediaTrackGraph thread on MessagePort messages

Status: RESOLVED FIXED

(Core :: Web Audio, enhancement, P2)

Description

[Karl Tomlinson (:karlt)]

This is necessary for OfflineAudioContexts that have completed and realtime AudioContexts that are suspended.

It might involve nsIThreadInternal.observer with nsIThreadObserver.onDispatchedEvent() calling MediaTrackGraphImpl::EnsureNextIteration().

Comment 1

[Karl Tomlinson (:karlt)]

wip: https://hg.mozilla.org/try/pushloghtml?changeset=0788aaffdf755c42eece91ef42cb8e2f66d76650

Comment 2

[Karl Tomlinson (:karlt)]

Since https://hg.mozilla.org/integration/autoland/rev/6b5d3b9abd98, a driver no longer has a GraphImpl() method but has access to a GraphInterface. That GraphInterface is not necessarily a MediaTrackGraphImpl in general, but is a MediaTrackGraphImpl for OfflineClockDriver.

Two options to register the graph as a thread observer from the offline rendering thread are:

1. Make GraphInterface an nsISupports for safe dynamic cast to nsIThreadObserver.

2. Use a GraphRunner thread from OfflineClockDriver.

At this stage I'm going with option 1 to avoid twice as many offline threads as necessary. Option 2 has the advantage of being more consistent with the realtime model. There's not much code required to maintain the differences, but that benefit may become more compelling when "media.audiograph.single_thread.enabled" is removed.

Comment 3

[Karl Tomlinson (:karlt)]

These patches keep the DestinationNode track after offline completion, but seem to interfere with MediaRecorder's detection of completion:
https://treeherder.mozilla.org/#/jobs?repo=try&revision=3cac8921c7961c9006ed89711c6395047715d27f

Comment 4

[Karl Tomlinson (:karlt)]

Attachment: Bug 1596655 move OnCompleteTask to anonymous namespace r?padenot

This will allow the class to be used from AudioDestinationNode.

Depends on D65067

Comment 5

[Karl Tomlinson (:karlt)]

Attachment: Bug 1596655 move offline complete event logic from engine to node r?padenot

This will allow the node to drop its self reference without
AudioContext::Shutdown().

Depends on D65068

Comment 6

[Karl Tomlinson (:karlt)]

Attachment: Bug 1596655 Keep DestinationNode track after offline completion for worklets r?padenot

This keeps the worklet/graph thread alive for control and MessagePort messages
until there are no references to main thread objects.

In test_getUserMedia_audioCapture, the track of the OscillatorNode being
recorded does not end, and, even if it did, MediaRecorder would not listen for
ended notification on the node's track but its pipe track. Recording via
DestinationNode works around this because the DestinationNode track ends and
MediaRecorder listens to this track directly because it has
EXTERNAL_OUTPUT. Previously MediaRecorder would stop when the MediaTrackGraph
was shutdown, but this is no longer happening.

Depends on D65069

Comment 7

[Karl Tomlinson (:karlt)]

Attachment: Bug 1596655 Make GraphInterface an nsISupports for safe dynamic cast r?padenot

This will allow OfflineClockDriver to obtain an nsIThreadObserver from the
GraphInterface when it is a MediaGraphTrackImpl.

Depends on D65070

Comment 8

[Karl Tomlinson (:karlt)]

Attachment: Bug 1596655 Wake up MediaTrackGraph thread on MessagePort messages r?padenot

Depends on D65071

Comment 9

[Karl Tomlinson (:karlt)]

Attachment: Bug 1596655 test MessagePort while AudioContext is not running r?padenot

Depends on D65072

Comment 10

[Karl Tomlinson (:karlt)]

https://treeherder.mozilla.org/#/jobs?repo=try&revision=1d3125f70e3e7de31e3fa6112438673e5068322f

Comment 11

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/282dc3f24c33
move OnCompleteTask to anonymous namespace r=padenot
https://hg.mozilla.org/integration/autoland/rev/28b225e74955
move offline complete event logic from engine to node r=padenot
https://hg.mozilla.org/integration/autoland/rev/93abf903275b
Keep DestinationNode track after offline completion for worklets r=padenot
https://hg.mozilla.org/integration/autoland/rev/e147eeeb5126
Make GraphInterface an nsISupports for safe dynamic cast r=padenot
https://hg.mozilla.org/integration/autoland/rev/ebbf0af66075
Wake up MediaTrackGraph thread on MessagePort messages r=padenot
https://hg.mozilla.org/integration/autoland/rev/50e126834071
test MessagePort while AudioContext is not running r=padenot

Comment 12

[Web Platform Test Sync Bot (Matrix: #interop:mozilla.org)]

Created web-platform-tests PR https://github.com/web-platform-tests/wpt/pull/22078 for changes under testing/web-platform/tests

Comment 13

[Web Platform Test Sync Bot (Matrix: #interop:mozilla.org)]

Upstream web-platform-tests status checks passed, PR will merge once commit reaches central.

Comment 14

[Web Platform Test Sync Bot (Matrix: #interop:mozilla.org)]

Upstream PR merged by moz-wptsync-bot

Comment 15

[Oana Pop-Rus]

https://hg.mozilla.org/mozilla-central/rev/282dc3f24c33
https://hg.mozilla.org/mozilla-central/rev/28b225e74955
https://hg.mozilla.org/mozilla-central/rev/93abf903275b
https://hg.mozilla.org/mozilla-central/rev/e147eeeb5126
https://hg.mozilla.org/mozilla-central/rev/ebbf0af66075
https://hg.mozilla.org/mozilla-central/rev/50e126834071