Open Bug 1625375 Opened 4 years ago Updated 2 years ago

Disable worklet MessagePort event handling when AudioContext is suspended for BF cache

Categories

(Core :: Web Audio, enhancement, P3)

enhancement

Tracking

()

People

(Reporter: karlt, Unassigned)

References

Details

Currently when the window is frozen, the AudioContext suspends all its streams, but MessagePort events continue to be processed. Usually this is not a problem because main thread script is paused and so there are usually no new messages. It is possible, however, for entangled MessagePorts to both be on the worklet thread, in which case messages can continue to be posted.

A solution to bug 1625372 may help here.

What does "BF cache" mean?

If gathering the case correctly MessagePort event handling should not be disabled when AudioContext state is "suspended".

An AudioWorkletProcessor can be constructed after AudioContext.suspend() is executed, then input audio data is accumulated to the amount where process() can output audio, then a message posted to main thread or a Worker which then executes AudioContext.resume() so that the clock currentTime is not progressing when no audio is being output, e.g.,

            const ac = new AudioContext({
              sampleRate,
              numberOfChannels,
              latencyHint,
            });
            await ac.suspend(); // we do not need currentTime to progress until input data is accumulated elsewhere
            console.log(ac.baseLatency);
            ac.onstatechange = handleEvents;
            await ac.audioWorklet.addModule('audioWorklet.js');
            // optionally pass MediaStreamTrack to resolve()
            resolve();
            const aw = new AudioWorkletNode(
              ac,
              'audio-data-worklet-stream',
              workletOptions
            );

In AudioWorkletProcessor

       if (this.i === minSamples) {
          console.log('this.buffers.size:' + this.buffers.size);
          this.port.postMessage({
            start: true,
          });
        }

then in main thread (index.html)

            worker.onmessage = async e => {
              // use suspend(), resume() to synchronize to degree possible
              if (e.data.start) {
                await ac.resume();
              }
              if (e.data.ended) {
                console.log(e.data.currentTime, e.data.currentFrame);
                button.disabled = false;
                button.textContent = 'Start Audio Playback';
                await ac.suspend();
                aw.disconnect();
                aw.port.close();
                await ac.close();
                worker.terminate();
              }
            };

which is useful for attempting to synchronize HTMLMediaElement.currentTime with AudioContext.currentTime when audio output is passed to a MediaStreamTrack and the MediaStream containing the track is set as srcObject at <video> or <audio>, otherwise the two currentTimes can be offset.

Please educate yourself about the bf cache (which stands for Back-Forward cache), what you're saying is of course supported by Gecko, and is unrelated.

This is about the case when the page is not active anymore, but has been navigated away from.

(In reply to Paul Adenot (:padenot) from comment #2)

Please educate yourself about the bf cache (which stands for Back-Forward cache

That is precisely what am doing: Asked what the term means.

This is about the case when the page is not active anymore, but has been navigated away from.

That should not matter. Who says the page is not intended to be active simply because navigated to another tab for a moment? Could still have the requirement for code to continue running irrespective of how many tabs are open or which tab has focus.

(In reply to guest271314 from comment #3)

That should not matter. Who says the page is not intended to be active simply because navigated to another tab for a moment? Could still have the requirement for code to continue running irrespective of how many tabs are open or which tab has focus.

This is not what is being discussed. Navigating away from a page is a technical term: https://html.spec.whatwg.org/multipage/browsing-the-web.html#navigating-across-documents. This is for when a page is in the bfcache, which is a cache that preserves state when a page has been navigated away from. At this stage, it shouldn't execute script, so events should be blocked.

#5 Will look into BF Cache more.

Why should that affect AudioWorklet? A use case could be precisely using AudioWorklet "in the background".

Also filed an issue re getDisplayMedia() freeze https://bugzilla.mozilla.org/show_bug.cgi?id=1549095.

#4

This is not what is being discussed. Navigating away from a page is a technical term: https://html.spec.whatwg.org/multipage/browsing-the-web.html#navigating-across-documents. This is for when a page is in the bfcache, which is a cache that preserves state when a page has been navigated away from. At this stage, it shouldn't execute script, so events should be blocked.

Does this bug relate to the page where AudioWorklet is being used is navigated away from, not simply stopping event handling when the page loses focus because clicked on another tab?

(In reply to guest271314 from comment #7)

Does this bug relate to the page where AudioWorklet is being used is navigated away from, not simply stopping event handling when the page loses focus because clicked on another tab?

Yes, this is the title of the bug.

Depends on: 1642849
No longer depends on: 1625372
Severity: normal → S3
You need to log in before you can comment on or make changes to this bug.