Open Bug 1735412 Opened 4 years ago Updated 2 months ago

OscillatorNode#start grabs system audio and prevents automatic input switch on headphones

Categories

(Core :: Web Audio, defect, P2)

Product:
Core
Component:
Web Audio
Platform:
x86_64
Windows 10
Type:
defect

Tracking

()

People

(Reporter: saschanaz, Unassigned)

References

Details

(Keywords: parity-chrome)

Attachments

(2 files)

beep.html
732 bytes, text/html
Details
silence-20s.mp3
156.53 KB, audio/mpeg
Details
Attached file beep.html —
  1. Get headphones connected to multiple devices (devices e.g. Surface Headphones support this)
  2. Open the attachment beep.html
  3. Click beep button and check it shortly beeps and stops.
  4. Play some music on another device.
  5. Refresh the tab

Expected: The headphones should automatically switch to that device.
Actual: No switch happens before refresh, but the switch immediately happens after refresh.

Hi :padenot, this is the issue I described in Slack.

Flags: needinfo?(padenot)
See Also: → 1735427

I'm going to explain why this happens, and why it has to be like that, and then we can try to think on how to make this perceptually better.

This happens because the AudioContext's semantic is to open an output audio stream that can render low-latency interactive audio. It is expected that silence is continuously played, and that as soon as audio needs to be output, it is output. We're talking sub-10ms latencies here, sub-5ms being quite common. Of course in Bluetooth those latencies are a lot higher.

Opening an audio stream to a Bluetooth device is in the 150-250ms range, so eagerly closing the audio stream means that the beginning of the audio could (and would) not be hearable (depending on the device, but this is the case in practice). This means that some notifications would be inaudible. In reality, notification sounds shouldn't be implemented using an AudioContext, but it seems like this is not well understood.

Authors can (and should) release the audio device when it is not needed. For example, one can do:

<!DOCTYPE html>
<meta charset="utf-8">
<title>Beep</title>

<button id="key" onpointerdown="notePressed(event)", onpointerup="noteReleased()">Beep</button>

<script>
  const audioContext = new AudioContext();
  const mainGainNode = audioContext.createGain();
  mainGainNode.connect(audioContext.destination);

  function playTone() {
    const osc = audioContext.createOscillator();
    osc.connect(mainGainNode);

    osc.type = "triangle";
    osc.frequency.value = 262;
    osc.start();

    return osc;
  }

  let osc;
  function notePressed(event) {
    event.target.setPointerCapture(event.pointerId);
    ac.resume().then(() => { osc = playTone(); });
  }

  function noteReleased(event) {
    osc.stop();
    ac.suspend();
  }

</script>

Refreshing a tab effectively immediately suspends the AudioContext so this is why you're seeing what you're seeing. The reason why the audio output device is not grabbed before clicking the button is that osc.start() effectively calls AudioContext.resume() underneath, and this can only work from a touch/click handler.

We compute the "audibility" of the audio signal that's passing through the destination, so we have the information to release, but quite a lot of testing and thought should go into this before shipping it, as the impact is not small. I've been thinking about having an API in our audio IO library that reports the type of transport of the underlying audio stream (essentially wired / bluetooth), and this would help here.

All in all, I think we should do something, but it's going to be hard. I want to do something about shutting down the audio subsystem, starting with the media element (this would fix 1735427), so I'll be thinking and testing things soon. The same happens in all other browsers if I'm not mistaken, there is some room for web compat issues as well.

Flags: needinfo?(padenot)

(In reply to Paul Adenot (:padenot) from comment #2)

The same happens in all other browsers if I'm not mistaken, there is some room for web compat issues as well.

I think I'm seeing a timeout-based release on Chrome on Linux.

I think I'm seeing a timeout-based release on Chrome on Linux.

I see the same on Chrome on Windows. Adding parity-chrome to reflect it.

Keywords: parity-chrome

And thank you for the details!

Before fixing it I think the tab should keep showing the speaker icon on the tab, so that users can see which tab is grabbing their headphones. That will be weird, but I think playing a silent mp3 file via <audio> behaves same. Muting should ideally release the audio but that's another issue (bug 1735427).

(In reply to Kagami :saschanaz from comment #5)

I think playing a silent mp3 file via <audio> behaves same.

It doesn't, this icon is based on audibility, not playback state, see the attached file.

Attached audio silence-20s.mp3 —

Oh you are right, sorry 😬

Severity: -- → S3
Priority: -- → P2
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Creator:
Created:
Updated:
Size: