1671323 - Substantial audio latency when resuming video playback

Status: NEW

(Core :: Audio/Video: cubeb, defect, P3)

Description

[adam1.byrd]

User Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:81.0) Gecko/20100101 Firefox/81.0

Steps to reproduce:

When resuming a video from a paused state I'm seeing on the order of 500 ms before hearing audio. The video plays during this duration and the audio is simply dropped. It also occurs when playing a video initially. The first half second of audio is never played. It only occurs with some bluetooth devices.

I've confirmed this issue on two Windows machines.
The issue does not occur in Chrome and does occur in Edge.
I've reproduced the issue on both Youtube and Vimeo.
The issue occurs with Powerbeats Pro Wireless, but not with Boltune BT-BH020, SteelSeries H Wireless / Sibera 800, or with analog devices.

My suspicion is that starting an audio stream with a bluetooth device takes a very long time on Windows with some devices. Chrome seems to work around this by keeping the stream active for a longer period of time when pausing a video. If you leave it paused long enough though it will eventually show the same delay. However, Chrome never appears to drop audio when initially playing a video.

If I have any other process open that has an active audio steam the delays no longer occur in Firefox. If I pause and unpause a video very quickly (within less than a second) there is no delay.

Firefox 81.0.2
Windows 2004

Expected results:

1. Pausing and unpausing a video within a reasonable period of time should not drop/delay audio.
2. Initially playing a video should not drop/delay audio.

Perhaps Firefox should keep the audio stream open longer when pausing video. 30 seconds? For the entire video? Except when on battery power?

Ideally Firefox would not have this latency at all, but I don't know whether or not that is possible Window's bluetooth implementation. If nothing else, I'd prefer if the entire video remained paused until the audio stream is ready to avoid dropping content.

I recognize that this is mostly a Windows issue, rather than a Firefox one. However, it makes for a fairly unpleasant experience with video content, particularly with educational content where pausing to absorb information can be frequent. Missing a word or two each time can be at best irritating and at worst counter-productive as you may need to rewind to catch important details.

Comment 1

[Virginia (please send need info request to Brindusa Tot)]

Hi Adam,

Thank you for your report.

I did notice a small latency issue for example on Youtube or Netflix videos, on both Firefox and Chrome browsers when using Windows 10 and a bluetooth headset (Jabra Evolve2 65). As you pointed out, this doesn't seem to be a Firefox only issue, but I'll go ahead and add a product and component to this ticket so the corresponding team can investigate and hopefully, provide any insights on this one.

Regards,
Virginia

Comment 2

[adam1.byrd]

For additional context, the severity seems to depend on the specific Bluetooth device. With my Boltune earbuds the duration of dropped audio is short, less than 50 ms I'd estimate. With others, it's substantially worse. My Powerbeats Pro Wireless are on the order of 500 ms and it's enough to lose multiple spoken words in videos.

I can only repro this in Chrome if I pause a video for 5-10 seconds. In Firefox pausing for ~1 second reproduces the issue.

Comment 3

[John Lin [:jhlin][:jolin]]

Chunmin, this sounds like a BT audio issue. Could you please help take a look? Thanks!

Comment 4

[C.M.Chang[:chunmin]]

I cannot reproduce this with my Bose QC 35II. The delay time on Chrome, Edge, and Firefox Nightly has no much differences. Not sure what's going on here. Adam, could you paste the information under the Media section in about:support page? It may give us a bit more information to take further action. On the other hand, does the problem occur if you use Firefox Nightly?

Comment 5

[C.M.Chang[:chunmin]]

Paul, do you have any idea where we should focus on? I remember we've made some changes for the Bluetooth headset a while ago (but that's for WebRTC stuff)?

Comment 6

[Jean-Yves Avenard [:jya]]

The problem is seen because when we pause the stream, we suspend the audio playback and will close it. We could continue to play silence instead.

This issue can also be seen when using an HDMI amplifier that takes a while to sync.

There are downsides in never stopping the audio stream though.

Comment 7

[Paul Adenot (:padenot)]

There are multiple issues here, reading comment 0 (thanks for the detailed description by the way!):

• We should keep the audio stream opened and rolling for a bit (pushing silence) when it matters. For this we need a cubeb API that exposes the type of transport (bluetooth, hdmi, regular wired, etc.), because for a large proportion of setup this doesn't matter and it's best to stop and release the system audio resources immediately. We also need this cubeb API that would report the transport type for other things, let's just do it.
• We should take into account stream start time when pushing audio, and check whether the audio stack of the system drops the audio (this sounds unlikely because it works in Chrome), to not miss the first few hundreds of milliseconds (as described). We got away with it for a long time because audio systems that take a long time to start up weren't as common as they are today, but what we're doing is wrong. This is another instance of our long-standing problem where our media stack is both push and pull. This wouldn't have happened if it was pull-only.

Comment 8

[BugBot [:suhaib / :marco/ :calixte]]

Redirect a needinfo that is pending on an inactive user to the triage owner.
:padenot, since the bug has recent activity, could you have a look please?

For more information, please visit auto_nag documentation.