Video and Sound (Youtube) asynchronous if AirPlay is enabled

RESOLVED FIXED in Firefox 61

Status

()

defect
P2
normal
RESOLVED FIXED
7 years ago
11 months ago

People

(Reporter: Nomis101, Assigned: kinetik)

Tracking

(Blocks 1 bug)

unspecified
mozilla61
x86
macOS
Points:
---
Dependency tree / graph

Firefox Tracking Flags

(platform-rel +, firefox61 fixed)

Details

(Whiteboard: [platform-rel-Youtube])

Reporter

Description

7 years ago
If you enable AirPlay on Mountain Lion, than the sound has a delay of a few seconds (I think because of error correction). Apple has solved this for Quicktime and Quicklook by also delaying the video the same time. This also works in Safari 6 if you watch Youtube videos. So the sound and the video is synchronous again if AirPlay is enabled (works for Youtube, but not for other video sites). If you watch Youtube videos in Firefox, than this doesn't work and Video and Sound is asynchronous.

10.8 12A269
Assignee

Comment 1

7 years ago
Thanks for the report.  We can fix this for HTML 5 video if that's where the issue lies, but Flash's audio is out of our control and will require an update from Adobe if you're seeing this problem with Flash video playback.

Anyway, this isn't too surprising for our side; the way we calculate the audio playback position depends on the audio callback running frequently with low latency (see http://mxr.mozilla.org/mozilla-central/source/media/libsydneyaudio/src/sydney_audio_mac.c#431).  We're using that technique to work around issues we had with CoreAudio's clock reporting bad values on suspend/resume and device changes.

Fixing this correctly probably means moving back to using CoreAudio's clock, which'll mean we need to find another way to address the other issues.
Reporter

Comment 2

7 years ago
Exactly, sorry I haven't mentioned this. In Safari I use the YouTube5 extension, so the videos in Youtube are all HTML5. If I switch to Flash, than it's also asynchronous in Safari. In Firefox both, HTML5 and Flash, is asynchronous.
Component: Audio/Video → Audio/Video: Playback
Fixing this would be appreciated. It's annoying to have to use another browser for youtube links :)
(In reply to contact from comment #3)
> Fixing this would be appreciated. It's annoying to have to use another
> browser for youtube links :)

Is this issue still reproducible? Is the audio early or late?
(In reply to Anthony Jones (:kentuckyfriedtakahe, :k17e) from comment #4)
> Is this issue still reproducible? Is the audio early or late?

If reproducible (I can't check at the moment, not having an AirPlay device nearby), the sound was late. Basically it seemed like Firefox did not do what The Other Browser does here - it holds the video playback for a few seconds to sync it with the output audio buffer of the audio device. In The Other Browser it works with video only (not interactive content), which would support the observation that it's about delaying the video rendering to sync with audio. 
In Firefox, last time I checked, simply audio and video start their playback together, but when streaming through AirPlay, audio gets a delay.
Reporter

Comment 6

3 years ago
(In reply to contact from comment #5)
> In Firefox, last time I checked, simply audio and video start their playback
> together, but when streaming through AirPlay, audio gets a delay.

I checked just know with latest Nightly. And this is still the case. Over Airplay the sound is coming a few seconds too late.
Assignee

Comment 7

3 years ago
The stream clock in cubeb_audiounit.c (audiounit_stream_get_position) doesn't adjust for hardware or playout latency.  We have the code (added much later) in audiostream_stream_get_latency to query the hardware latency and the playout latency looks trivial to calculate, so it would be fairly simple to fix this.
Assignee: nobody → kinetik
platform-rel: --- → ?
Whiteboard: [platform-rel-Youtube]
platform-rel: ? → +
Blake - do you think this would be a good fit for Chun Min after he's finished with 5.1 stuff?
Flags: needinfo?(bwu)
Assignee

Comment 9

2 years ago
(In reply to Matthew Gregan [:kinetik] from comment #7)
> The stream clock in cubeb_audiounit.c (audiounit_stream_get_position)
> doesn't adjust for hardware or playout latency.  We have the code (added
> much later) in audiostream_stream_get_latency to query the hardware latency
> and the playout latency looks trivial to calculate, so it would be fairly
> simple to fix this.

It turns out this doesn't work as assumed, so there wasn't a quick fix for this.  The playout latency reported by audiostream_stream_get_latency doesn't seem to include delay induced by AirPlay (although it should be correctly queried via the device_latency_frames request, possibly a bug there).  Looking around at the time, it looked like the solution was to register notifications to detect when the audio device switched to and from an AirPlay device (which we now have most/all of the code for) and then add a fixed 2000ms offset to the clock to repair A/V sync.

Some history of VLC trying to fix this: https://trac.videolan.org/vlc/ticket/7127
And eventually reverting the changes due to problems: https://trac.videolan.org/vlc/ticket/14042
I can see this problem on my macOS Sierra but audio comes earlier than video. FWIW, Chrome doesn't have this problem. 

It is an very interesting bug. Yeah, it is a good bug for Chunmin to check and know more about cubeb. He should be fascinated to this. 

Matthew,
Do you mind letting Chunmin to check this bug?
Flags: needinfo?(bwu)
Summary: [10.8] Video and Sound (Youtube) asynchronous if AirPlay is enabled → Video and Sound (Youtube) asynchronous if AirPlay is enabled
Assignee

Updated

2 years ago
Assignee: kinetik → cchang
I tested with AppleTV 3 in the meeting room but it works fine. The audio and video are synchronous. Blake will help us to test with his Apple TV 2 and see what could we find.
I tried the latest nightly with my Apple TV 3 again. And now I cannot see AV unsync problem. It looks like something magically changed on the nightly and it fixed this problem somehow. 

ChunMin, 
BTW, I was wrong about the version[1] of Apple TV in our office. It should be Apple TV "4".  

[1]https://support.apple.com/en-us/HT200008
Status: NEW → RESOLVED
Last Resolved: 2 years ago
Resolution: --- → FIXED

Comment 13

2 years ago
I still have a 2 second latency between audio and video when connected to a speaker via AirPlay.
I'm running on OSX 10.12.3, tested on both FF 56.0a1 and 54.0.1
Can you please re-open this bug?
Assignee

Comment 14

2 years ago
(In reply to samer from comment #13)
> I still have a 2 second latency between audio and video when connected to a
> speaker via AirPlay.
> I'm running on OSX 10.12.3, tested on both FF 56.0a1 and 54.0.1
> Can you please re-open this bug?

Thanks for the report.  Which device and version are you using at the output side, an Apple TV (which version?) or something else?
Status: RESOLVED → REOPENED
Resolution: FIXED → ---

Comment 15

2 years ago
Sorry, I should have mentioned that I'm using a Raspberry Pi running Shairport-Sync[0]. I have previously filed a bug report[1] to the maintainer, but he claims that the issue is particular to Firefox:

> There is always some delay when you send audio to an Airplay device such as Shairport Sync or the Apple Airport Express. The delay is set by the audio sender and is in the range of about 400 milliseconds to over two seconds. Most senders set the delay at two seconds, including the Mac, when you're routing system audio to an AirPlay device. To compensate for the delay, Safari and Chrome and the Quicktime Player delay video part of a movie by two seconds so that the video and audio are in sync. Firefox doesn't do it. Also, Safari and Chrome don't do it for all YouTube content.\

I can confirm that I am only facing the problem with Firefox, not Safari or Chrome.

[0] https://github.com/mikebrady/shairport-sync/
[1] https://github.com/mikebrady/shairport-sync/issues/161)
I have the same issue. Playing a youtube video with Firefox results in audio and video being out of sync. Chrome and Safari play the same youtube video just fine with audio matching the video.

I'm on a MacBook Pro 5,3 with OS X 10.11.6 and Firefox 54.0.1 (64-bit). I use an Apple Airport Express (1st Generation, v7.6.8) as my Wi-Fi router and to stream audio via Airplay.

Steps to reproduce:

1. Connect to Airport Express
2. Set the Audio Output Device to the Airplay device
3. Open a youtube video

Expected result: Video plays with audio and video in sync.
Actual result: Video plays, but audio lags approximately 2 seconds behind.
Mass change P1->P2 to align with new Mozilla triage process
Priority: P1 → P2
Hi Reporters, 
Can you help test with Firefox 57 to see if this is still a problem or not?
Thanks.
(In reply to Blake Wu [:bwu][:blakewu] from comment #18)
> Hi Reporters, 
> Can you help test with Firefox 57 to see if this is still a problem or not?
> Thanks.

Unfortunately I have no access to my Airport Express at the moment, so I cannot test.

Comment 20

2 years ago
The problem persists for me when running Firefox 57 beta.
MacOS 10.12 + Raspberry Pi running Shairport-sync.
(In reply to samer from comment #20)
> The problem persists for me when running Firefox 57 beta.
> MacOS 10.12 + Raspberry Pi running Shairport-sync.
I cannot repro this bug with the Firefox 57.0b11 and the latest nightly 58 on macOS 10.13 (17A405) with Apple TV (3rd generation). 

What is your Apple TV version? 
https://support.apple.com/en-us/HT200008

Comment 22

2 years ago
(In reply to Blake Wu [:bwu][:blakewu] from comment #21)
> (In reply to samer from comment #20)
> > The problem persists for me when running Firefox 57 beta.
> > MacOS 10.12 + Raspberry Pi running Shairport-sync.
> I cannot repro this bug with the Firefox 57.0b11 and the latest nightly 58
> on macOS 10.13 (17A405) with Apple TV (3rd generation). 
> 
> What is your Apple TV version? 
> https://support.apple.com/en-us/HT200008

As mentioned above, I'm emulating an Apple TV using a Raspberry Pi and Shairport-sync (https://github.com/mikebrady/shairport-sync)
Problem still persists with Firefox 56 and Denon AVR X2000 receiver.
If now AV unsync issue only happens on Shairport-sync, not Apple TV, it would be better to file another bug and close this one.
(In reply to Blake Wu [:bwu][:blakewu] from comment #24)
> If now AV unsync issue only happens on Shairport-sync, not Apple TV, it
> would be better to file another bug and close this one.

As I mentioned, not only on Shairport-sync, but also on AirPlay compatible amplifiers.

Comment 26

2 years ago
Can confirm also experiencing audio latency when using airplay.

Firefox: 58.0b7 (64-bit)
MacOS 10.13.1
Airplay Audio Device: Sony Speaker RDP-XA900iP

Latency does not exist when using Safari (Version 11.0.1 (13604.3.5)) or other applications like iTunes.

Comment 27

a year ago
The problem persists with Firefox 59.0.1 on macOS 10.13.3. There extists a fixed 2 second audio delay when airplaying to Apple HomePod. The problem doesn't affect Safari or Chrome.

For your reference this is the chromium bug regarding this problem: https://bugs.chromium.org/p/chromium/issues/detail?id=783282
Assignee

Comment 28

a year ago
(In reply to asesg4 from comment #27)
> For your reference this is the chromium bug regarding this problem:
> https://bugs.chromium.org/p/chromium/issues/detail?id=783282

Thanks, that's useful.  We've got code that queries kAudioDevicePropertyLatency for device enumeration, but we're not using it for latency adjustment.  From a quick test with my ATV4, I get surprisingly low values (~150ms)... I'll experiment a bit more and see if I can get it working correctly.
Assignee: chun.m.chang → kinetik
Component: Audio/Video: Playback → Audio/Video: cubeb
Assignee

Comment 29

a year ago
Nightly test build here: https://index.taskcluster.net/v1/task/gecko.v2.try.revision.d7512296fab96bae8ab91979bca41ae8b892db62.firefox.macosx64-opt/artifacts/public/build/target.dmg

This looks and sounds right for local playback and using AirPlay to an Apple TV 4 (which is reporting ~150ms).  I'd appreciate feedback on how well this works for others, particular anyone with device that has a ~2s delay.
NI to myself so I test with my Bluetooth setup at home that has high latency.
Flags: needinfo?(padenot)

Comment 31

a year ago
The issue is fixed in the test build. Audio and video now syncs on my HomePod. Thanks.

Comment 32

a year ago
Nightly works for me!
Emulating airplay using a Raspberry Pi with Shairport-Sync.

Thank you so much :kinetik
Assignee

Comment 34

a year ago
Thanks for the testing and feedback, asesg4 and samer!  I'll post a link to the "final" test build once it completes.
Assignee

Comment 35

a year ago
Try push is actually: https://treeherder.mozilla.org/#/jobs?repo=try&revision=6732f8777511c0b5bece2dcb9908980f6c4da84c

Previous attempt failed because trychooser generates try instructions that the decision task rejects. :-/

Comment 37

a year ago
Pushed by mgregan@mozilla.com:
https://hg.mozilla.org/integration/mozilla-inbound/rev/5cf87a14afb5
Apply latency correction to audio stream position on macOS. r=padenot

Comment 38

a year ago
bugherder
https://hg.mozilla.org/mozilla-central/rev/5cf87a14afb5
Status: REOPENED → RESOLVED
Last Resolved: 2 years agoa year ago
Resolution: --- → FIXED
Target Milestone: --- → mozilla61

Updated

11 months ago
Flags: needinfo?(padenot)
You need to log in before you can comment on or make changes to this bug.