1833981 - Audio skipping/jittering on live streams on Twitch

Status: VERIFIED FIXED

(Core :: Audio/Video: Playback, defect)

Description

[theraptor01]

User Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:109.0) Gecko/20100101 Firefox/115.0

Steps to reproduce:

Watching live streams on Twitch.com.

Actual results:

audio jittering a lot like it's skipping.

Expected results:

Audio playing just fine like normal.

This problem doesn't seem to happen on Firefox stable 113.0.1, but it does happen on Firefox Nightly 115.0a1.

Also, ads on Twitch doesn't seem to be affected. Only the live streams.

Comment 1

[BugBot [:suhaib / :marco/ :calixte]]

The Bugbug bot thinks this bug should belong to the 'Core::Audio/Video: Playback' component, and is moving the bug to that component. Please correct in case you think the bot is wrong.

Comment 2

[theraptor01]

i tried Nightly with the "Restart with add-ons disabled" option in "About:profiles" section, and it still happen, even in 160p and even in 480p.

Comment 3

[Paul Adenot (:padenot)]

Reporter, thanks for filing an issue. Would you mind attempting to reproduce this using an up-to-date Nightly build (we've landed and fixed quite a few issues between now and five last week when you filed this) and with Media logging enabled, as shown in this video; https://paul.cx/public/about-logging-presentation.webm ? Please Share the profile with the button on the top right and paste the link here, or send the downloaded profile privately to me at <[e-mail]>, so one of use in the media team can look into it.

Additionally, attaching the raw data from about:support (that has information about your setup), and telling us an approximate time or release this started to happen would help a lot.

Thanks!

Comment 4

[theraptor01]

@Paul Adenot: when i wrote this reply 7h ago, it still happened to me.
Also, this problem doesn't happen on regular stable Firefox.
I will email the Profile log, and raw data, to you.

Comment 5

[Paul Adenot (:padenot)]

The timestamps out of the decoder are a bit off on Windows it seems like, I can repro, but rarely, sometimes it's immediate on some streams, sometimes it takes a long time, sometimes it never reproduces.

Comment 6

[Paul Adenot (:padenot)]

It would be cool to be able to have the original timestamps like all other decoders. Alastor, do you know if there's a good way to do it in the current code? Otherwise, I suspect something weird with TimeUnit::FromHns, maybe we should be rounding instead of flooring here: https://searchfox.org/mozilla-central/source/dom/media/TimeUnits.cpp#387.

Comment 7

[Paul Adenot (:padenot)]

(In reply to Paul Adenot (:padenot) from comment #5)

The timestamps out of the decoder are a bit off on Windows it seems like, I can repro, but rarely, sometimes it's immediate on some streams, sometimes it takes a long time, sometimes it never reproduces.

to be specific, the profile (sent privately) shows that the AudioSink is inserting silence because it thinks there are discontinuities in the AAC stream, because a TimeUnit goes wrong somewhere. It works well on all platforms so I'm pretty sure this is in the WMF audio decoder code.

Comment 8

[theraptor01]

Yes, it's not consistant, sometimes it's immediate, sometimes it takes a very long time, sometimes it's just fine, until you change stream. It's a very strange bug.

Comment 9

[Alastor Wu [:alwu]]

This might be related with this issue. That also happens on Twitch and the stream is AAC. As you said you are able to reproduce it locally, could you turn on this log to see if the decoder outputs wrong samples?

It would be cool to be able to have the original timestamps like all other decoders.

Is mOriginalTime not enough? If we want to do similar things on all decoders, the only task this proxy does is to dispatch tasks on another thread, maybe we can consider to extend it?

Comment 10

[amelia]

I am seeing very similar behavior on OS X as well. Difference is that audio is fine, but the video is stuttering.

Saw it about a week ago, stopped for awhile, and them came back last night. Only seems to happen on Twitch, every other video site is fine.

---

Version: 115.0a1
Build ID: 20230526094428
User Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:109.0) Gecko/20100101 Firefox/115.0
OS: Darwin 22.5.0 Darwin Kernel Version 22.5.0: Mon Apr 24 20:52:43 PDT 2023; root:xnu-8796.121.2~5/RELEASE_ARM64_T8112

https://profiler.firefox.com/public/rvq4jd449p8n2s9gzw3a5j4c6vxrxmpae4ssfj0

Comment 12

[Alice0775 White]

1st Regression window:
https://hg.mozilla.org/integration/autoland/pushloghtml?fromchange=fa170b66896e1972488dc85979c3df6bd3a07ae6&tochange=f1ecfbba7ed20ecd0284b301ced7f055f4eae575

And Baked out:
https://hg.mozilla.org/integration/autoland/pushloghtml?fromchange=dcf5abb28a147961d06a936703e82ae5d1e4219b&tochange=12072128e408db78a4dad4b82e1d57636b9637e5

2nd Regression window:
https://hg.mozilla.org/integration/autoland/pushloghtml?fromchange=a708c20c4e36970e5a9bb9b244a6865ad6bdcc92&tochange=928afb6ad2a5dc2f68d9d0b29181162aaac56866

Regressed by: Tentatively, Mark Bug 1703812 as regressor.

Comment 13

[Paul Adenot (:padenot)]

Thanks all for confirming, and providing information (here and privately by email), thanks to the profiles and logs I was able to characterize the issue and I have a patch that fixes this, I'm writing some tests so that this doesn't happen again, and I'll be landing this soon.

Comment 14

[BugBot [:suhaib / :marco/ :calixte]]

The bug is marked as tracked for firefox115 (nightly). We have limited time to fix this, the soft freeze is in 3 days. However, the bug still isn't assigned.

:jimm, could you please find an assignee for this tracked bug? Given that it is a regression and we know the cause, we could also simply backout the regressor. If you disagree with the tracking decision, please talk with the release managers.

For more information, please visit BugBot documentation.

Comment 15

[Paul Adenot (:padenot)]

Clearing Jim's NI, I got this.

Comment 16

[Paul Adenot (:padenot)]

Attachment: Bug 1833981 - When constructing a TimeUnit from a number in HNS, instead of flooring to avoid losing precision. r?azebrowski

Comment 17

[Paul Adenot (:padenot)]

(In reply to amelia from comment #10)

I am seeing very similar behavior on OS X as well. Difference is that audio is fine, but the video is stuttering.

Saw it about a week ago, stopped for awhile, and them came back last night. Only seems to happen on Twitch, every other video site is fine.

I've opened bug 1835663 to track this bug, the cause is different. Thanks for reporting in any case!

Comment 18

[awsderty]

Turning on the low-latency sound of TWITCH will cause problems, and turning off the low-latency sound will improve.
The official default is to watch the live broadcast to turn on the low-latency mode

Comment 19

[awsderty]

(In reply to awsderty from comment #18)

Turning on the low-latency sound of TWITCH will cause problems, and turning off the low-latency sound will improve.
The official default is to watch the live broadcast to turn on the low-latency mode

Comment 20

[awsderty]

(In reply to awsderty from comment #19)

(In reply to awsderty from comment #18)

Turning on the low-latency sound of TWITCH will cause problems, and turning off the low-latency sound will improve.
The official default is to watch the live broadcast to turn on the low-latency mode

Comment 21

[Paul Adenot (:padenot)]

Thanks for the details about low-latency, this helps, I now understand why it happens, and I'll be getting a fix in today or tomorrow.

Comment 22

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/87967198f72a
When converting a TimeUnit to another base approximately, round instead of flooring. r=media-playback-reviewers,azebrowski

Comment 23

[Natalia Csoregi [:nataliaCs]]

Backed out for causing failures on test_DrainOnMissingData_mp4.html

• backout: https://hg.mozilla.org/integration/autoland/rev/3aba6297714ae26655f51ce64b2b1f582feee95e
• push: https://treeherder.mozilla.org/jobs?repo=autoland&group_state=expanded&selectedTaskRun=YxQd2BkLRVmcmJLfspxFmQ.0&revision=87967198f72a5a7192d0307cf1d51bbeb54c9e3d
• failure log: https://treeherder.mozilla.org/logviewer?job_id=417633742&repo=autoland&lineNumber=2316

[task 2023-05-31T11:14:50.827Z] 11:14:50     INFO -  TEST-PASS | dom/media/mediasource/test/test_DrainOnMissingData_mp4.html | fetchWithXHR load uri='bipbop/bipbop_video1.m4s' status=200
[task 2023-05-31T11:14:50.827Z] 11:14:50     INFO -  Loading buffer: [0, 23860)
[task 2023-05-31T11:14:50.827Z] 11:14:50     INFO -  SourceBuffer buffered ranges grew from TimeRanges: [0, 0.368344) to TimeRanges: [0, 0.801667)
[task 2023-05-31T11:14:50.827Z] 11:14:50     INFO -  Buffered messages finished
[task 2023-05-31T11:14:50.828Z] 11:14:50  WARNING -  TEST-UNEXPECTED-FAIL | dom/media/mediasource/test/test_DrainOnMissingData_mp4.html | Video has correct duration: 0.801667 - got 0.801667, expected 0.801666
[task 2023-05-31T11:14:50.828Z] 11:14:50     INFO -      SimpleTest.is@SimpleTest/SimpleTest.js:507:14
[task 2023-05-31T11:14:50.828Z] 11:14:50     INFO -      @dom/media/mediasource/test/test_DrainOnMissingData_mp4.html:41:5
[task 2023-05-31T11:14:50.828Z] 11:14:50     INFO -      async*runWithMSE@dom/media/mediasource/test/mediasource.js:31:11
[task 2023-05-31T11:14:50.828Z] 11:14:50     INFO -      async*@dom/media/mediasource/test/test_DrainOnMissingData_mp4.html:14:11
[task 2023-05-31T11:14:50.829Z] 11:14:50     INFO -  TEST-PASS | dom/media/mediasource/test/test_DrainOnMissingData_mp4.html | Video has correct currentTime.

Comment 24

[Natalia Csoregi [:nataliaCs]]

Also failing:

• TEST-UNEXPECTED-FAIL | dom/media/test/test_bug686942.html | Test timed out!
• TEST-UNEXPECTED-FAIL | dom/media/test/test_eme_pssh_in_moof.html | Test timed out!
• TEST-UNEXPECTED-FAIL | dom/media/test/test_seek_out_of_range.html | Test timed out!

Comment 25

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/e59aefc4904e
When constructing a TimeUnit from a number in HNS, instead of flooring to avoid losing precision. r=media-playback-reviewers,azebrowski

Comment 27

[Sandor Molnar[:smolnar]]

https://hg.mozilla.org/mozilla-central/rev/e59aefc4904e

Comment 28

[theraptor01]

it's still doing it...

Comment 29

[Alastor Wu [:alwu]]

The change won't reflect on Nightly immediately, you might need to wait for tomorrow's Nightly. If this still happens, please help us capture profiled result again, thanks.

Comment 30

[theraptor01]

(In reply to Alastor Wu [:alwu] from comment #29)

The change won't reflect on Nightly immediately, you might need to wait for tomorrow's Nightly. If this still happens, please help us capture profiled result again, thanks.

alright, i thought it was pushed to the new Nightly immediately.

Comment 31

[Ina Popescu, Desktop QA]

I've reproduced this issue using Nightly 115.0a1 (2023-05-18) following the STR from Comment 0 on Windows 10 x64.
Verified as fixed on the latest Firefox 115.0 version on Windows 10 x64, Ubuntu 22.04 and macOS 13 where the issue no longer persists.