1718223 - AudioStream starvation when playback rate >1 while playing encrypted audio

Status: RESOLVED FIXED

(Core :: Audio/Video, defect, P3)

Description

[Paul Vitale [:iidebyo]]

Streaming (encrypted?) audio at playback rates >1 causes an audio popping/stuttering effect. No audio appears to be dropped -- only interlaced with this effect.

I've been experiencing this using audible's web player. I can reproduce this on nightly (Windows 10) and FF 89.0.1 (64-bit) on Windows&MacOS.

1. Navigate to https://www.audible.com/
2. Login to play any book or podcast from your Library
   2b. "Purchase" a free book from their collection at https://www.audible.com/ep/FreeListens
3. Navigate to https://www.audible.com/library/titles
   3b. Reduce volume to save your ears
4. Play anything in your library by clicking "Listen now"
5. Change the player (a pop-up) speed by clicking the "1.0x" at the bottom-center.
   5b. >=2 is recommended to notice the effect quickly.

There should be no audio popping/stuttering when playing audio at rates >1. Google Chrome did not reproduce this effect for me.

Comment 1

[Paul Vitale [:iidebyo]]

I ran FF with MOZ_LOG=AudioStream:5 to see several messages like:

[Child 17312: AudioIPC0]: W/AudioStream 23460bb8ac0 lost 441 frames [1]

I think I found a tangential bug which doesn't solve the issue but reduces the severity a little, seemingly [2]. toPopFrames is never decremented so more audio data can be "consumed" [3] than written to the buffer for cubeb. I only noticed in this streaming/encrypted case and not for local files.

I tried reducing the decryption throttling but it had little effect. I think it's due to MediaFormatReader/MediaFormatDemuxer only working on a single sample at a time and the sample being as short as audio is buffered with respect to the increased playback rate. However, I don't really know what I'm doing -- so I made this bug report.

[1] https://searchfox.org/mozilla-central/source/dom/media/AudioStream.cpp#643
[2] https://searchfox.org/mozilla-central/source/dom/media/AudioStream.cpp#549
[3] https://searchfox.org/mozilla-central/source/dom/media/mediasink/AudioSink.cpp#288

Comment 2

[Paul Vitale [:iidebyo]]

Attachment: Bug 1718223 - Ensure enough audio data is queued for playback rates up to 4.0 r?jolin

Comment 3

[Alastor Wu [:alwu]]

Would you mind to attach the log with MOZ_LOG=MediaDecoder:5,MediaFormatReader:5,AudioStream:5 to let us check what's happened there? Also, not sure I understand your point on toPopFrames, because that will be dynamically adjusted by the playback rate. So if you change the playback rate to 1x, it's reasonable to consume more audio samples.
Thank you.

Comment 4

[Paul Vitale [:iidebyo]]

Yeah, the toPopFrames bit doesn't apply once I delved a bit past the surface.

Comment 5

[Paul Vitale [:iidebyo]]

Attachment: bug1718223.child-2.moz_log

MOZ_LOG="MediaDecoder:5,MediaFormatReader:5,AudioStream:5"

Comment 6

[Paul Vitale [:iidebyo]]

Attachment: Bug 1718223 - Allow pref to control max throughput of EME decryption

Adds a pref media.eme.max-throughput-ms to allow users to modify EME's maximum decryption rate. The default value is 200ms which is the same as the old hard-coded value.

Comment 7

[Paul Vitale [:iidebyo]]

Attachment: Bug 1718223 - Allow pref to control max throughput of EME decryption r?bryce

Adds a pref media.eme.max-throughput-ms to allow users to modify EME's maximum decryption rate. The default value is 200ms which is the same as the old hard-coded value.

Comment 8

[Paul Vitale [:iidebyo]]

Attachment: Bug 1718223 - Allow pref to control max throughput of EME decryption r?bryce

Adds a pref media.eme.max-throughput-ms to allow users to modify EME's maximum decryption rate. The default value is 200ms which is the same as the old hard-coded value.

Comment 9

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/75eac140369b
Allow pref to control max throughput of EME decryption r=bryce

Comment 10

[Atila Butkovits]

Backed out for causing build bustages on EMEDecoderModule.cpp.

Backout link: https://hg.mozilla.org/integration/autoland/rev/7459e2e98b4b3bc9dd3582330969c726af510ac4

Push with failures: https://treeherder.mozilla.org/jobs?repo=autoland&selectedTaskRun=DZriG2luRDWYP2SFwqpTzA.0&resultStatus=testfailed%2Cbusted%2Cexception%2Cretry%2Cusercancel&revision=75eac140369bc246f85955f1f492e378a9916b37

Failure log: https://treeherder.mozilla.org/logviewer?job_id=348820676&repo=autoland&lineNumber=10083

Comment 11

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/6eaf205eb754
Allow pref to control max throughput of EME decryption r=bryce

Comment 12

[Marian-Vasile Laza]

https://hg.mozilla.org/mozilla-central/rev/6eaf205eb754