Closed Bug 1718223 Opened 5 months ago Closed 3 months ago

AudioStream starvation when playback rate >1 while playing encrypted audio

Categories

(Core :: Audio/Video, defect, P3)

defect

Tracking

()

RESOLVED FIXED
93 Branch
Tracking Status
firefox93 --- fixed

People

(Reporter: iidebyo, Assigned: iidebyo)

Details

Attachments

(2 files, 3 obsolete files)

Streaming (encrypted?) audio at playback rates >1 causes an audio popping/stuttering effect. No audio appears to be dropped -- only interlaced with this effect.

I've been experiencing this using audible's web player. I can reproduce this on nightly (Windows 10) and FF 89.0.1 (64-bit) on Windows&MacOS.

  1. Navigate to https://www.audible.com/
  2. Login to play any book or podcast from your Library
    2b. "Purchase" a free book from their collection at https://www.audible.com/ep/FreeListens
  3. Navigate to https://www.audible.com/library/titles
    3b. Reduce volume to save your ears
  4. Play anything in your library by clicking "Listen now"
  5. Change the player (a pop-up) speed by clicking the "1.0x" at the bottom-center.
    5b. >=2 is recommended to notice the effect quickly.

There should be no audio popping/stuttering when playing audio at rates >1. Google Chrome did not reproduce this effect for me.

I ran FF with MOZ_LOG=AudioStream:5 to see several messages like:

[Child 17312: AudioIPC0]: W/AudioStream 23460bb8ac0 lost 441 frames [1]

I think I found a tangential bug which doesn't solve the issue but reduces the severity a little, seemingly [2]. toPopFrames is never decremented so more audio data can be "consumed" [3] than written to the buffer for cubeb. I only noticed in this streaming/encrypted case and not for local files.

I tried reducing the decryption throttling but it had little effect. I think it's due to MediaFormatReader/MediaFormatDemuxer only working on a single sample at a time and the sample being as short as audio is buffered with respect to the increased playback rate. However, I don't really know what I'm doing -- so I made this bug report.

[1] https://searchfox.org/mozilla-central/source/dom/media/AudioStream.cpp#643
[2] https://searchfox.org/mozilla-central/source/dom/media/AudioStream.cpp#549
[3] https://searchfox.org/mozilla-central/source/dom/media/mediasink/AudioSink.cpp#288

Assignee: nobody → paul.m.vitale
Status: NEW → ASSIGNED

Would you mind to attach the log with MOZ_LOG=MediaDecoder:5,MediaFormatReader:5,AudioStream:5 to let us check what's happened there? Also, not sure I understand your point on toPopFrames, because that will be dynamically adjusted by the playback rate. So if you change the playback rate to 1x, it's reasonable to consume more audio samples.
Thank you.

Severity: -- → S3
Flags: needinfo?(paul.m.vitale)
Priority: -- → P3

Yeah, the toPopFrames bit doesn't apply once I delved a bit past the surface.

MOZ_LOG="MediaDecoder:5,MediaFormatReader:5,AudioStream:5"

Flags: needinfo?(paul.m.vitale)
Summary: AudioStream starvation when playback rate >1 → AudioStream starvation when playback rate >1 while playing encrypted audio

Adds a pref media.eme.max-throughput-ms to allow users to modify EME's maximum decryption rate. The default value is 200ms which is the same as the old hard-coded value.

Attachment #9229052 - Attachment is obsolete: true

Adds a pref media.eme.max-throughput-ms to allow users to modify EME's maximum decryption rate. The default value is 200ms which is the same as the old hard-coded value.

Attachment #9236675 - Attachment is obsolete: true

Adds a pref media.eme.max-throughput-ms to allow users to modify EME's maximum decryption rate. The default value is 200ms which is the same as the old hard-coded value.

Attachment #9236345 - Attachment is obsolete: true
Pushed by bvandyk@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/75eac140369b
Allow pref to control max throughput of EME decryption r=bryce
Pushed by bvandyk@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/6eaf205eb754
Allow pref to control max throughput of EME decryption r=bryce
Flags: needinfo?(paul.m.vitale)
Status: ASSIGNED → RESOLVED
Closed: 3 months ago
Resolution: --- → FIXED
Target Milestone: --- → 93 Branch
You need to log in before you can comment on or make changes to this bug.