1842375 - H264 playback audio desync

Status: REOPENED

(Core :: Audio/Video: Playback, defect, P2)

Description

[riku]

Attachment: demo.mp4

User Agent: Mozilla/5.0 (X11; Linux x86_64; rv:109.0) Gecko/20100101 Firefox/115.0

Steps to reproduce:

Say I have an mp4 like this:
10 seconds of audio.
8 seconds of video, offset +2 seconds.
Both streams start at the same time.

This didn't happen before version 115 and doesn't seem to happen with other codecs like VP9 for example.

Actual results:

Video and audio both start at the same time, regardless of when they actually start in the file.

Expected results:

The video stream should be delayed until it begins.

Comment 1

[riku]

Attachment: demo.webm

Same demo but as a webm, no problem here.

Comment 2

[BugBot [:suhaib / :marco/ :calixte]]

The Bugbug bot thinks this bug should belong to the 'Core::Audio/Video: Playback' component, and is moving the bug to that component. Please correct in case you think the bot is wrong.

Comment 3

[Alice0775 White]

I can reproduce the issue on Nightly 117.0a1 Windows10.

Regression window:
https://hg.mozilla.org/integration/autoland/pushloghtml?fromchange=9fa4a7ae19238256fcd261c727ad2b08c6f1a4fd&tochange=a708c20c4e36970e5a9bb9b244a6865ad6bdcc92

Comment 4

[BugBot [:suhaib / :marco/ :calixte]]

:padenot, since you are the author of the regressor, bug 1830206, could you take a look? Also, could you set the severity field?

For more information, please visit BugBot documentation.

Comment 5

[Paul Adenot (:padenot)]

Range is https://hg.mozilla.org/integration/autoland/pushloghtml?fromchange=a708c20c4e36970e5a9bb9b244a6865ad6bdcc92&tochange=928afb6ad2a5dc2f68d9d0b29181162aaac56866, likely regressor is bug 1817997.

Comment 6

[Alice0775 White]

(In reply to Paul Adenot (:padenot) from comment #5)

Range is https://hg.mozilla.org/integration/autoland/pushloghtml?fromchange=a708c20c4e36970e5a9bb9b244a6865ad6bdcc92&tochange=928afb6ad2a5dc2f68d9d0b29181162aaac56866, likely regressor is bug 1817997.

Confirmed. (Sorry, I made a copy/paste mistake.)

Comment 7

[Paul Adenot (:padenot)]

Fix in https://github.com/mozilla/mp4parse-rust/pull/407, thanks for reporting!

Comment 8

[Paul Adenot (:padenot)]

Attachment: Bug 1842375 - Update mp4parse-rust to 12142fda2ba0870. r?#media-playback-reviewers

Depends on D184213

Comment 9

[Paul Adenot (:padenot)]

Attachment: Bug 1842375 - mach vendor rust. r?#media-playback-reviewers

Depends on D184225

Comment 10

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/1a7c4231566f
Update mp4parse-rust to 12142fda2ba0870. r=media-playback-reviewers,kinetik
https://hg.mozilla.org/integration/autoland/rev/5a58ca98f84d
mach vendor rust. r=kinetik

Comment 11

[Noemi Erli[:noemi_erli]]

https://hg.mozilla.org/mozilla-central/rev/1a7c4231566f
https://hg.mozilla.org/mozilla-central/rev/5a58ca98f84d

Comment 12

[BugBot [:suhaib / :marco/ :calixte]]

The patch landed in nightly and beta is affected.
:padenot, is this bug important enough to require an uplift?

• If yes, please nominate the patch for beta approval.
• If no, please set status-firefox116 to wontfix.

For more information, please visit BugBot documentation.

Comment 13

[Paul Adenot (:padenot)]

We'll let this ride the trains normally because we're releasing in 7 days, but we might want it in ESR, because it's a real regression.

Comment 14

[Paul Adenot (:padenot)]

Comment on attachment 9345055 [details]
Bug 1842375 - Update mp4parse-rust to 12142fda2ba0870. r?#media-playback-reviewers

ESR Uplift Approval Request

• If this is not a sec:{high,crit} bug, please state case for ESR consideration: Content breakage, severe audio/video desynchronization with otherwise valid media files
• User impact if declined: Completely incorrect rendering, it's not possible to work around it, since the video files are already online.
• Fix Landed on Version: 117
• Risk to taking this patch: Low
• Why is the change risky/not risky? (and alternatives if risky): This is reverting the code to its previous behavior.

Comment 15

[Paul Adenot (:padenot)]

This needs more work.

Comment 16

[Paul Adenot (:padenot)]

https://pernos.co/debug/dMCdlYgSTbxji8hRKCAQkA/index.html#f{m[AXvA,pl8_,t[5Q,OhdZ_,f{e[AXvA,pl8_,s{afzAFYAAA,bAZY,uEbRuvQ,oEc0SJQ___/ shows that we don't have a moof parser (this file isn't fragmented), so we end up using a timescale of 15360 from the mdhd box instead of picking the correct timescale of 1000 from the mvhd box.

Matthew, do you have any pointer in the spec on what scale we should use here, so I can write a proper fix? The old code seemed to have the correct behaviour, so I guess I can just look at it.

Comment 17

[Emilio Cobos Álvarez (:emilio)]

[REO hat on] Paul can you set severity on this bug? Probably unlikely to get a fix for 118 right?

Comment 18

[Paul Adenot (:padenot)]

Indeed, I'll get a fix in the next cycle.

Comment 19

[Ryan VanderMeulen [:RyanVM]]

(In reply to Paul Adenot (:padenot) from comment #18)

Indeed, I'll get a fix in the next cycle.

Hi Paul, any updates on this bug?

Comment 20

[Donal Meehan [:dmeehan]]

Synced outside of bugzilla to follow-up on Comment 19.
There is a specific fix that should land in time for Fx123.

Comment 22

[Brenda]

I get the same symptoms, the incorrect audio playback, with Quicktime/mov, which is not using H264.

Comment 23

[Matthew Gregan [:kinetik]]

(In reply to Donal Meehan [:dmeehan] from comment #20)

Synced outside of bugzilla to follow-up on Comment 19.
There is a specific fix that should land in time for Fx123.

I'm working on this, but the fix is wide ranging than I initially suspected so the estimate I gave to Paul was wrong. Ideally this would land in Fx124, but there's not much time left so may slip to Fx125.