1924070 - [macOS/linux] Threejs video does not play

Status: RESOLVED FIXED

(Core :: Audio/Video: Playback, defect)

Description

[Alexandru Trif, Desktop Test Engineering [:atrif]]

Found in

• 133.0a1 (2024-10-10)

Affected versions

• 133.0a1 (2024-10-10)
• 132.0b5
• 131.0.2 / 131.0.3

Tested platforms

• Affected platforms: macOS 12, Ubuntu 24.04
• Unaffected platforms: Windows 10

Steps to reproduce

1. Open https://threejs.org/examples/?q=video#webgl_materials_video and press play.

Expected result

• Video play.

Actual result

• Video remains black.

Regression range

• The issue reproduces with Firefox 130.0a1 (2024-07-08) on Ubuntu 24.04. I will search for a regression range asap.

Additional notes

• This works with Firefox 128.3.1esr.

Comment 1

[Timothy Nikkel (:tnikkel)]

(In reply to Alexandru Trif, Desktop Test Engineering [:atrif] from comment #0)

Regression range

• The issue reproduces with Firefox 130.0a1 (2024-07-08) on Ubuntu 24.04. I will search for a regression range asap.

I see this statement in a lot of bugs, but a regression range never follows. It seems to be in some template? Could you remove it from the template unless you are actually planning on finding the regression range?

Comment 2

[Timothy Nikkel (:tnikkel)]

https://hg.mozilla.org/integration/autoland/pushloghtml?fromchange=efb4ec030ec0222094fa18c0aea59f1944aa36fc&tochange=98d171e8ce2530fa3f67289b77f276fa9d0545b8

Comment 3

[BugBot [:suhaib / :marco]]

:padenot, since you are the author of the regressor, bug 1860492, could you take a look? Also, could you set the severity field?

For more information, please visit BugBot documentation.

Comment 4

[Masatoshi Kimura [:emk]]

<video id="video" loop crossOrigin="anonymous" playsinline style="display:none">
<source src="textures/sintel.ogv" type='video/ogg; codecs="theora, vorbis"'>
<source src="textures/sintel.mp4" type='video/mp4; codecs="avc1.42E01E, mp4a.40.2"'>
</video>

Firefox fails to found a suitable video format on macOS/linux? Can Firefox play https://threejs.org/examples/textures/sintel.mp4 alone on macOS/linux?

Comment 5

[Timothy Nikkel (:tnikkel)]

(In reply to Masatoshi Kimura [:emk] from comment #4)

Firefox fails to found a suitable video format on macOS/linux? Can Firefox play https://threejs.org/examples/textures/sintel.mp4 alone on macOS/linux?

That url does not play on my mac. That's been the same since 2020.

Comment 6

[Alexandru Trif, Desktop Test Engineering [:atrif]]

(In reply to Timothy Nikkel (:tnikkel) from comment #1)

(In reply to Alexandru Trif, Desktop Test Engineering [:atrif] from comment #0)

Regression range

• The issue reproduces with Firefox 130.0a1 (2024-07-08) on Ubuntu 24.04. I will search for a regression range asap.

I see this statement in a lot of bugs, but a regression range never follows. It seems to be in some template? Could you remove it from the template unless you are actually planning on finding the regression range?

https://hg.mozilla.org/integration/autoland/pushloghtml?fromchange=efb4ec030ec0222094fa18c0aea59f1944aa36fc&tochange=98d171e8ce2530fa3f67289b77f276fa9d0545b8

Hello! Thank you for the pushlog. I am confirming the regression range on Ubuntu 24.04/macOS 12 as well. Note that the issue does not reproduce with Firefox 128.3.1 esr on macOS 12 and Ubuntu 24.04 for some reason.
Regarding the regression range, it's part of our template and our responsibility to perform it. This template is typically used when urgent tasks with deadlines take priority, and there isn't enough time to conduct the regression range on that day if it's not a recent regression and we get back to it as soon as time permits. If a regression range is urgent, a ni? can be requested from the reporter. Thank you!

Comment 7

[Paul Adenot (:padenot)]

It is expected that the theora video doesn't play. It isn't expected that the mp4 doesn't play.

Comment 8

[Paul Adenot (:padenot)]

John, can you look at this mp4? https://pernos.co/debug/9pxgcCbicAi9Flo8cmtFrQ/index.html#f{m[BTcp,AQV4_,t[ASk,P1pI_,f{e[BTcp,/ms_,s{acruKIAAA,bAZc,uD5Vm/A,oD57MHg___/ is a pernosco recording of this.

Comment 9

[BugBot [:suhaib / :marco]]

Set release status flags based on info from the regressing bug 1860492

Comment 10

[Paul Adenot (:padenot)]

John is currently off, Ashley?

Comment 11

[Jim Mathies [:jimm]]

(In reply to Timothy Nikkel (:tnikkel) from comment #5)

(In reply to Masatoshi Kimura [:emk] from comment #4)

Firefox fails to found a suitable video format on macOS/linux? Can Firefox play https://threejs.org/examples/textures/sintel.mp4 alone on macOS/linux?

That url does not play on my mac. That's been the same since 2020.

This plays on my Mac M2 just fine.

Comment 12

[az [:az]]

If we check the Pernosco recording, it looks like we're processing two packets, the second of which fails. I'll call them packet1 and packet2. We have:

packet1->size = 320
packet1->pos = 26061

packet2->size = 57
packet2->pos = 26379

If we add packet1->size + packet1->pos, we end up at 26381, packet2->pos is 26379, two less. I was wondering the overlapping packet data here could be related?

The invalid data failure is being returned from the second execution of avcodec_send_packet which returns -1094995529 which is AVERROR_INVALIDDATA. The Pernosco data for avcodec_send_packet has a bunch optimized out, but I think we're failing here with the call to decode_receive_frame_internal:

https://searchfox.org/mozilla-central/rev/5b93d2870ab36ba9cfa144fa7518d54e5439caa9/media/ffvpx/libavcodec/decode.c#699-703

And then in decode_receive_frame_internal, I think we're failing at decode_simple_receive_frame here:
https://searchfox.org/mozilla-central/rev/5b93d2870ab36ba9cfa144fa7518d54e5439caa9/media/ffvpx/libavcodec/decode.c#580-582

Comment 13

[Jens Stutte [:jstutte]]

I assume 135 is still affected, please correct the flags otherwise.

Comment 14

[Ryan VanderMeulen [:RyanVM]]

Can we find an assignee for this?

Comment 15

[az [:az]]

:jolin, would you have any thoughts on the packet / debug info from comment 12?

Comment 16

[Jim Mathies [:jimm]]

We've found this is working on Nightly, so we're searching for a regression range.

Comment 17

[Jim Mathies [:jimm]]

Plays on windows 11 137 fine.

Comment 18

[John Lin [:jhlin][:jolin]]

:jolin, would you have any thoughts on the packet / debug info from comment 12?

I think it is because of AnnexB::ConvertSampleToAVCC() called by H264ChangeMonitor::CheckForChange(). The source MP4 has 2-byte NAL units and the conversion changes them to 4-byte, so the sizes will be increased by 2.

Comment 19

[John Lin [:jhlin][:jolin]]

After some investigation, I found the issues on macOS and Linux are different in my local tests.

In Firefox 135 on macOS, the source video file plays but the three.js example doesn't show visual content while audio plays fine. However, when I shrink the window to make the canvas small enough, the video would show up. I think it related to the gfx rather than media. :atrif, could you please confirm that it behaves similar on your side?

On the other hand, the source video doesn't play on my Linux machine and seems caused by the 2-byte to 4-byte conversion mentioned in comment 18. It looks like the original NAL unit size (2) was used by the decoder to parse the converted 4-byte NAL unit sample and it fails. I will upload a patch to address this.

Comment 20

[Alexandru Trif, Desktop Test Engineering [:atrif]]

(In reply to John Lin [:jhlin][:jolin] from comment #19)

After some investigation, I found the issues on macOS and Linux are different in my local tests.

In Firefox 135 on macOS, the source video file plays but the three.js example doesn't show visual content while audio plays fine. However, when I shrink the window to make the canvas small enough, the video would show up. I think it related to the gfx rather than media. :atrif, could you please confirm that it behaves similar on your side?

Hmm I cannot play the video with 135.0.1 and 137.0a1 (2024-02-25) on macOS 12 even if I shrink the window. Also no sound on my end. I cannot play the source video either, it says that the video cannot be played because it's corrupt. If more information is needed please let me know.

Comment 21

[John Lin [:jhlin][:jolin]]

Attachment: Bug 1924070 - modify H.264 extradata to match sample conversion code.

Comment 22

[John Lin [:jhlin][:jolin]]

Attachment: Bug 1924070 - add 2-byte NALU AVC video test. r=alwu

Comment 23

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/004d1a94627f
modify H.264 extradata to match sample conversion code. r=media-playback-reviewers,alwu
https://hg.mozilla.org/integration/autoland/rev/072267a4c98d
add 2-byte NALU AVC video test. r=alwu

Comment 24

[amarc]

https://hg.mozilla.org/mozilla-central/rev/004d1a94627f
https://hg.mozilla.org/mozilla-central/rev/072267a4c98d

Comment 25

[BugBot [:suhaib / :marco]]

The patch landed in nightly and beta is affected.
:jhlin, is this bug important enough to require an uplift?

• If yes, please nominate the patch for beta approval.
• If no, please set status-firefox137 to wontfix.

For more information, please visit BugBot documentation.