1903543 - Tiktok cover image and feed-preview-image are stuck on blank / "Loading..." after uploading an image in Firefox

Status: RESOLVED FIXED

(Core :: Audio/Video: Web Codecs, defect, P2)

Description

[Calin Tanase]

Attachment: Upload video FF vs Chrome.mp4

Environment:
Operating system: Windows 10
Firefox version: Nightly 129.0a1 (2024-06-18)

Preconditions:
• must be logged in

Steps to reproduce:

1. Go to https://www.tiktok.com/tiktokstudio
2. Click on "Upload".
3. Upload a video.
4. Observe the layout and functionalities.

Expected Behavior:
The layout is the same as on Chrome.

Actual Behavior:
The layout is different and has different functionalities.

Notes:

1. Screenshot attached
2. Reproducible regardless of the ETP status
3. Reproducible on Firefox Release as well
4. Not reproducible on Chrome
5. Issue found during WebCompat team [Top100] websites testing

Comment 1

[Jeff Muizelaar [:jrmuizel]]

I see the same UI in Chrome and Firefox. Calin, can you still reproduce this?

Comment 2

[Jeff Muizelaar [:jrmuizel]]

Attachment: A canvas-rr recording of the problem

(wrong bug)

Comment 3

[Jeff Muizelaar [:jrmuizel]]

(wrong bug)

Comment 4

[Calin Tanase]

Attachment: Nightly vs Chrome (update).png

The UI looks the same now but the cover image and feed preview is no longer displayed. Might be related to 1933910.

Comment 5

[Daniel Holbert [:dholbert]]

(In reply to Calin Tanase from comment #4)

the cover image and feed preview is no longer displayed.

Are you still seeing that issue?

I just tested this and the Cover and Feed Preview image rendered just fine, using a still frame from my uploaded video. (In my case, the uploaded video was a brief .webm screencast that I just recorded locally.) So if this is reliably broken for you with a certain video file, it'd be great to have that known-broken file available for testing -- would you mind sharing your known-to-reproduce-this-issue video file, e.g. as an attachment here or on gdrive, so that other folks can use it to test to try to reproduce your experience?

[Assuming this is still broken, this is probably closest to the media team, so let's start this out in their queue for diagnosis, pending confirmation from Calin that this is still reproducible with a particular media file. --> Making that adjustment and updating bug title to reflect the modified scope here per comment 4.]

Comment 6

[Calin Tanase]

Attachment: Video sample.mp4

The issue is still reproducible for me, the cover and feed preview are still not shown. I've attached the video I'm reproducing the issue with.

Comment 7

[Daniel Holbert [:dholbert]]

Thanks. I can't repro on LInux using that file -- here's a performance profile of me getting expected-results in a fresh user-profile there:
https://share.firefox.dev/3VxO62O

However, I am able to reproduce in Nightly on Windows 11 (using my main browsing profile there). I'll capture and share a performance profile shortly there.

Comment 8

[Daniel Holbert [:dholbert]]

Here's a performance profile of the bug reproducing in Nightly on Windows 11, using a fresh Firefox profile (where I've done nothing besides log in to TikTok and try to upload this video):
https://share.firefox.dev/3Bvk9cY

Comment 9

[Daniel Holbert [:dholbert]]

Here's a a somewhat longer profile of the bug in Nightly on Windows 11:
https://share.firefox.dev/4gdBOoL

In this one, I click into the "Edit Cover" UI several times. The first time, the film-strip and chosen-cover preview are both blank. The second time, the film-strip is populated. The third time, both are populated. Not sure what to make of that; it's just some clues.

Another clue: TikTok has some helpful UserTiming markers which are present in all profiles here: loadCover, some markers with VIDEO_COVER_PREVIEW_READY in the name, followed shortly by load events for elements like img@24fe320ea80 class="jsx-510587813 cover-image". When those events fire, the "Loading..." overlay disappears from the cover element, and DevTools shows me that the img element looks like this:

<img src="blob:https://www.tiktok.com/d1c11463-8dcf-4957-8b5c-6f594e97e238" crossorigin="anonymous" class="jsx-510587813 cover-image">

...where the blob URI is a 640x360 fully-transparent image.

This feels a little like bug 1927346, where maybe we're drawing an image to a canvas to get a screenshot, and that ends up failing in some way....

Comment 10

[Daniel Holbert [:dholbert]]

Attachment: the blob-URI image that TikTok is showing here

the blob URI is a 640x360 fully-transparent image.

Here's that blob URI image, for what it's worth.

Comment 11

[Daniel Holbert [:dholbert]]

Attachment: the blob-URI image that TikTok is showing here

sorry, here's the actual blob (the previous one was copypasted via clipboard which changed it in some way I think)

Comment 12

[Daniel Holbert [:dholbert]]

Testing on my Windows 11 machine where the issue reproduces, this seems to have been a regression in 2023. Builds from early 2023 give expected-results here.

Last good build: 2023-06-29
First-bad build: 2023-06-30
Regression Pushlog: https://hg.mozilla.org/mozilla-central/pushloghtml?fromchange=c5ac7e957828eeda97a3af6fc706f41aa567e48e&tochange=0abcd848f8218f0e2d339bbd497ac60f4a3b752c

In that range, bug 1831451 seems like the most likely patch-stack to have impacted video-related stuff. (Its bug title makes it out to be linux-specific, but it looks like it involved a whole bunch of patches which might've had unforeseen behavior impact on other platforms (?) too.)

Tentatively flagging as a regression from bug 1831451.

Comment 13

[BugBot [:suhaib / :marco]]

Set release status flags based on info from the regressing bug 1831451

:chunmin, since you are the author of the regressor, bug 1831451, could you take a look?

For more information, please visit BugBot documentation.

Comment 14

[Daniel Holbert [:dholbert]]

Comparing DevTools Web Console content from the last-good vs. first-bad builds (2023-06-29 vs 2023-06-30) for the STR, I'm noticing that the last-good build has:

ReferenceError: VideoDecoder is not defined 7947.3dd75cf7.js:1:9625

...whereas the first-bad build has this instead:

InvalidStateError: An attempt was made to use an object that is not, or is no longer, usable 2 capture.0dca6d63.js:2
InvalidStateError: VideoDecoder.decode: Decoder must be configured first 2 capture.0dca6d63.js:2
InvalidStateError: VideoDecoder.decode: Decoder must be configured first 7947.3dd75cf7.js:1
InvalidStateError: VideoDecoder.decode: Decoder must be configured first 7947.3dd75cf7.js:1

And modern-day Nightly has this instead at the same location in my logcat-spew, instead of the above logging:

DOMException: OffscreenCanvasRenderingContext2D.drawImage: Passed-in video frame is "broken"
[...]
Uncaught DOMException: CanvasRenderingContext2D.drawImage: Passed-in video frame is "broken"

So:

• the original regression here does indeed seem to be about the VideoDecoder API that was featured in bug 1831451. Presumably in Nightly 2023-06-29 (before bug 1831451 landed), TikTok was falling back to some other method for extracting a video frame, I guess? And then VideoDecoder became available which makes their logic go down a route to use that API, and they run into trouble somewhere.
• Whatever original issue we were hitting (with the InvalidStateError): we've apparently since addressed that, but we're now hitting this new Passed-in video frame is "broken" error when drawing video frames to the canvas.

Also, FWIW - I tested this with software WebRender force-enabled on Windows 11 just in case that made a difference (since that mitigates similar-ish bug 1927346); but that doesn't help here. The bug is still reproducible on Windows 11 regardless of whether I'm using hardware vs. software WebRender. (And it's not-reproducible-for-me on Linux regardless of whether I'm using hardware vs. software webrender.)

Comment 15

[Daniel Holbert [:dholbert]]

The Passed-in video frame is "broken" exception (which shows up in console.log in modern-day Nightly here) seems to live here in-tree:
https://searchfox.org/mozilla-central/rev/9783996dbd86f999cab50ea426079a7b10f28a2f/dom/canvas/CanvasRenderingContext2D.cpp#5675
and it was added in bug 1781527, so this is presumably somewhat related to that bug.

Comment 16

[Jim Mathies [:jimm]]

Andrew, looks like this regressed when we landed video frame canvas support for webcodecs. Take a look when you have a chance.

Comment 17

[C.M.Chang[:chunmin]]

I've tested this on Windows, MacOS, and Ubuntu. It looks like the issue can only be reproduced on Windows.

The VideoFrame is produced from VideoDecoder here. If SurfaceFromElementResult::GetSourceSurface() returns a null, maybe we can see a warning: Cannot readback from shared texture because no texture is available from D3D11ShareHandleImage::GetAsSourceSurface() or D3D11ZeroCopyTextureImage::GetAsSourceSurface()?

Comment 18

[Jim Mathies [:jimm]]

Andrew thought this might be in Sotaro's area of expertise.

Comment 19

[Sotaro Ikeda [:sotaro]]

D3D11ZeroCopyTextureImage::GetAsSourceSurface() failure was caused by hitting D3D11DXVA2Manager count limit in DXVA2Manager::CreateD3D11DXVA().

DXVA2Manager is allocated only for color conversion. Then DXVA2Manager::CreateD3D11DXVA() should succeed.

In future, it seems better to replace color conversion with DXVA2Manager by VideoProcessorD3D11.

Comment 20

[Sotaro Ikeda [:sotaro]]

Attachment: Bug 1903543 - Ensure DXVA2Manager::CreateD3D11DXVA() succeeds when DXVA2Manager is only used for color transformation

Comment 21

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/f6f93e221761
Ensure DXVA2Manager::CreateD3D11DXVA() succeeds when DXVA2Manager is only used for color transformation r=media-playback-reviewers,padenot

Comment 22

[Goloman Adrian[:agoloman]]

https://hg.mozilla.org/mozilla-central/rev/f6f93e221761

Comment 23

[Negritas Sergiu, Desktop QA]

Hello! I have managed to reproduce the issue with firefox 129.0a1(2024-06-19), I can confirm that the issue is fixed with firefox 137.0a1(2025-02-06) and 136.0b2 on Windows 10, Ubuntu 22.04 and MacOS 15.2..

I will update the status and flags of this issue.

Have a nice day!