1827116 - (webgpu-external-textures) [meta] External texture API missing in WebGPU (`GPUDevice.importExternalTexture`, `GPUExternalTexture`, `texture_external`)

Status: NEW

(Core :: Graphics: WebGPU, defect, P1)

Description

[mighdoll]

User Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:109.0) Gecko/20100101 Firefox/113.0

Steps to reproduce:

visit thimbleberry.dev and click the webcam button, or the sample media and then the 'jump' video

Actual results:

The video path processing path fails because:
importExternalTexture is undefined

src here: https://github.com/mighdoll/thimbleberry/blob/main/image-demo/src/ImageProcessing.ts

Comment 1

[Erich Gubler [:ErichDonGubler] (he/him)]

This bug happens because GPUDevice.importExternalTexture (orig. spec PR, current spec) is part of the current v1 draft of the WebGPU API spec, but not yet implemented. The effort involved to resolve this is not yet clear. Assigning this bug to myself to figure it out.

Comment 4

[Erich Gubler [:ErichDonGubler] (he/him)]

This API is definitely necessary for webgpu-v1-samples' videoUploading and videoUploadingCodecs demos, which is a relatively high priority for us to resolve in Firefox's WebGPU implementation. However, we believe we should unblock some other bugs that block that specific issue before attending to this one specifically. I'm going to unassign myself for now, since I'm not giving any active attention to this bug.

Comment 5

[Erich Gubler [:ErichDonGubler] (he/him)]

For reference of the future assignee to this bug, here's what I was able to find relevant to this set of APIs; we basically need to implement these pieces of the current drafts of the v1 spec.:

• Section 6.4 of the v1 WebGPU API spec. (GPUExternalTexture).
• Section 5.4.4 of the v1 WGSL spec. (External sampled texture types).

Comment 6

[Jim Blandy :jimb]

Upstream: https://github.com/gfx-rs/naga/issues/1894

Comment 7

[Erich Gubler [:ErichDonGubler] (he/him)]

VideoFrame blocks the videoUploadingWebCodecs demo specifically.

Comment 8

[Kelsey Gilbert [:jgilbert]]

I'll take this, up to but excluding VideoFrame for now.

Comment 9

[Erich Gubler [:ErichDonGubler] (he/him)]

A relevant wgpu upstream PR: https://github.com/gfx-rs/wgpu/pull/3019

Comment 10

[Sotaro Ikeda [:sotaro]]

In chromium, followings seem to be related to ExternalTexture.

• GPUDevice::importExternalTexture()
• GPUExternalTexture::FromHTMLVideoElement()
• GPUExternalTexture::FromVideoFrame()
• GPUExternalTexture::CreateImpl()
• WebGPUMailboxTexture::FromVideoFrame()
• GPUQueue::copyExternalImageToTexture()
• GPUQueue::CopyFromVideoElement()
• DeviceBase::APICreateExternalTexture()
• QueueBase::APICopyExternalTextureForBrowser()
• BuildExternalTextureTransformBindings()
• GenerateExternalTextureBindings()

third_party/blink/renderer/modules/webgpu/gpu_external_texture.cc
third_party/blink/renderer/modules/webgpu/external_texture_helper.cc
third_party/dawn/src/tint/lang/core/ir/transform/multiplanar_external_texture.cc
third_party/dawn/src/tint/lang/wgsl/ast/transform/multiplanar_external_texture.cc
third_party/dawn/src/tint/lang/core/type/external_texture.cc

Comment 11

[Jim Blandy :jimb]

Jamie says that his work will be supporting both HTMLVideoElement and VideoFrame. I think this is everything the spec requires.

The GPUDevice.importExternalTexture method takes a GPUExternalTextureDescriptor to describe what's being imported. The WebIDL for that is:

dictionary GPUExternalTextureDescriptor
         : GPUObjectDescriptorBase {
    required (HTMLVideoElement or VideoFrame) source;
    PredefinedColorSpace colorSpace = "srgb";
};

Comment 12

[Jamie Nicol [:jnicol]]

I've been using this as a meta for all external texture work. Updating the title to make that clear, as the specific error message indicated in the title will be fixed soon but there will be remaining work to do

Comment 13

[Jamie Nicol [:jnicol]]

ugh ignore me I had misread the title

Comment 14

[Jim Blandy :jimb]

Jamie, since we've shipped this on Windows and macOS, can we close this bug?

Comment 15

[Jamie Nicol [:jnicol]]

If you think shipping on Windows and macOS counts as "done", since we're not yet shipping WebGPU on Linux or Android, then sure.

I've filed bug 2008705 and bug 2008706 for the remaining Linux and Android implementations. We can either make them block this bug and leave it open, or close this bug and perhaps link them elsewhere?. What do you think?