1902076 - onnxruntime webgpu backend doesn't work in Firefox

Status: RESOLVED FIXED

(Core :: Graphics: WebGPU, defect, P3)

Description

[Jeff Muizelaar [:jrmuizel]]

This shows up on https://huggingface.co/spaces/Xenova/whisper-webgpu and https://huggingface.co/spaces/Xenova/experimental-phi3-webgpu.

In the console I see: Unsupported device: "webgpu". Should be one of: wasm"

Both of those pages are trying to use the webgpu backend of onnxruntime-web via transformers.js

Comment 1

[Jeff Muizelaar [:jrmuizel]]

WebGPU is not detected because it's not enabled in workers.

Comment 2

[Jeff Muizelaar [:jrmuizel]]

When I turn on WebGPU in workers I get a shader validation error and then a crash in nsGlobalWindowInner::GetConsole

Comment 3

[Jim Blandy :jimb]

Marking P1: popular demo

Comment 4

[Erich Gubler [:ErichDonGubler] (he/him)]

The crash mentioned by :jrmuizel in comment 2 happens while failing to get a console for the worker context. I wrote that code, and, unfortunately, I'm not surprised it doesn't work in worker contexts. Once that is fixed, we'll likely have workable diagnostics around shader validation errors. Filed a bug for this: https://bugzilla.mozilla.org/show_bug.cgi?id=1902117

Comment 5

[Jeff Muizelaar [:jrmuizel]]

On https://huggingface.co/spaces/Xenova/experimental-phi3-webgpu with Erich's fix for bug 1902117 I get the following shader validation errors:

Shader validation error: 
   ┌─ BlockwiseMatMulNBits:78:13
   │
78 │             word_offset += 2;
   │             ^^^^^^^^^^^^^^^^ naga::Expression [163]



Shader validation error: 
   ┌─ RotaryEmbedding:55:62
   │
55 │                 u32(i32(position_ids[position_ids_idx].x)) + select(0, bsnh[1], position_ids_idx == 0);
   │                                                              ^^^^^^ naga::Expression [43]



Shader validation error: 
   ┌─ Concat:40:3
   │  
40 │ ╭   fn calculateInputIndex(index: u32) -> u32 {
41 │ │     let sizeInConcatAxis = array<u32, 1u>(uniforms.sizeInConcatAxis0);
42 │ │     for (var i: u32 = 0u; i < 1; i += 1u ) {
43 │ │       if (index < sizeInConcatAxis[i]) {
   │ │                   ^^^^^^^^^^^^^^^^^^^ naga::Expression [12]
   · │
46 │ │     }
47 │ │     return 1u;
   │ ╰──────────────^ naga::Function [4]

Comment 6

[Erich Gubler [:ErichDonGubler] (he/him)]

:jrmuizel: :teoxoy and I have determined that those are because of incomplete automatic type conversions, which are tracked at bug 1829879.

Comment 7

[Jeff Muizelaar [:jrmuizel]]

I currently get:

Error: no available backend found. ERR: [webgpu] TypeError: e.requestAdapterInfo is not a function

Comment 8

[Jeff Muizelaar [:jrmuizel]]

Chrome as of 131 gives the same requestAdapterInfo is not a function issue on whisper-wgpu

Comment 9

[Erich Gubler [:ErichDonGubler] (he/him)]

:jrmuizel: That was changed in Firefox to conform to spec. with bug 1919119. onnxruntime has fixed their upstream at microsoft/onnxruntime#21065 and released with 1.19.0, but it seems that xenova/whisper-web (which I think backs this demo) does not yet consume that fix.

Comment 10

[Erich Gubler [:ErichDonGubler] (he/him)]

Oh, actually, this issue is for onnxruntime; I think this has been resolved. Closing.