1829957 - (webgpu-webgpt) [meta] Ensure that WebGPT works

Status: NEW

(Core :: Graphics: WebGPU, task, P3)

Description

[Erich Gubler [:ErichDonGubler] (he/him)]

WebGPT is a popular project that uses large language models to deliver an experience similar to OpenAI's ChatGPT, but running the model client-side using WebGPU, instead of on a server. Our WebGPU implementation doesn't successfully run it yet, but we'd really like it to. This bug captures the remaining work identified as dependencies for making that happen.

WebGPT has its source hosted on GitHub, but for convenience, they've hosted a demo here: https://www.kmeans.org/

Comment 1

[Erich Gubler [:ErichDonGubler] (he/him)]

In order to actually test WebGPU in these pages, you have to:

1. Click one of the Load Shakespeare Model or Load GPT2 model buttons. Observe that controls below are now enabled.
2. Open the JS console (i.e., via Ctrl + Shift + I).
3. Click Generate Text (the prompt doesn't matter yet). Observe that a multitude of WebGPU shader compilation errors are raised.

Comment 2

[Andy Leiserson [:aleiserson]]

Current compilation error:

Shader validation error: Entry point main at Compute is invalid
   ┌─ :46:43
   │
46 │         threadVariance = threadVariance + pow(input_array[row * N + i] - row_mean, 2);
   │                                           ^^^ naga::Expression [70]
   │
   = Expression [70] is invalid
   = Argument [1] to Pow as expression [69] has an invalid type.

Comment 3

[Erich Gubler [:ErichDonGubler] (he/him)]

I can run the demo now without shader compilation errors, but now I notice a couple of problems:

1. The visual area below text entry only is ever a small black rectangle. 😐
2. The application OOMs fairly quickly after clicking Generate Text. Yikes!