1746250 - Hook up fuzzing for WebGPU code

Status: RESOLVED FIXED

(Core :: Graphics: WebGPU, task, P1)

Description

[Dzmitry Malyshau [:kvark]]

In order to be confident in security of WebGPU, we need to fuzz it heavily and see what shows up.

Comment 1

[Daniel Veditz [:dveditz]]

I've seen bugs filed that came from oss-fuzz. Do we know what that fuzzer covers and what it doesn't? Is it a fuzzer specifically for our implementation, or more generic for the API so it works in Chrome, too? If so are there parts of our implementation that differs from Chrome that we'd want to craft a fuzzer to exercise specifically?

It could be that our fuzzing team will decide the oss-fuzz one is adequate. If oss-fuzz is operating strictly at the library level we may want to adapt something that runs in the browser itself to make sure all the other layers are playing nicely together.

You probably should reach out to them (email, or #fuzzing:mozilla.org on matrix). That team is unlikely to be looking in your component and find this bug on their own.

Comment 2

[Dzmitry Malyshau [:kvark]]

Yes, for sure we'll reach out to them! This bug is just here to block bug 1746245. Basically, for tracking.
I haven't looked into this myself yet in depth, so I can't answer your other questions straight away.

Comment 3

[Tyson Smith [:tsmith]]

I am not aware of a projects targeting WebGPU in oss-fuzz. The list of projects can be found here: https://github.com/google/oss-fuzz/tree/master/projects.

The first step would be to make sure all required prefs are set in prefpicker: https://github.com/MozillaSecurity/prefpicker/blob/master/prefpicker/templates/browser-fuzzing.yml. Either let me know what needs to be added or feel free to open a PR. The existing DOM fuzzers will then be able to start fuzzing. The fuzzing team may need to update the DOM fuzzers (browser) to achieve as much coverage as possible (CC'ing jkratzer).

A libfuzzer based fuzzer can be created. If a dev can take the lead and build a fuzzer the team would be happy to handle automation and triage results. This is similar to creating gtests and user accessible APIs should be prioritized. More information can be found here: https://firefox-source-docs.mozilla.org/tools/fuzzing/fuzzing_interface.html

Comment 4

[Daniel Veditz [:dveditz]]

pre-pandemic there must've been something in oss-fuzz: you filed bug 1610652

Comment 5

[Tyson Smith [:tsmith]]

That was found via the CompositorManagerParentIPC fuzz target. This target is unreliable and is a lot of maintenance and has been deprecated.

Comment 6

[Kelsey Gilbert [:jgilbert]]

We've had some success with fuzzing webgl, so hopefully we can build on that design.

Comment 7

[Jim Blandy :jimb]

Jason's filing blockers on webgpu-fuzz. This bug is about setting up fuzzing - can it be closed, or is there more work to do?

Comment 8

[Jason Kratzer [:jkratzer]]

I'm not really sure what the difference is between this and bug 1752407. :kvark, mind weighing in here?

Comment 9

[Dzmitry Malyshau [:kvark]]

webgpu-fuzz is a meta-bug to depend on all discovered fuzzing bugs. It's not meant to be closed, it's a live tracker.
The current bug 1746250 is about a general task of implementing fuzzing for WebGPU. It's not limited to bug 1750020. This bug will be closed when we have all of the infrastructure necessary to move forward with enabling on Nightly.

Comment 10

[Jim Mathies [:jimm]]

Since this bug is about hooking various fuzzers up to WebGPU interfaces, I wonder if it should be moved to a fuzzing product/component in Bugzilla vs. where it's filed currently?

Comment 11

[Jim Mathies [:jimm]]

Comments in a duped meta, Dzmitry mentioned -

fuzzing WGSL parser (https://github.com/gfx-rs/naga/issues/1704)

^ Filed against Naga by Dzmitry with no activity. I guess this is something the WebGPU team will need to hook up?

fuzzing Naga IR (https://github.com/gfx-rs/naga/pull/1668)
Bug 1750019 - cargo-fuzz WGSL

^ Changes to help setup for use of cargo fuzz AFAICT. We also have a cargo fuzz related bug as a blocker to this bug, but I'm not sure what that involves.

fuzzing IPDL entry points on the content side (already in progress)

^ Does anyone have a bug # tracking this work?

fuzzing WebGPU JS API (bug 1746250)

^ This bug.

Jim, can we try to get these fuzzing related tasks nailed down a bit better here?

Comment 12

[Jason Kratzer [:jkratzer]]

(In reply to Jim Mathies [:jimm] from comment #10)

Since this bug is about hooking various fuzzers up to WebGPU interfaces, I wonder if it should be moved to a fuzzing product/component in Bugzilla vs. where it's filed currently?

:jimm, I don't have a preference where it's filed though there's not really a fuzzing specific component that we could file it under. This may be better as a board on Jira.

Comment 13

[Jim Blandy :jimb]

Jim, can we try to get these fuzzing related tasks nailed down a bit better here?

Yeah - I'll try to make this all easier to follow.

The Naga IR fuzzing found a bunch of missing points in the validator; these were fixed here: https://github.com/gfx-rs/naga/pull/1668

I think Dzmitry had started working on the WGSL fuzzing, but this is something that Mozilla's fuzzing team could help with as part of the JS API fuzzing. In particular, feeding interesting strings to GPUDevice::createShaderModule will be very similar, although using Naga directly would avoid any unexpected browser influences.

Comment 14

[Jason Kratzer [:jkratzer]]

This is certainly something we can help with, however we wouldn't do it via the JS API. It's simply too slow and isn't deterministic enough to use something like libfuzzer. What we discussed with Dzmitry was the possibility of adapting something like GraphicsFuzz for WGSL.

Comment 15

[Jim Blandy :jimb]

We will be tracking fuzzing of WebGPU shader validation in bug 1760648.

Comment 16

[Jim Blandy :jimb]

Note that bug 1750019 is not a duplicate of bug 1760648. The former is for fuzzing the WGSL front end, while the latter covers Naga IR validation - the next stage of the pipeline. It's worthwhile to have the fuzzers work on both points, because IR validation is meant to stand on its own, without trusting the front ends. Generating IR directly will help us cover cases that are difficult to provoke the WGSL front end into generating.

Comment 17

[Jim Blandy :jimb]

(In reply to Jim Mathies [:jimm] from comment #11)

So here's the state of play:

We have four entry points that are worth fuzzing. Ordered from 'front' (content JS) to 'back' (platform graphics APIs), they are:

• Fuzzing the WebGPU JS API, as defined in WebGPU.webidl

  Covered by bug fuzzing-webgpu.

  Fuzzing is running. Fuzzblockers are P1 tasks for Jim.

• Fuzzing the WebGPU IPDL protocol, as defined in PWebGPU.ipdl

  Covered by bug 1760666.

  IPDL fuzzing is still experimental. As of this writing, Christian (decoder) is running it locally on his machines, but it can't be scaled in its current form. Bug 1760666 covers implementing this once the harness is ready.

• Fuzzing the WebGPU Shading Language (WGSL) front end, in the Naga crate.

  Covered by bug 1750019.

  To some extent this is already covered by JS API fuzzing: if the fuzzer can create an Adapter and a Device, then calls to Device::createShaderModule talk to the WGSL front end pretty directly. However, I think it would still be valuable to fuzz the Naga crate directly.

• Fuzzing the WGSL validator, in the Naga crate.

  Covered by bug 1760648.

  Naga is straightforward to use in isolation, without Firefox or even wgpu. Fuzzing Naga directly would give the fuzzer better access to complex code that is crucial to WebGPU's safety story. See that bug for more details.

There are the GitHub issues naga#1704 and naga#1668, but we don't really have control over the open-source project's priorities. I think Mozilla should track its own work in Bugzilla, as suggested above.

Comment 18

[Jim Blandy :jimb]

This bug has value as a metabug for getting various kinds of fuzzing hooked up, but we've got API fuzzing going and producing results, so I'm moving this to webgpu-mvp.