Bugzilla

Discussed the way we might resolve this a bit with :jgilbert, :teoxoy, and :jimb, since it's become relevant with :jgilbert's work on bug 1831263. Our tentative direction is to use a hybrid manual-automatic WPT test chunking approach based on the upstream tree structure, instead of the “linear” chunking we do. The things we're thinking of getting with it:

- Hand-pick a set of test paths that are smoke-level tests, much like WebGL's “core” tasks in CI are currently structured.
    - Maybe split the tests b/w API and shaders?
- “auto”-chunk everything else into CTS test files based on, say, unique path components to a depth of 5, i.e., `webgpu:api,operation,adapter,requestDevice` becomes its own WPT test file.
- When a set of tests from the above are taking too long, we split it. We feel that it should be much easier to understand and resolve longer running sets of tests when they're divided conceptually, so this seems tractable to plan on doing.

Discussed the way we might resolve this a bit with :jgilbert, :teoxoy, and :jimb, since it's become relevant with :jgilbert's work on bug 1831263. Our tentative direction is to use a hybrid manual-automatic WPT test chunking approach based on the upstream tree structure, instead of the “linear” chunking we do. The things we're thinking of getting with it:

- Hand-pick a set of test paths that are smoke-level tests, much like WebGL's “core” tasks in CI are currently structured.
    - Maybe split the tests b/w API and shaders?
- “auto”-chunk everything else into CTS test files based on, say, unique path components to a depth of 5, i.e., `webgpu:api,operation,adapter,requestDevice,*` and all other tests underneath it becomes their own WPT test file.
- When a set of tests from the above are taking too long, we split it. We feel that it should be much easier to understand and resolve longer running sets of tests when they're divided conceptually, so this seems tractable to plan on doing.

Discussed the way we might resolve this a bit with :jgilbert, :teoxoy, and :jimb, since it's become relevant with :jgilbert's work on bug 1831263. Our tentative direction is to use a hybrid manual-automatic WPT test chunking approach based on the upstream tree structure, instead of the “linear” chunking we do. The things we're thinking of getting with it:

- Hand-pick a set of test paths that are relatively fast smoke-level tests, much like WebGL's “core” tasks in CI are currently structured. This is intended allow folks to triage whether or not they've broken something fundamental quickly.
    - Maybe split the tests b/w API and shaders?
- “auto”-chunk everything else into CTS test files based on, say, unique path components to a depth of 5, i.e., `webgpu:api,operation,adapter,requestDevice,*` and all other tests underneath it becomes their own WPT test file.
- When a set of tests from the above are taking too long, we split it, and add task chunks as necessary. We feel that it should be much easier to understand and resolve longer running sets of tests when they're divided conceptually, so this seems tractable to plan on doing.