Closed Bug 1891894 Opened 2 years ago Closed 2 years ago

[wpt-sync] Sync PR 45753 - Reland "webnn: Migrate Conv2d validation tests to WPTs"

Categories

(Testing :: web-platform-tests, task, P4)

Product:
Testing
Component:
web-platform-tests
Type:
task

Tracking

(firefox127 fixed)

Status:
RESOLVED FIXED
Milestone:
127 Branch
Tracking Flags:
Tracking Status
firefox127 --- fixed

People

(Reporter: wpt-sync, Unassigned)

References

(
URL
)

Details

(Whiteboard: [wptsync downstream])

Sync web-platform-tests PR 45753 into mozilla-central (this bug is closed when the sync is complete).

PR: https://github.com/web-platform-tests/wpt/pull/45753
Details from upstream follow.

mei1127 <shanxing.mei@intel.com> wrote:

Reland "webnn: Migrate Conv2d validation tests to WPTs"

This is a reland of commit 93d2ab44dcde13d2b3ba9c408c75877489e03d49

Original change's description:

webnn: Migrate Conv2d validation tests to WPTs

This CL adds WPT tests for Conv2d and removes the unit tests
MLGraphBuilderTest.Conv2dTest, MLGraphTestMojo.Conv2dTest and
MLGraphTest.Conv2dTest.

Bug: 327337526, 328026885
Change-Id: I2328bd2c9f9056d8cef562f45d479606545e41ba
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/5413394
Reviewed-by: Austin Sullivan \<asully@chromium.org>
Reviewed-by: ningxin hu \<ningxin.hu@intel.com>
Commit-Queue: Shanxing Mei \<shanxing.mei@intel.com>
Cr-Commit-Position: refs/heads/main@{#1287852}

Bug: 327337526, 328026885
Change-Id: If067dbd8d825163a67907ea1995ea9ff1043904f

Reviewed-on: https://chromium-review.googlesource.com/5460412
WPT-Export-Revision: 7bbae929ed2b76dac6338a66346da0e01a7e1809

CI Results

Ran 9 Firefox configurations based on mozilla-central, and Firefox, Chrome, and Safari on GitHub CI

Total 2 tests and 53 subtests

Status Summary

Firefox

OK : 2
FAIL: 106

Chrome

OK : 2
PASS: 40
FAIL: 66

Safari

OK : 2
FAIL: 106

Links

Gecko CI (Treeherder)
GitHub PR Head
GitHub PR Base

Details

New Tests That Don't Pass

  • /webnn/validation_tests/conv2d.https.any.html [wpt.fyi]: OK [GitHub], SKIP [Gecko-android-em-7.0-x86_64-lite-qr-opt-geckoview, Gecko-android-em-7.0-x86_64-qr-debug-geckoview, Gecko-android-em-7.0-x86_64-qr-opt-geckoview, Gecko-linux1804-64-qr-debug, Gecko-linux1804-64-qr-opt, Gecko-windows11-32-2009-qr-debug, Gecko-windows11-32-2009-qr-opt, Gecko-windows11-64-2009-qr-debug, Gecko-windows11-64-2009-qr-opt] (Chrome: OK, Safari: OK)
    • assert_not_equals(navigator.ml, undefined, "ml property is defined on navigator"): FAIL (Chrome: PASS, Safari: FAIL)
    • [conv2d] throw if input is from another builder: FAIL (Chrome: PASS, Safari: FAIL)
    • [conv2d] throw if filter is from another builder: FAIL (Chrome: PASS, Safari: FAIL)
    • [conv2d] throw if bias option is from another builder: FAIL (Chrome: PASS, Safari: FAIL)
    • [conv2d] throw if activation option is from another builder: FAIL (Chrome: PASS, Safari: FAIL)
    • [conv2d] Test with default options.: FAIL (Chrome: PASS, Safari: FAIL)
    • [conv2d] Test with padding.: FAIL (Chrome: PASS, Safari: FAIL)
    • [conv2d] Test with strides and padding.: FAIL (Chrome: PASS, Safari: FAIL)
    • [conv2d] Test with strides and asymmetric padding.: FAIL (Chrome: PASS, Safari: FAIL)
    • [conv2d] Test depthwise conv2d by setting groups to input channels.: FAIL (Chrome: PASS, Safari: FAIL)
    • [conv2d] Test depthwise conv2d with groups, inputLayout="nhwc" and filterLayout="ihwo".: FAIL (Chrome: PASS, Safari: FAIL)
    • [conv2d] Test with dilations, inputLayout="nhwc" and filterLayout="ihwo".: FAIL (Chrome: PASS, Safari: FAIL)
    • [conv2d] Test with inputLayout="nchw" and filterLayout="oihw".: FAIL (Chrome: PASS, Safari: FAIL)
    • [conv2d] Test with inputLayout="nchw" and filterLayout="hwio".: FAIL (Chrome: PASS, Safari: FAIL)
    • [conv2d] Test with inputLayout="nchw" and filterLayout="ohwi".: FAIL (Chrome: PASS, Safari: FAIL)
    • [conv2d] Test with inputLayout="nchw" and filterLayout="ihwo".: FAIL (Chrome: PASS, Safari: FAIL)
    • [conv2d] Test with inputLayout="nhwc" and filterLayout="oihw".: FAIL (Chrome: PASS, Safari: FAIL)
    • [conv2d] Test with inputLayout="nhwc" and filterLayout="hwio".: FAIL (Chrome: PASS, Safari: FAIL)
    • [conv2d] Test with inputLayout="nhwc" and filterLayout="ohwi".: FAIL (Chrome: PASS, Safari: FAIL)
    • [conv2d] Test with inputLayout="nhwc" and filterLayout="ihwo".: FAIL (Chrome: PASS, Safari: FAIL)
    • [conv2d] Throw if the input is not a 4-D tensor.: FAIL (Chrome: FAIL, Safari: FAIL)
    • [conv2d] Throw if the filter is not a 4-D tensor.: FAIL (Chrome: FAIL, Safari: FAIL)
    • [conv2d] Throw if the filter data type doesn't match the input data type.: FAIL (Chrome: FAIL, Safari: FAIL)
    • [conv2d] Throw if the length of padding is not 4.: FAIL (Chrome: FAIL, Safari: FAIL)
    • [conv2d] Throw if the length of strides is not 2.: FAIL (Chrome: FAIL, Safari: FAIL)
    • [conv2d] Throw if strideHeight is smaller than 1.: FAIL (Chrome: FAIL, Safari: FAIL)
    • [conv2d] Throw if strideWidth is smaller than 1.: FAIL (Chrome: FAIL, Safari: FAIL)
    • [conv2d] Throw if the length of dilations is not 2.: FAIL (Chrome: FAIL, Safari: FAIL)
    • [conv2d] Throw if dilationHeight is smaller than 1.: FAIL (Chrome: FAIL, Safari: FAIL)
    • [conv2d] Throw if dilationWidth is smaller than 1.: FAIL (Chrome: FAIL, Safari: FAIL)
    • [conv2d] Throw if inputChannels % groups is not 0.: FAIL (Chrome: FAIL, Safari: FAIL)
    • [conv2d] Throw if inputChannels / groups is not equal to filterInputChannels.: FAIL (Chrome: FAIL, Safari: FAIL)
    • [conv2d] Throw if the groups is smaller than 1.: FAIL (Chrome: FAIL, Safari: FAIL)
    • [conv2d] Throw due to overflow when calculating the effective filter height.: FAIL (Chrome: FAIL, Safari: FAIL)
    • [conv2d] Throw due to overflow when calculating the effective filter width.: FAIL (Chrome: FAIL, Safari: FAIL)
    • [conv2d] Throw due to overflow when dilation height is too large.: FAIL (Chrome: FAIL, Safari: FAIL)
    • [conv2d] Throw due to overflow when dilation width is too large.: FAIL (Chrome: FAIL, Safari: FAIL)
    • [conv2d] Throw due to underflow when calculating the output height.: FAIL (Chrome: FAIL, Safari: FAIL)
    • [conv2d] Throw due to underflow when calculating the output width.: FAIL (Chrome: FAIL, Safari: FAIL)
    • [conv2d] Throw if the bias is not a 1-D tensor.: FAIL (Chrome: FAIL, Safari: FAIL)
    • [conv2d] Throw if the bias shape is not equal to [output_channels] with filterLayout="oihw".: FAIL (Chrome: FAIL, Safari: FAIL)
    • [conv2d] Throw if the bias shape is not equal to [output_channels] with filterLayout="hwio".: FAIL (Chrome: FAIL, Safari: FAIL)
    • [conv2d] Throw if the bias shape is not equal to [output_channels] with filterLayout="ohwi".: FAIL (Chrome: FAIL, Safari: FAIL)
    • [conv2d] Throw if the bias shape is not equal to [output_channels] with filterLayout="ihwo".: FAIL (Chrome: FAIL, Safari: FAIL)
    • [conv2d] Throw if the bias data type doesn't match input data type.: FAIL (Chrome: FAIL, Safari: FAIL)
    • [conv2d] Throw if inputChannels / groups is not equal to filterInputChannels with inputLayout="nchw" and filterLayout="oihw".: FAIL (Chrome: FAIL, Safari: FAIL)
    • [conv2d] Throw if inputChannels / groups is not equal to filterInputChannels with inputLayout="nchw" and filterLayout="hwio".: FAIL (Chrome: FAIL, Safari: FAIL)
    • [conv2d] Throw if inputChannels / groups is not equal to filterInputChannels with inputLayout="nchw" and filterLayout="ohwi".: FAIL (Chrome: FAIL, Safari: FAIL)
    • [conv2d] Throw if inputChannels / groups is not equal to filterInputChannels with inputLayout="nchw" and filterLayout="ihwo".: FAIL (Chrome: FAIL, Safari: FAIL)
    • [conv2d] Throw if inputChannels / groups is not equal to filterInputChannels with inputLayout="nhwc" and filterLayout="oihw".: FAIL (Chrome: FAIL, Safari: FAIL)
    • [conv2d] Throw if inputChannels / groups is not equal to filterInputChannels with inputLayout="nhwc" and filterLayout="hwio".: FAIL (Chrome: FAIL, Safari: FAIL)
    • [conv2d] Throw if inputChannels / groups is not equal to filterInputChannels with inputLayout="nhwc" and filterLayout="ohwi".: FAIL (Chrome: FAIL, Safari: FAIL)
    • [conv2d] Throw if inputChannels / groups is not equal to filterInputChannels with inputLayout="nhwc" and filterLayout="ihwo".: FAIL (Chrome: FAIL, Safari: FAIL)
  • /webnn/validation_tests/conv2d.https.any.worker.html [wpt.fyi]: OK [GitHub], SKIP [Gecko-android-em-7.0-x86_64-lite-qr-opt-geckoview, Gecko-android-em-7.0-x86_64-qr-debug-geckoview, Gecko-android-em-7.0-x86_64-qr-opt-geckoview, Gecko-linux1804-64-qr-debug, Gecko-linux1804-64-qr-opt, Gecko-windows11-32-2009-qr-debug, Gecko-windows11-32-2009-qr-opt, Gecko-windows11-64-2009-qr-debug, Gecko-windows11-64-2009-qr-opt] (Chrome: OK, Safari: OK)
    • assert_not_equals(navigator.ml, undefined, "ml property is defined on navigator"): FAIL (Chrome: PASS, Safari: FAIL)
    • [conv2d] throw if input is from another builder: FAIL (Chrome: PASS, Safari: FAIL)
    • [conv2d] throw if filter is from another builder: FAIL (Chrome: PASS, Safari: FAIL)
    • [conv2d] throw if bias option is from another builder: FAIL (Chrome: PASS, Safari: FAIL)
    • [conv2d] throw if activation option is from another builder: FAIL (Chrome: PASS, Safari: FAIL)
    • [conv2d] Test with default options.: FAIL (Chrome: PASS, Safari: FAIL)
    • [conv2d] Test with padding.: FAIL (Chrome: PASS, Safari: FAIL)
    • [conv2d] Test with strides and padding.: FAIL (Chrome: PASS, Safari: FAIL)
    • [conv2d] Test with strides and asymmetric padding.: FAIL (Chrome: PASS, Safari: FAIL)
    • [conv2d] Test depthwise conv2d by setting groups to input channels.: FAIL (Chrome: PASS, Safari: FAIL)
    • [conv2d] Test depthwise conv2d with groups, inputLayout="nhwc" and filterLayout="ihwo".: FAIL (Chrome: PASS, Safari: FAIL)
    • [conv2d] Test with dilations, inputLayout="nhwc" and filterLayout="ihwo".: FAIL (Chrome: PASS, Safari: FAIL)
    • [conv2d] Test with inputLayout="nchw" and filterLayout="oihw".: FAIL (Chrome: PASS, Safari: FAIL)
    • [conv2d] Test with inputLayout="nchw" and filterLayout="hwio".: FAIL (Chrome: PASS, Safari: FAIL)
    • [conv2d] Test with inputLayout="nchw" and filterLayout="ohwi".: FAIL (Chrome: PASS, Safari: FAIL)
    • [conv2d] Test with inputLayout="nchw" and filterLayout="ihwo".: FAIL (Chrome: PASS, Safari: FAIL)
    • [conv2d] Test with inputLayout="nhwc" and filterLayout="oihw".: FAIL (Chrome: PASS, Safari: FAIL)
    • [conv2d] Test with inputLayout="nhwc" and filterLayout="hwio".: FAIL (Chrome: PASS, Safari: FAIL)
    • [conv2d] Test with inputLayout="nhwc" and filterLayout="ohwi".: FAIL (Chrome: PASS, Safari: FAIL)
    • [conv2d] Test with inputLayout="nhwc" and filterLayout="ihwo".: FAIL (Chrome: PASS, Safari: FAIL)
    • [conv2d] Throw if the input is not a 4-D tensor.: FAIL (Chrome: FAIL, Safari: FAIL)
    • [conv2d] Throw if the filter is not a 4-D tensor.: FAIL (Chrome: FAIL, Safari: FAIL)
    • [conv2d] Throw if the filter data type doesn't match the input data type.: FAIL (Chrome: FAIL, Safari: FAIL)
    • [conv2d] Throw if the length of padding is not 4.: FAIL (Chrome: FAIL, Safari: FAIL)
    • [conv2d] Throw if the length of strides is not 2.: FAIL (Chrome: FAIL, Safari: FAIL)
    • [conv2d] Throw if strideHeight is smaller than 1.: FAIL (Chrome: FAIL, Safari: FAIL)
    • [conv2d] Throw if strideWidth is smaller than 1.: FAIL (Chrome: FAIL, Safari: FAIL)
    • [conv2d] Throw if the length of dilations is not 2.: FAIL (Chrome: FAIL, Safari: FAIL)
    • [conv2d] Throw if dilationHeight is smaller than 1.: FAIL (Chrome: FAIL, Safari: FAIL)
    • [conv2d] Throw if dilationWidth is smaller than 1.: FAIL (Chrome: FAIL, Safari: FAIL)
    • [conv2d] Throw if inputChannels % groups is not 0.: FAIL (Chrome: FAIL, Safari: FAIL)
    • [conv2d] Throw if inputChannels / groups is not equal to filterInputChannels.: FAIL (Chrome: FAIL, Safari: FAIL)
    • [conv2d] Throw if the groups is smaller than 1.: FAIL (Chrome: FAIL, Safari: FAIL)
    • [conv2d] Throw due to overflow when calculating the effective filter height.: FAIL (Chrome: FAIL, Safari: FAIL)
    • [conv2d] Throw due to overflow when calculating the effective filter width.: FAIL (Chrome: FAIL, Safari: FAIL)
    • [conv2d] Throw due to overflow when dilation height is too large.: FAIL (Chrome: FAIL, Safari: FAIL)
    • [conv2d] Throw due to overflow when dilation width is too large.: FAIL (Chrome: FAIL, Safari: FAIL)
    • [conv2d] Throw due to underflow when calculating the output height.: FAIL (Chrome: FAIL, Safari: FAIL)
    • [conv2d] Throw due to underflow when calculating the output width.: FAIL (Chrome: FAIL, Safari: FAIL)
    • [conv2d] Throw if the bias is not a 1-D tensor.: FAIL (Chrome: FAIL, Safari: FAIL)
    • [conv2d] Throw if the bias shape is not equal to [output_channels] with filterLayout="oihw".: FAIL (Chrome: FAIL, Safari: FAIL)
    • [conv2d] Throw if the bias shape is not equal to [output_channels] with filterLayout="hwio".: FAIL (Chrome: FAIL, Safari: FAIL)
    • [conv2d] Throw if the bias shape is not equal to [output_channels] with filterLayout="ohwi".: FAIL (Chrome: FAIL, Safari: FAIL)
    • [conv2d] Throw if the bias shape is not equal to [output_channels] with filterLayout="ihwo".: FAIL (Chrome: FAIL, Safari: FAIL)
    • [conv2d] Throw if the bias data type doesn't match input data type.: FAIL (Chrome: FAIL, Safari: FAIL)
    • [conv2d] Throw if inputChannels / groups is not equal to filterInputChannels with inputLayout="nchw" and filterLayout="oihw".: FAIL (Chrome: FAIL, Safari: FAIL)
    • [conv2d] Throw if inputChannels / groups is not equal to filterInputChannels with inputLayout="nchw" and filterLayout="hwio".: FAIL (Chrome: FAIL, Safari: FAIL)
    • [conv2d] Throw if inputChannels / groups is not equal to filterInputChannels with inputLayout="nchw" and filterLayout="ohwi".: FAIL (Chrome: FAIL, Safari: FAIL)
    • [conv2d] Throw if inputChannels / groups is not equal to filterInputChannels with inputLayout="nchw" and filterLayout="ihwo".: FAIL (Chrome: FAIL, Safari: FAIL)
    • [conv2d] Throw if inputChannels / groups is not equal to filterInputChannels with inputLayout="nhwc" and filterLayout="oihw".: FAIL (Chrome: FAIL, Safari: FAIL)
    • [conv2d] Throw if inputChannels / groups is not equal to filterInputChannels with inputLayout="nhwc" and filterLayout="hwio".: FAIL (Chrome: FAIL, Safari: FAIL)
    • [conv2d] Throw if inputChannels / groups is not equal to filterInputChannels with inputLayout="nhwc" and filterLayout="ohwi".: FAIL (Chrome: FAIL, Safari: FAIL)
    • [conv2d] Throw if inputChannels / groups is not equal to filterInputChannels with inputLayout="nhwc" and filterLayout="ihwo".: FAIL (Chrome: FAIL, Safari: FAIL)

Tests Disabled in Gecko Infrastructure

  • /webnn/validation_tests/conv2d.https.any.html [wpt.fyi]: OK [GitHub], SKIP [Gecko-android-em-7.0-x86_64-lite-qr-opt-geckoview, Gecko-android-em-7.0-x86_64-qr-debug-geckoview, Gecko-android-em-7.0-x86_64-qr-opt-geckoview, Gecko-linux1804-64-qr-debug, Gecko-linux1804-64-qr-opt, Gecko-windows11-32-2009-qr-debug, Gecko-windows11-32-2009-qr-opt, Gecko-windows11-64-2009-qr-debug, Gecko-windows11-64-2009-qr-opt] (Chrome: OK, Safari: OK)
  • /webnn/validation_tests/conv2d.https.any.worker.html [wpt.fyi]: OK [GitHub], SKIP [Gecko-android-em-7.0-x86_64-lite-qr-opt-geckoview, Gecko-android-em-7.0-x86_64-qr-debug-geckoview, Gecko-android-em-7.0-x86_64-qr-opt-geckoview, Gecko-linux1804-64-qr-debug, Gecko-linux1804-64-qr-opt, Gecko-windows11-32-2009-qr-debug, Gecko-windows11-32-2009-qr-opt, Gecko-windows11-64-2009-qr-debug, Gecko-windows11-64-2009-qr-opt] (Chrome: OK, Safari: OK)
Pushed by wptsync@mozilla.com: https://hg.mozilla.org/integration/autoland/rev/660733cf6a01 [wpt PR 45753] - Reland "webnn: Migrate Conv2d validation tests to WPTs", a=testonly

https://hg.mozilla.org/mozilla-central/rev/660733cf6a01

Status: NEW → RESOLVED
Closed: 2 years ago
status-firefox127: --- → fixed
Resolution: --- → FIXED
Target Milestone: --- → 127 Branch
You need to log in before you can comment on or make changes to this bug.