Closed Bug 1889113 Opened 1 year ago Closed 1 year ago

[wpt-sync] Sync PR 45466 - webnn: Migrate Conv2d validation tests to WPTs

Categories

(Testing :: web-platform-tests, task, P4)

Product:
Testing
Component:
web-platform-tests
Type:
task

Tracking

(firefox127 fixed)

Status:
RESOLVED FIXED
Milestone:
127 Branch
Tracking Flags:
Tracking Status
firefox127 --- fixed

People

(Reporter: wpt-sync, Unassigned)

References

(
URL
)

Details

(Whiteboard: [wptsync downstream])

Sync web-platform-tests PR 45466 into mozilla-central (this bug is closed when the sync is complete).

PR: https://github.com/web-platform-tests/wpt/pull/45466
Details from upstream follow.

mei1127 <shanxing.mei@intel.com> wrote:

webnn: Migrate Conv2d validation tests to WPTs

This CL adds WPT tests for Conv2d and removes the unit tests MLGraphBuilderTest.Conv2dTest, MLGraphTestMojo.Conv2dTest
and MLGraphTest.Conv2dTest.

Change-Id: I2328bd2c9f9056d8cef562f45d479606545e41ba

Reviewed-on: https://chromium-review.googlesource.com/5413394
WPT-Export-Revision: 6a5d4dec783b9954ed4eaeedf4c7f7b6863c5b85

Whiteboard: [wptsync downstream] → [wptsync downstream error]
Whiteboard: [wptsync downstream error] → [wptsync downstream]

CI Results

Ran 9 Firefox configurations based on mozilla-central, and Firefox, Chrome, and Safari on GitHub CI

Total 2 tests and 54 subtests

Status Summary

Firefox

OK : 2
FAIL: 108

Chrome

OK : 2
PASS: 40
FAIL: 68

Safari

OK : 2
FAIL: 108

Links

Gecko CI (Treeherder)
GitHub PR Head
GitHub PR Base

Details

New Tests That Don't Pass

  • /webnn/validation_tests/conv2d.https.any.worker.html [wpt.fyi]: OK [GitHub], SKIP [Gecko-android-em-7.0-x86_64-lite-qr-opt-geckoview, Gecko-android-em-7.0-x86_64-qr-debug-geckoview, Gecko-android-em-7.0-x86_64-qr-opt-geckoview, Gecko-linux1804-64-qr-debug, Gecko-linux1804-64-qr-opt, Gecko-windows11-32-2009-qr-debug, Gecko-windows11-32-2009-qr-opt, Gecko-windows11-64-2009-qr-debug, Gecko-windows11-64-2009-qr-opt] (Chrome: OK, Safari: OK)
    • assert_not_equals(navigator.ml, undefined, "ml property is defined on navigator"): FAIL (Chrome: PASS, Safari: FAIL)
    • [conv2d] throw if input is from another builder: FAIL (Chrome: PASS, Safari: FAIL)
    • [conv2d] throw if filter is from another builder: FAIL (Chrome: PASS, Safari: FAIL)
    • [conv2d] throw if bias option is from another builder: FAIL (Chrome: PASS, Safari: FAIL)
    • [conv2d] throw if activation option is from another builder: FAIL (Chrome: PASS, Safari: FAIL)
    • [conv2d] Test with default options.: FAIL (Chrome: PASS, Safari: FAIL)
    • [conv2d] Test with padding.: FAIL (Chrome: PASS, Safari: FAIL)
    • [conv2d] Test with strides and padding.: FAIL (Chrome: PASS, Safari: FAIL)
    • [conv2d] Test with strides and asymmetric padding.: FAIL (Chrome: PASS, Safari: FAIL)
    • [conv2d] Test depthwise conv2d by setting groups to input channels.: FAIL (Chrome: PASS, Safari: FAIL)
    • [conv2d] Test depthwise conv2d with groups, inputLayout="nhwc" and filterLayout="ihwo".: FAIL (Chrome: PASS, Safari: FAIL)
    • [conv2d] Test with dilations, inputLayout="nhwc" and filterLayout="ihwo".: FAIL (Chrome: PASS, Safari: FAIL)
    • [conv2d] Test with inputLayout="nchw" and filterLayout="oihw".: FAIL (Chrome: PASS, Safari: FAIL)
    • [conv2d] Test with inputLayout="nchw" and filterLayout="hwio".: FAIL (Chrome: PASS, Safari: FAIL)
    • [conv2d] Test with inputLayout="nchw" and filterLayout="ohwi".: FAIL (Chrome: PASS, Safari: FAIL)
    • [conv2d] Test with inputLayout="nchw" and filterLayout="ihwo".: FAIL (Chrome: PASS, Safari: FAIL)
    • [conv2d] Test with inputLayout="nhwc" and filterLayout="oihw".: FAIL (Chrome: PASS, Safari: FAIL)
    • [conv2d] Test with inputLayout="nhwc" and filterLayout="hwio".: FAIL (Chrome: PASS, Safari: FAIL)
    • [conv2d] Test with inputLayout="nhwc" and filterLayout="ohwi".: FAIL (Chrome: PASS, Safari: FAIL)
    • [conv2d] Test with inputLayout="nhwc" and filterLayout="ihwo".: FAIL (Chrome: PASS, Safari: FAIL)
    • [conv2d] Throw if the output operand's number of elements is too large.: FAIL (Chrome: FAIL, Safari: FAIL)
    • [conv2d] Throw if the input is not a 4-D tensor.: FAIL (Chrome: FAIL, Safari: FAIL)
    • [conv2d] Throw if the filter is not a 4-D tensor.: FAIL (Chrome: FAIL, Safari: FAIL)
    • [conv2d] Throw if the filter data type doesn't match the input data type.: FAIL (Chrome: FAIL, Safari: FAIL)
    • [conv2d] Throw if the length of padding is not 4.: FAIL (Chrome: FAIL, Safari: FAIL)
    • [conv2d] Throw if the length of strides is not 2.: FAIL (Chrome: FAIL, Safari: FAIL)
    • [conv2d] Throw if strideHeight is smaller than 1.: FAIL (Chrome: FAIL, Safari: FAIL)
    • [conv2d] Throw if strideWidth is smaller than 1.: FAIL (Chrome: FAIL, Safari: FAIL)
    • [conv2d] Throw if the length of dilations is not 2.: FAIL (Chrome: FAIL, Safari: FAIL)
    • [conv2d] Throw if dilationHeight is smaller than 1.: FAIL (Chrome: FAIL, Safari: FAIL)
    • [conv2d] Throw if dilationWidth is smaller than 1.: FAIL (Chrome: FAIL, Safari: FAIL)
    • [conv2d] Throw if inputChannels % groups is not 0.: FAIL (Chrome: FAIL, Safari: FAIL)
    • [conv2d] Throw if inputChannels / groups is not equal to filterInputChannels.: FAIL (Chrome: FAIL, Safari: FAIL)
    • [conv2d] Throw if the groups is smaller than 1.: FAIL (Chrome: FAIL, Safari: FAIL)
    • [conv2d] Throw due to overflow when calculating the effective filter height.: FAIL (Chrome: FAIL, Safari: FAIL)
    • [conv2d] Throw due to overflow when calculating the effective filter width.: FAIL (Chrome: FAIL, Safari: FAIL)
    • [conv2d] Throw due to overflow when dilation height is too large.: FAIL (Chrome: FAIL, Safari: FAIL)
    • [conv2d] Throw due to overflow when dilation width is too large.: FAIL (Chrome: FAIL, Safari: FAIL)
    • [conv2d] Throw due to underflow when calculating the output height.: FAIL (Chrome: FAIL, Safari: FAIL)
    • [conv2d] Throw due to underflow when calculating the output width.: FAIL (Chrome: FAIL, Safari: FAIL)
    • [conv2d] Throw if the bias is not a 1-D tensor.: FAIL (Chrome: FAIL, Safari: FAIL)
    • [conv2d] Throw if the bias shape is not equal to [output_channels] with filterLayout="oihw".: FAIL (Chrome: FAIL, Safari: FAIL)
    • [conv2d] Throw if the bias shape is not equal to [output_channels] with filterLayout="hwio".: FAIL (Chrome: FAIL, Safari: FAIL)
    • [conv2d] Throw if the bias shape is not equal to [output_channels] with filterLayout="ohwi".: FAIL (Chrome: FAIL, Safari: FAIL)
    • [conv2d] Throw if the bias shape is not equal to [output_channels] with filterLayout="ihwo".: FAIL (Chrome: FAIL, Safari: FAIL)
    • [conv2d] Throw if the bias data type doesn't match input data type.: FAIL (Chrome: FAIL, Safari: FAIL)
    • [conv2d] Throw if inputChannels / groups is not equal to filterInputChannels with inputLayout="nchw" and filterLayout="oihw".: FAIL (Chrome: FAIL, Safari: FAIL)
    • [conv2d] Throw if inputChannels / groups is not equal to filterInputChannels with inputLayout="nchw" and filterLayout="hwio".: FAIL (Chrome: FAIL, Safari: FAIL)
    • [conv2d] Throw if inputChannels / groups is not equal to filterInputChannels with inputLayout="nchw" and filterLayout="ohwi".: FAIL (Chrome: FAIL, Safari: FAIL)
    • [conv2d] Throw if inputChannels / groups is not equal to filterInputChannels with inputLayout="nchw" and filterLayout="ihwo".: FAIL (Chrome: FAIL, Safari: FAIL)
    • [conv2d] Throw if inputChannels / groups is not equal to filterInputChannels with inputLayout="nhwc" and filterLayout="oihw".: FAIL (Chrome: FAIL, Safari: FAIL)
    • [conv2d] Throw if inputChannels / groups is not equal to filterInputChannels with inputLayout="nhwc" and filterLayout="hwio".: FAIL (Chrome: FAIL, Safari: FAIL)
    • [conv2d] Throw if inputChannels / groups is not equal to filterInputChannels with inputLayout="nhwc" and filterLayout="ohwi".: FAIL (Chrome: FAIL, Safari: FAIL)
    • [conv2d] Throw if inputChannels / groups is not equal to filterInputChannels with inputLayout="nhwc" and filterLayout="ihwo".: FAIL (Chrome: FAIL, Safari: FAIL)
  • /webnn/validation_tests/conv2d.https.any.html [wpt.fyi]: OK [GitHub], SKIP [Gecko-android-em-7.0-x86_64-lite-qr-opt-geckoview, Gecko-android-em-7.0-x86_64-qr-debug-geckoview, Gecko-android-em-7.0-x86_64-qr-opt-geckoview, Gecko-linux1804-64-qr-debug, Gecko-linux1804-64-qr-opt, Gecko-windows11-32-2009-qr-debug, Gecko-windows11-32-2009-qr-opt, Gecko-windows11-64-2009-qr-debug, Gecko-windows11-64-2009-qr-opt] (Chrome: OK, Safari: OK)
    • assert_not_equals(navigator.ml, undefined, "ml property is defined on navigator"): FAIL (Chrome: PASS, Safari: FAIL)
    • [conv2d] throw if input is from another builder: FAIL (Chrome: PASS, Safari: FAIL)
    • [conv2d] throw if filter is from another builder: FAIL (Chrome: PASS, Safari: FAIL)
    • [conv2d] throw if bias option is from another builder: FAIL (Chrome: PASS, Safari: FAIL)
    • [conv2d] throw if activation option is from another builder: FAIL (Chrome: PASS, Safari: FAIL)
    • [conv2d] Test with default options.: FAIL (Chrome: PASS, Safari: FAIL)
    • [conv2d] Test with padding.: FAIL (Chrome: PASS, Safari: FAIL)
    • [conv2d] Test with strides and padding.: FAIL (Chrome: PASS, Safari: FAIL)
    • [conv2d] Test with strides and asymmetric padding.: FAIL (Chrome: PASS, Safari: FAIL)
    • [conv2d] Test depthwise conv2d by setting groups to input channels.: FAIL (Chrome: PASS, Safari: FAIL)
    • [conv2d] Test depthwise conv2d with groups, inputLayout="nhwc" and filterLayout="ihwo".: FAIL (Chrome: PASS, Safari: FAIL)
    • [conv2d] Test with dilations, inputLayout="nhwc" and filterLayout="ihwo".: FAIL (Chrome: PASS, Safari: FAIL)
    • [conv2d] Test with inputLayout="nchw" and filterLayout="oihw".: FAIL (Chrome: PASS, Safari: FAIL)
    • [conv2d] Test with inputLayout="nchw" and filterLayout="hwio".: FAIL (Chrome: PASS, Safari: FAIL)
    • [conv2d] Test with inputLayout="nchw" and filterLayout="ohwi".: FAIL (Chrome: PASS, Safari: FAIL)
    • [conv2d] Test with inputLayout="nchw" and filterLayout="ihwo".: FAIL (Chrome: PASS, Safari: FAIL)
    • [conv2d] Test with inputLayout="nhwc" and filterLayout="oihw".: FAIL (Chrome: PASS, Safari: FAIL)
    • [conv2d] Test with inputLayout="nhwc" and filterLayout="hwio".: FAIL (Chrome: PASS, Safari: FAIL)
    • [conv2d] Test with inputLayout="nhwc" and filterLayout="ohwi".: FAIL (Chrome: PASS, Safari: FAIL)
    • [conv2d] Test with inputLayout="nhwc" and filterLayout="ihwo".: FAIL (Chrome: PASS, Safari: FAIL)
    • [conv2d] Throw if the output operand's number of elements is too large.: FAIL (Chrome: FAIL, Safari: FAIL)
    • [conv2d] Throw if the input is not a 4-D tensor.: FAIL (Chrome: FAIL, Safari: FAIL)
    • [conv2d] Throw if the filter is not a 4-D tensor.: FAIL (Chrome: FAIL, Safari: FAIL)
    • [conv2d] Throw if the filter data type doesn't match the input data type.: FAIL (Chrome: FAIL, Safari: FAIL)
    • [conv2d] Throw if the length of padding is not 4.: FAIL (Chrome: FAIL, Safari: FAIL)
    • [conv2d] Throw if the length of strides is not 2.: FAIL (Chrome: FAIL, Safari: FAIL)
    • [conv2d] Throw if strideHeight is smaller than 1.: FAIL (Chrome: FAIL, Safari: FAIL)
    • [conv2d] Throw if strideWidth is smaller than 1.: FAIL (Chrome: FAIL, Safari: FAIL)
    • [conv2d] Throw if the length of dilations is not 2.: FAIL (Chrome: FAIL, Safari: FAIL)
    • [conv2d] Throw if dilationHeight is smaller than 1.: FAIL (Chrome: FAIL, Safari: FAIL)
    • [conv2d] Throw if dilationWidth is smaller than 1.: FAIL (Chrome: FAIL, Safari: FAIL)
    • [conv2d] Throw if inputChannels % groups is not 0.: FAIL (Chrome: FAIL, Safari: FAIL)
    • [conv2d] Throw if inputChannels / groups is not equal to filterInputChannels.: FAIL (Chrome: FAIL, Safari: FAIL)
    • [conv2d] Throw if the groups is smaller than 1.: FAIL (Chrome: FAIL, Safari: FAIL)
    • [conv2d] Throw due to overflow when calculating the effective filter height.: FAIL (Chrome: FAIL, Safari: FAIL)
    • [conv2d] Throw due to overflow when calculating the effective filter width.: FAIL (Chrome: FAIL, Safari: FAIL)
    • [conv2d] Throw due to overflow when dilation height is too large.: FAIL (Chrome: FAIL, Safari: FAIL)
    • [conv2d] Throw due to overflow when dilation width is too large.: FAIL (Chrome: FAIL, Safari: FAIL)
    • [conv2d] Throw due to underflow when calculating the output height.: FAIL (Chrome: FAIL, Safari: FAIL)
    • [conv2d] Throw due to underflow when calculating the output width.: FAIL (Chrome: FAIL, Safari: FAIL)
    • [conv2d] Throw if the bias is not a 1-D tensor.: FAIL (Chrome: FAIL, Safari: FAIL)
    • [conv2d] Throw if the bias shape is not equal to [output_channels] with filterLayout="oihw".: FAIL (Chrome: FAIL, Safari: FAIL)
    • [conv2d] Throw if the bias shape is not equal to [output_channels] with filterLayout="hwio".: FAIL (Chrome: FAIL, Safari: FAIL)
    • [conv2d] Throw if the bias shape is not equal to [output_channels] with filterLayout="ohwi".: FAIL (Chrome: FAIL, Safari: FAIL)
    • [conv2d] Throw if the bias shape is not equal to [output_channels] with filterLayout="ihwo".: FAIL (Chrome: FAIL, Safari: FAIL)
    • [conv2d] Throw if the bias data type doesn't match input data type.: FAIL (Chrome: FAIL, Safari: FAIL)
    • [conv2d] Throw if inputChannels / groups is not equal to filterInputChannels with inputLayout="nchw" and filterLayout="oihw".: FAIL (Chrome: FAIL, Safari: FAIL)
    • [conv2d] Throw if inputChannels / groups is not equal to filterInputChannels with inputLayout="nchw" and filterLayout="hwio".: FAIL (Chrome: FAIL, Safari: FAIL)
    • [conv2d] Throw if inputChannels / groups is not equal to filterInputChannels with inputLayout="nchw" and filterLayout="ohwi".: FAIL (Chrome: FAIL, Safari: FAIL)
    • [conv2d] Throw if inputChannels / groups is not equal to filterInputChannels with inputLayout="nchw" and filterLayout="ihwo".: FAIL (Chrome: FAIL, Safari: FAIL)
    • [conv2d] Throw if inputChannels / groups is not equal to filterInputChannels with inputLayout="nhwc" and filterLayout="oihw".: FAIL (Chrome: FAIL, Safari: FAIL)
    • [conv2d] Throw if inputChannels / groups is not equal to filterInputChannels with inputLayout="nhwc" and filterLayout="hwio".: FAIL (Chrome: FAIL, Safari: FAIL)
    • [conv2d] Throw if inputChannels / groups is not equal to filterInputChannels with inputLayout="nhwc" and filterLayout="ohwi".: FAIL (Chrome: FAIL, Safari: FAIL)
    • [conv2d] Throw if inputChannels / groups is not equal to filterInputChannels with inputLayout="nhwc" and filterLayout="ihwo".: FAIL (Chrome: FAIL, Safari: FAIL)

Tests Disabled in Gecko Infrastructure

  • /webnn/validation_tests/conv2d.https.any.worker.html [wpt.fyi]: OK [GitHub], SKIP [Gecko-android-em-7.0-x86_64-lite-qr-opt-geckoview, Gecko-android-em-7.0-x86_64-qr-debug-geckoview, Gecko-android-em-7.0-x86_64-qr-opt-geckoview, Gecko-linux1804-64-qr-debug, Gecko-linux1804-64-qr-opt, Gecko-windows11-32-2009-qr-debug, Gecko-windows11-32-2009-qr-opt, Gecko-windows11-64-2009-qr-debug, Gecko-windows11-64-2009-qr-opt] (Chrome: OK, Safari: OK)
  • /webnn/validation_tests/conv2d.https.any.html [wpt.fyi]: OK [GitHub], SKIP [Gecko-android-em-7.0-x86_64-lite-qr-opt-geckoview, Gecko-android-em-7.0-x86_64-qr-debug-geckoview, Gecko-android-em-7.0-x86_64-qr-opt-geckoview, Gecko-linux1804-64-qr-debug, Gecko-linux1804-64-qr-opt, Gecko-windows11-32-2009-qr-debug, Gecko-windows11-32-2009-qr-opt, Gecko-windows11-64-2009-qr-debug, Gecko-windows11-64-2009-qr-opt] (Chrome: OK, Safari: OK)
Pushed by wptsync@mozilla.com: https://hg.mozilla.org/integration/autoland/rev/8e0f0cf866e1 [wpt PR 45466] - webnn: Migrate Conv2d validation tests to WPTs, a=testonly

https://hg.mozilla.org/mozilla-central/rev/8e0f0cf866e1

Status: NEW → RESOLVED
Closed: 1 year ago
status-firefox127: --- → fixed
Resolution: --- → FIXED
Target Milestone: --- → 127 Branch
You need to log in before you can comment on or make changes to this bug.