Closed Bug 1592955 Opened 5 years ago Closed 4 years ago

[wpt-sync] Sync PR 20017 - Add support for crashtests

Categories

(Testing :: web-platform-tests, task, P4)

task

Tracking

(firefox73 fixed)

RESOLVED FIXED
mozilla73
Tracking Status
firefox73 --- fixed

People

(Reporter: mozilla.org, Unassigned)

References

()

Details

(Whiteboard: [wptsync downstream])

Sync web-platform-tests PR 20017 into mozilla-central (this bug is closed when the sync is complete).

PR: https://github.com/web-platform-tests/wpt/pull/20017
Details from upstream follow.

James Graham <james@hoppipolla.co.uk> wrote:

Add support for crashtests

See https://github.com/web-platform-tests/rfcs/pull/33

Whiteboard: [wptsync downstream] → [wptsync downstream error]
Whiteboard: [wptsync downstream error] → [wptsync downstream]

CI Results

Ran 13 Firefox configurations based on mozilla-central, and Firefox, Chrome, and Safari on GitHub CI

Total 10 tests and 1 subtests

Status Summary

Firefox

OK : 5[[u'Gecko-android-em-7.0-x86_64-debug-geckoview', u'Gecko-android-em-7.0-x86_64-opt-geckoview', u'Gecko-linux64-asan-opt', u'Gecko-linux64-debug', u'Gecko-linux64-opt', u'Gecko-linux64-qr-debug', u'Gecko-linux64-qr-opt', u'Gecko-windows10-64-debug', u'Gecko-windows10-64-opt', u'Gecko-windows10-64-qr-debug', u'Gecko-windows10-64-qr-opt', u'Gecko-windows7-32-debug', u'Gecko-windows7-32-opt']] 6[['GitHub']]
PASS: 7[[u'Gecko-windows10-64-qr-debug', u'Gecko-windows10-64-qr-opt']] 8[[u'Gecko-android-em-7.0-x86_64-debug-geckoview', u'Gecko-android-em-7.0-x86_64-opt-geckoview', u'Gecko-linux64-asan-opt', u'Gecko-linux64-debug', u'Gecko-linux64-opt', u'Gecko-linux64-qr-debug', u'Gecko-linux64-qr-opt', u'Gecko-windows10-64-debug', u'Gecko-windows10-64-opt', u'Gecko-windows7-32-debug', u'Gecko-windows7-32-opt', 'GitHub']]
FAIL: 1[[u'Gecko-android-em-7.0-x86_64-debug-geckoview', u'Gecko-android-em-7.0-x86_64-opt-geckoview', u'Gecko-linux64-asan-opt', u'Gecko-linux64-debug', u'Gecko-linux64-opt', u'Gecko-linux64-qr-debug', u'Gecko-linux64-qr-opt', u'Gecko-windows10-64-debug', u'Gecko-windows10-64-opt', u'Gecko-windows7-32-debug', u'Gecko-windows7-32-opt']] 2[[u'Gecko-windows10-64-qr-debug', u'Gecko-windows10-64-qr-opt']] 11[['GitHub']]

Chrome

OK : 6
PASS: 19

Safari

OK : 6
PASS: 17
FAIL: 2

Links

Gecko CI (Treeherder)
GitHub PR Head
GitHub PR Base

Details

Firefox-only Failures

/payment-request/payment-request-constructor-thcrash.https.html
Don't crash if PaymentDetailsInit.total.label is an abusive length: FAIL
Don't crash if the request id has an abusive length: FAIL
Don't crash if the PaymentShippingOptions.amount.value is an abusive length: FAIL
Don't crash if details.displayItems has an abusive number of items: FAIL
Don't crash if PaymentMethodData.supportedMethods is an abusive length: FAIL
Don't crash if PaymentItem.label is an abusive length: FAIL
Don't crash if there is an abusive number of payment methods in the methodData sequence: FAIL
Don't crash if details.shippingOptions has an abusive number of items: FAIL
Don't crash if total.amount.value is an abusive length: FAIL
Don't crash if PaymentShippingOptions.label is an abusive length: FAIL

New Tests That Don't Pass

/css/css-typed-om/set-var-reference-thcrash.html
Do not crash when referencing a variable with CSSVariableReferenceValue: FAIL (Chrome: PASS, Safari: FAIL)
/payment-request/payment-request-constructor-thcrash.https.html: OK[GitHub], SKIP[Gecko-android-em-7.0-x86_64-debug-geckoview,Gecko-android-em-7.0-x86_64-opt-geckoview,Gecko-linux64-asan-opt,Gecko-linux64-debug,Gecko-linux64-opt,Gecko-linux64-qr-debug,Gecko-linux64-qr-opt,Gecko-windows10-64-debug,Gecko-windows10-64-opt,Gecko-windows10-64-qr-debug,Gecko-windows10-64-qr-opt,Gecko-windows7-32-debug,Gecko-windows7-32-opt] (Chrome: OK, Safari: OK)
Don't crash if PaymentDetailsInit.total.label is an abusive length: FAIL (Chrome: PASS, Safari: PASS)
Don't crash if the request id has an abusive length: FAIL (Chrome: PASS, Safari: PASS)
Don't crash if the PaymentShippingOptions.amount.value is an abusive length: FAIL (Chrome: PASS, Safari: PASS)
Don't crash if details.displayItems has an abusive number of items: FAIL (Chrome: PASS, Safari: PASS)
Don't crash if PaymentMethodData.supportedMethods is an abusive length: FAIL (Chrome: PASS, Safari: PASS)
Don't crash if PaymentItem.label is an abusive length: FAIL (Chrome: PASS, Safari: PASS)
Don't crash if there is an abusive number of payment methods in the methodData sequence: FAIL (Chrome: PASS, Safari: PASS)
Don't crash if details.shippingOptions has an abusive number of items: FAIL (Chrome: PASS, Safari: PASS)
Don't crash if total.amount.value is an abusive length: FAIL (Chrome: PASS, Safari: PASS)
Don't crash if PaymentShippingOptions.label is an abusive length: FAIL (Chrome: PASS, Safari: PASS)
/css/css-images/infinite-radial-gradient-refcrash.html: FAIL[Gecko-windows10-64-qr-debug,Gecko-windows10-64-qr-opt], PASS[Gecko-android-em-7.0-x86_64-debug-geckoview,Gecko-android-em-7.0-x86_64-opt-geckoview,Gecko-linux64-asan-opt,Gecko-linux64-debug,Gecko-linux64-opt,Gecko-linux64-qr-debug,Gecko-linux64-qr-opt,Gecko-windows10-64-debug,Gecko-windows10-64-opt,Gecko-windows7-32-debug,Gecko-windows7-32-opt,GitHub] (Chrome: PASS, Safari: FAIL)

Tests Disabled in Gecko Infrastructure

/payment-request/payment-request-constructor-thcrash.https.html: OK[GitHub], SKIP[Gecko-android-em-7.0-x86_64-debug-geckoview,Gecko-android-em-7.0-x86_64-opt-geckoview,Gecko-linux64-asan-opt,Gecko-linux64-debug,Gecko-linux64-opt,Gecko-linux64-qr-debug,Gecko-linux64-qr-opt,Gecko-windows10-64-debug,Gecko-windows10-64-opt,Gecko-windows10-64-qr-debug,Gecko-windows10-64-qr-opt,Gecko-windows7-32-debug,Gecko-windows7-32-opt] (Chrome: OK, Safari: OK)

Pushed by wptsync@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/f542df8107f7
[wpt PR 20017] - Add support for crashtests, a=testonly
https://hg.mozilla.org/integration/autoland/rev/6e141620d77b
[wpt PR 20017] - Update wpt metadata, a=testonly
Status: NEW → RESOLVED
Closed: 4 years ago
Resolution: --- → FIXED
Target Milestone: --- → mozilla73
You need to log in before you can comment on or make changes to this bug.