Closed Bug 1438563 Opened 6 years ago Closed 6 years ago

[wpt-sync] PR 9540 - Add initial test set for Accessible Name and Description Computation 1.1

Categories

(Testing :: web-platform-tests, enhancement)

Product:
Testing
Component:
web-platform-tests
Type:
enhancement
Priority:
Not set
Severity:
normal

Tracking

(firefox61 fixed)

Status:
RESOLVED FIXED
Milestone:
mozilla61
Tracking Flags:
Tracking Status
firefox61 --- fixed

People

(Reporter: mozilla.org, Unassigned)

Details

(Whiteboard: [wptsync downstream]) 

Note that many of the tests are currently named with a number. Those
tests were taken from the original (1.0) test suite created by the
PFWG, and the file names are being preserved to facilitate comparison
of results.
Ran 4472[linux32-opt,linux64-debug,linux64-opt], 4417[linux64-stylo-disabled-debug,linux64-stylo-disabled-opt], 14373[linux32-debug] tests and 59680[linux64-stylo-disabled-debug], 60514[linux32-debug], 60515[linux64-debug], 59695[linux64-stylo-disabled-opt], 60538[linux32-opt], 60541[linux64-opt] subtests
OK     : 776[linux64-stylo-disabled-debug], 828[linux32-debug,linux64-debug], 835[linux32-opt], 836[linux64-opt], 783[linux64-stylo-disabled-opt]
PASS   : 65019[linux32-debug], 56996[linux64-debug], 56205[linux64-stylo-disabled-debug], 57019[linux32-opt], 56220[linux64-stylo-disabled-opt], 57022[linux64-opt]
FAIL   : 7043[linux32-opt,linux64-debug,linux64-opt], 8868[linux32-debug], 6999[linux64-stylo-disabled-debug,linux64-stylo-disabled-opt]
TIMEOUT: 17[linux32-debug], 13[linux32-opt,linux64-debug,linux64-opt,linux64-stylo-disabled-debug,linux64-stylo-disabled-opt]
ERROR  : 3
NOTRUN : 2

These new tests don't PASS on all platforms
/webdriver/tests/actions/key.py: SKIP[linux32-opt]
/webdriver/tests/actions/key_shortcuts.py: SKIP[linux32-opt]
/webdriver/tests/contexts/json_serialize_windowproxy.py
    test_frame: FAIL[linux32-opt]
    test_initial_window: FAIL[linux32-opt]
    test_window_open: FAIL[linux32-opt]
/webdriver/tests/contexts/maximize_window.py: SKIP[linux32-opt]
/webdriver/tests/cookies/add_cookie.py
    test_add_cookie_for_ip: FAIL[linux32-opt]
    test_add_domain_cookie: FAIL[linux32-opt]
    test_add_non_session_cookie: FAIL[linux32-opt]
    test_add_session_cookie: FAIL[linux32-opt]
    test_add_session_cookie_with_leading_dot_character_in_domain: FAIL[linux32-opt]
/webdriver/tests/cookies/delete_cookie.py
    test_handle_prompt_accept: FAIL[linux32-opt]
    test_unknown_cookie: FAIL[linux32-opt]
/webdriver/tests/cookies/get_named_cookie.py
    test_duplicated_cookie: FAIL[linux32-opt]
/webdriver/tests/element_retrieval/get_active_element.py
    test_handle_prompt_accept: FAIL[linux32-opt]
    test_handle_prompt_dismiss: FAIL[linux32-opt]
/webdriver/tests/execute_async_script/user_prompts.py
    test_handle_prompt_accept: FAIL[linux32-opt]
    test_handle_prompt_accept_and_notify: FAIL[linux32-opt]
    test_handle_prompt_default: FAIL[linux32-opt]
    test_handle_prompt_dismiss: FAIL[linux32-opt]
    test_handle_prompt_dismiss_and_notify: FAIL[linux32-opt]
    test_handle_prompt_ignore: FAIL[linux32-opt]
    test_handle_prompt_twice: FAIL[linux32-opt]
/webdriver/tests/execute_script/user_prompts.py
    test_handle_prompt_accept: FAIL[linux32-opt]
    test_handle_prompt_accept_and_notify: FAIL[linux32-opt]
    test_handle_prompt_default: FAIL[linux32-opt]
    test_handle_prompt_dismiss: FAIL[linux32-opt]
    test_handle_prompt_dismiss_and_notify: FAIL[linux32-opt]
    test_handle_prompt_ignore: FAIL[linux32-opt]
    test_handle_prompt_twice: FAIL[linux32-opt]
/webdriver/tests/fullscreen_window.py
    test_handle_prompt_accept: FAIL[linux32-opt]
    test_handle_prompt_missing_value: FAIL[linux32-opt]
/webdriver/tests/get_window_rect.py
    test_handle_prompt_accept: FAIL[linux32-opt]
/webdriver/tests/minimize_window.py
    test_handle_prompt_accept: FAIL[linux32-opt]
/webdriver/tests/navigation/get_title.py
    test_title_handle_prompt_accept: FAIL[linux32-opt]
    test_title_handle_prompt_dismiss: FAIL[linux32-opt]
/webdriver/tests/sessions/new_session/create_alwaysMatch.py
    test_valid[timeouts-value10]: FAIL[linux32-opt]
/webdriver/tests/sessions/new_session/create_firstMatch.py
    test_valid[timeouts-value10]: FAIL[linux32-opt]
/webdriver/tests/sessions/new_session/default_values.py
    test_desired: FAIL[linux32-opt]
    test_no_capabilites: FAIL[linux32-opt]
    test_valid_but_unmatchable_key: FAIL[linux32-opt]
/webdriver/tests/sessions/new_session/response.py
    test_resp_capabilites: FAIL[linux32-opt]
    test_resp_data: FAIL[linux32-opt]
/webdriver/tests/set_window_rect.py
    test_handle_prompt_accept: FAIL[linux32-opt]
/webdriver/tests/state/get_element_attribute.py
    test_handle_prompt_accept: FAIL[linux32-opt]
    test_handle_prompt_dismiss: FAIL[linux32-opt]
/webdriver/tests/state/get_element_property.py
    test_element: FAIL[linux32-opt]
    test_handle_prompt_accept: FAIL[linux32-opt]
    test_handle_prompt_dismiss: FAIL[linux32-opt]
/webdriver/tests/state/get_element_tag_name.py
    test_handle_prompt_accept: FAIL[linux32-opt]
    test_handle_prompt_dismiss: FAIL[linux32-opt]
/webdriver/tests/state/is_element_selected.py
    test_handle_prompt_accept: FAIL[linux32-opt]
    test_handle_prompt_dismiss: FAIL[linux32-opt]

These tests are disabled
/2dcontext/building-paths/canvas_complexshapes_beziercurveto_001.htm: [linux32-debug]
/WebCryptoAPI/generateKey/test_failures_RSA-OAEP.https.html
/WebCryptoAPI/generateKey/test_failures_RSA-PSS.https.html
/WebCryptoAPI/generateKey/test_failures_RSASSA-PKCS1-v1_5.https.html
/WebCryptoAPI/generateKey/test_successes_AES-CBC.https.html
/WebCryptoAPI/generateKey/test_successes_RSA-OAEP.https.html
/WebCryptoAPI/generateKey/test_successes_RSA-PSS.https.html
/WebCryptoAPI/generateKey/test_successes_RSASSA-PKCS1-v1_5.https.html
/_mozilla/dom/throttling/throttling-1.window.html: [linux32-debug,linux64-debug,linux64-stylo-disabled-debug]
/_mozilla/dom/throttling/throttling-2.window.html: [linux32-debug,linux64-debug,linux64-stylo-disabled-debug]
/_mozilla/dom/throttling/throttling-3.window.html: [linux32-debug,linux64-debug,linux64-stylo-disabled-debug]
/_mozilla/dom/throttling/throttling-4.window.html: [linux32-debug,linux64-debug,linux64-stylo-disabled-debug]
/_mozilla/dom/throttling/throttling-indexeddb.window.html: [linux32-debug,linux64-debug,linux64-stylo-disabled-debug]
/_mozilla/dom/throttling/throttling-webaudio.window.html: [linux32-debug,linux64-debug,linux64-stylo-disabled-debug]
/_mozilla/dom/throttling/throttling-webrtc.window.html: [linux32-debug,linux64-debug,linux64-stylo-disabled-debug]
/_mozilla/dom/throttling/throttling-ws.window.html: [linux32-debug,linux64-debug,linux64-stylo-disabled-debug]
/bluetooth/characteristic/getDescriptors/gen-characteristic-is-removed-with-uuid.https.html
/bluetooth/characteristic/getDescriptors/gen-characteristic-is-removed.https.html
/bluetooth/characteristic/getDescriptors/gen-descriptor-get-same-object.https.html
/content-security-policy/media-src/media-src-7_1.html
/css/compositing/mix-blend-mode/mix-blend-mode-animation.html: [linux32-debug]
/css/css-animations/animation-delay-010.html: [linux32-debug]
/css/css-backgrounds/background-334.html
/css/css-backgrounds/background-attachment-local/attachment-local-clipping-image-4.html: [linux32-debug]
/css/css-backgrounds/background-attachment-local/attachment-local-clipping-image-5.html: [linux32-debug]
/css/css-backgrounds/background-repeat/background-repeat-round-roundup.xht: [linux32-debug]
/css/css-backgrounds/border-image-slice-001.xht
/css/selectors/focus-within-001.html: [linux32-debug]
/css/selectors/focus-within-002.html: [linux32-debug]
/css/selectors/focus-within-003.html: [linux32-debug]
/css/selectors/focus-within-004.html: [linux32-debug]
/css/selectors/focus-within-005.html: [linux32-debug]
/css/selectors/focus-within-006.html: [linux32-debug]
/css/selectors/focus-within-007.html: [linux32-debug]
/css/selectors/focus-within-008.html: [linux32-debug]
/css/selectors/focus-within-010.html: [linux32-debug]
/css/vendor-imports/mozilla/mozilla-central-reftests/flexbox/flexbox-min-width-auto-002a.html: [linux32-debug]
/css/vendor-imports/mozilla/mozilla-central-reftests/flexbox/flexbox-min-width-auto-002c.html: [linux32-debug]
/css/vendor-imports/mozilla/mozilla-central-reftests/images3/object-fit-contain-svg-001e.html
/css/vendor-imports/mozilla/mozilla-central-reftests/images3/object-fit-contain-svg-001o.html
/css/vendor-imports/mozilla/mozilla-central-reftests/images3/object-fit-contain-svg-002e.html
/css/vendor-imports/mozilla/mozilla-central-reftests/images3/object-fit-contain-svg-002o.html
/css/vendor-imports/mozilla/mozilla-central-reftests/images3/object-fit-contain-svg-003e.html
/css/vendor-imports/mozilla/mozilla-central-reftests/images3/object-fit-contain-svg-003o.html
/css/vendor-imports/mozilla/mozilla-central-reftests/images3/object-fit-contain-svg-004e.html
/css/vendor-imports/mozilla/mozilla-central-reftests/images3/object-fit-contain-svg-004o.html
/css/vendor-imports/mozilla/mozilla-central-reftests/images3/object-fit-contain-svg-005e.html
/css/vendor-imports/mozilla/mozilla-central-reftests/images3/object-fit-contain-svg-005o.html
/css/vendor-imports/mozilla/mozilla-central-reftests/images3/object-fit-contain-svg-006e.html
/css/vendor-imports/mozilla/mozilla-central-reftests/images3/object-fit-contain-svg-006o.html
/css/vendor-imports/mozilla/mozilla-central-reftests/images3/object-fit-cover-svg-001e.html
/css/vendor-imports/mozilla/mozilla-central-reftests/images3/object-fit-cover-svg-001o.html
/css/vendor-imports/mozilla/mozilla-central-reftests/images3/object-fit-cover-svg-002e.html
/css/vendor-imports/mozilla/mozilla-central-reftests/images3/object-fit-cover-svg-002o.html
/css/vendor-imports/mozilla/mozilla-central-reftests/images3/object-fit-cover-svg-003e.html
/css/vendor-imports/mozilla/mozilla-central-reftests/images3/object-fit-cover-svg-003o.html
/css/vendor-imports/mozilla/mozilla-central-reftests/images3/object-fit-cover-svg-004e.html
/css/vendor-imports/mozilla/mozilla-central-reftests/images3/object-fit-cover-svg-004o.html
/css/vendor-imports/mozilla/mozilla-central-reftests/images3/object-fit-cover-svg-005e.html
/css/vendor-imports/mozilla/mozilla-central-reftests/images3/object-fit-cover-svg-005o.html
/css/vendor-imports/mozilla/mozilla-central-reftests/images3/object-fit-cover-svg-006e.html
/css/vendor-imports/mozilla/mozilla-central-reftests/images3/object-fit-cover-svg-006o.html
/css/vendor-imports/mozilla/mozilla-central-reftests/images3/object-fit-fill-svg-001e.html
/css/vendor-imports/mozilla/mozilla-central-reftests/images3/object-fit-fill-svg-001o.html
/css/vendor-imports/mozilla/mozilla-central-reftests/images3/object-fit-fill-svg-002e.html
/css/vendor-imports/mozilla/mozilla-central-reftests/images3/object-fit-fill-svg-002o.html
/css/vendor-imports/mozilla/mozilla-central-reftests/images3/object-fit-fill-svg-003e.html
/css/vendor-imports/mozilla/mozilla-central-reftests/images3/object-fit-fill-svg-003o.html
/css/vendor-imports/mozilla/mozilla-central-reftests/images3/object-fit-fill-svg-004e.html
/css/vendor-imports/mozilla/mozilla-central-reftests/images3/object-fit-fill-svg-004o.html
/css/vendor-imports/mozilla/mozilla-central-reftests/images3/object-fit-fill-svg-005e.html
/css/vendor-imports/mozilla/mozilla-central-reftests/images3/object-fit-fill-svg-005o.html
/css/vendor-imports/mozilla/mozilla-central-reftests/images3/object-fit-fill-svg-006e.html
/css/vendor-imports/mozilla/mozilla-central-reftests/images3/object-fit-fill-svg-006o.html
/css/vendor-imports/mozilla/mozilla-central-reftests/images3/object-fit-none-svg-001e.html
/css/vendor-imports/mozilla/mozilla-central-reftests/images3/object-fit-none-svg-001o.html
/css/vendor-imports/mozilla/mozilla-central-reftests/images3/object-fit-none-svg-002e.html
/css/vendor-imports/mozilla/mozilla-central-reftests/images3/object-fit-none-svg-002o.html
/css/vendor-imports/mozilla/mozilla-central-reftests/images3/object-fit-none-svg-003e.html
/css/vendor-imports/mozilla/mozilla-central-reftests/images3/object-fit-none-svg-003o.html
/css/vendor-imports/mozilla/mozilla-central-reftests/images3/object-fit-none-svg-004e.html
/css/vendor-imports/mozilla/mozilla-central-reftests/images3/object-fit-none-svg-004o.html
/css/vendor-imports/mozilla/mozilla-central-reftests/images3/object-fit-none-svg-005e.html
/css/vendor-imports/mozilla/mozilla-central-reftests/images3/object-fit-none-svg-005o.html
/css/vendor-imports/mozilla/mozilla-central-reftests/images3/object-fit-none-svg-006e.html
/css/vendor-imports/mozilla/mozilla-central-reftests/images3/object-fit-none-svg-006o.html
/css/vendor-imports/mozilla/mozilla-central-reftests/images3/object-fit-scale-down-svg-001e.html
/css/vendor-imports/mozilla/mozilla-central-reftests/images3/object-fit-scale-down-svg-001o.html
/css/vendor-imports/mozilla/mozilla-central-reftests/images3/object-fit-scale-down-svg-002e.html
/css/vendor-imports/mozilla/mozilla-central-reftests/images3/object-fit-scale-down-svg-002o.html
/css/vendor-imports/mozilla/mozilla-central-reftests/images3/object-fit-scale-down-svg-003e.html
/css/vendor-imports/mozilla/mozilla-central-reftests/images3/object-fit-scale-down-svg-003o.html
/css/vendor-imports/mozilla/mozilla-central-reftests/images3/object-fit-scale-down-svg-004e.html
/css/vendor-imports/mozilla/mozilla-central-reftests/images3/object-fit-scale-down-svg-004o.html
/css/vendor-imports/mozilla/mozilla-central-reftests/images3/object-fit-scale-down-svg-005e.html
/css/vendor-imports/mozilla/mozilla-central-reftests/images3/object-fit-scale-down-svg-005o.html
/css/vendor-imports/mozilla/mozilla-central-reftests/images3/object-fit-scale-down-svg-006e.html
/css/vendor-imports/mozilla/mozilla-central-reftests/images3/object-fit-scale-down-svg-006o.html
/css/vendor-imports/mozilla/mozilla-central-reftests/masking/mask-image-1c.html: [linux32-debug]
/css/vendor-imports/mozilla/mozilla-central-reftests/ui3/box-sizing-replaced-001.xht: [linux32-debug]
/dom/events/Event-timestamp-high-resolution.html: [linux32-opt,linux64-opt,linux64-stylo-disabled-opt]
/feature-policy/autoplay-allowed-by-feature-policy-attribute-redirect-on-load.https.sub.html
/feature-policy/autoplay-allowed-by-feature-policy-attribute.https.sub.html
/feature-policy/autoplay-allowed-by-feature-policy.https.sub.html
/feature-policy/autoplay-default-feature-policy.https.sub.html
/feature-policy/autoplay-disabled-by-feature-policy.https.sub.html
/feature-policy/picture-in-picture-allowed-by-feature-policy-attribute-redirect-on-load.https.sub.html
/feature-policy/picture-in-picture-allowed-by-feature-policy-attribute.https.sub.html
/feature-policy/picture-in-picture-allowed-by-feature-policy.https.sub.html
/feature-policy/picture-in-picture-default-feature-policy.https.sub.html
/feature-policy/picture-in-picture-disabled-by-feature-policy.https.sub.html
/html/editing/focus/focus-management/focus-event-targets-simple.html
/html/editing/focus/focus-management/focus-events.html: [linux32-opt,linux64-debug,linux64-opt,linux64-stylo-disabled-debug,linux64-stylo-disabled-opt]
/html/semantics/embedded-content/media-elements/track/track-element/track-cue-rendering-after-controls-added.html: [linux32-debug]
/mixed-content/link-prefetch-tag/no-opt-in/cross-origin-http/top-level/swap-scheme-redirect/blockable/no-opt-in-blocks.https.html
/mixed-content/link-prefetch-tag/no-opt-in/same-host-http/top-level/swap-scheme-redirect/blockable/no-opt-in-blocks.https.html
/webdriver/tests/actions/key.py: [linux32-debug,linux64-debug,linux64-opt]
/webdriver/tests/actions/key_shortcuts.py: [linux32-debug]
/webdriver/tests/contexts/maximize_window.py: [linux32-debug,linux64-debug,linux64-opt]
/webdriver/tests/element_click/select.py: [linux32-debug,linux64-debug]
/webvtt/rendering/cues-with-video/processing-model/2_tracks.html: [linux32-debug]
/webvtt/rendering/cues-with-video/processing-model/3_tracks.html: [linux32-debug]
/webvtt/rendering/cues-with-video/processing-model/align_start.html: [linux32-debug]
/webvtt/rendering/cues-with-video/processing-model/align_start_wrapped.html: [linux32-debug]
/webvtt/rendering/cues-with-video/processing-model/bidi/u05D0_first.html: [linux32-debug]
/webvtt/rendering/cues-with-video/processing-model/bidi/u0628_first.html: [linux32-debug]
/webvtt/rendering/cues-with-video/processing-model/dom_override_cue_align_position_line_size.html: [linux32-debug]
/webvtt/rendering/cues-with-video/processing-model/dom_override_cue_align_position_line_size_while_paused.html: [linux32-debug]
/webvtt/rendering/cues-with-video/processing-model/dom_override_cue_line.html: [linux32-debug]
/webvtt/rendering/cues-with-video/processing-model/dom_override_cue_text.html: [linux32-debug]
/webvtt/rendering/cues-with-video/processing-model/dom_override_cue_text_while_paused.html: [linux32-debug]
/webvtt/rendering/cues-with-video/processing-model/dom_override_remove_cue_while_paused.html: [linux32-debug]
/webvtt/rendering/cues-with-video/processing-model/evil/single_quote.html: [linux32-debug]
/webvtt/rendering/cues-with-video/processing-model/line_0_is_top.html: [linux32-debug]
/webvtt/rendering/cues-with-video/processing-model/media_with_controls.html: [linux32-debug]
/webvtt/rendering/cues-with-video/processing-model/repaint.html: [linux32-debug]
/webvtt/rendering/cues-with-video/processing-model/selectors/cue/background_properties.html: [linux32-debug]
/webvtt/rendering/cues-with-video/processing-model/selectors/cue/background_shorthand.html: [linux32-debug]
/webvtt/rendering/cues-with-video/processing-model/selectors/cue/background_shorthand_css_relative_url.html: [linux32-debug]
/webvtt/rendering/cues-with-video/processing-model/selectors/cue/white-space_pre-wrap_wrapped.html: [linux32-debug]
/webvtt/rendering/cues-with-video/processing-model/selectors/cue/white-space_pre_wrapped.html: [linux32-debug]
/webvtt/rendering/cues-with-video/processing-model/selectors/cue_function/bold_object/bold_white-space_normal_wrapped.html: [linux32-debug]
/webvtt/rendering/cues-with-video/processing-model/selectors/cue_function/bold_object/bold_white-space_pre-line_wrapped.html: [linux32-debug]
/webvtt/rendering/cues-with-video/processing-model/selectors/cue_function/class_object/class_white-space_normal_wrapped.html: [linux32-debug]
/webvtt/rendering/cues-with-video/processing-model/selectors/cue_function/class_object/class_white-space_pre-line_wrapped.html: [linux32-debug]
/webvtt/rendering/cues-with-video/processing-model/selectors/cue_function/italic_object/italic_white-space_normal_wrapped.html: [linux32-debug]
/webvtt/rendering/cues-with-video/processing-model/selectors/cue_function/italic_object/italic_white-space_pre-line_wrapped.html: [linux32-debug]
/webvtt/rendering/cues-with-video/processing-model/selectors/cue_function/underline_object/underline_white-space_normal_wrapped.html
/webvtt/rendering/cues-with-video/processing-model/selectors/cue_function/underline_object/underline_white-space_pre-line_wrapped.html
/webvtt/rendering/cues-with-video/processing-model/selectors/cue_function/voice_object/voice_white-space_normal_wrapped.html
/webvtt/rendering/cues-with-video/processing-model/selectors/cue_function/voice_object/voice_white-space_pre-line_wrapped.html
/webvtt/rendering/cues-with-video/processing-model/size_50.html: [linux32-debug]
Pushed by james@hoppipolla.co.uk:
https://hg.mozilla.org/integration/mozilla-inbound/rev/2b24b05fc9b7
[wpt PR 9540] - Add initial test set for Accessible Name and Description Computation 1.1, a=testonly
https://hg.mozilla.org/mozilla-central/rev/2b24b05fc9b7
Status: NEW → RESOLVED
Closed: 6 years ago
status-firefox61: --- → fixed
Resolution: --- → FIXED
Target Milestone: --- → mozilla61
You need to log in before you can comment on or make changes to this bug.