Temporarily disable some Tier 2 a11y_checks for Search and Address Bar
Categories
(Firefox :: Search, task, P3)
Tracking
()
Tracking | Status | |
---|---|---|
firefox120 | --- | fixed |
People
(Reporter: ayeddi, Assigned: ayeddi)
References
(Blocks 2 open bugs)
Details
(Keywords: access)
Attachments
(1 file)
We are working on enabling Tier 2 a11y-checks to ensure our products have basic accessibility built-in (bug 1692110) and before that, we need to prepare the existing code base. While we triage and investigate existent tests that would fail once the a11y-checks are enabled in the CI, we are going to skip the failing tests altogether and then handle each one of them individually.
This task is to skip with fail-if
condition of the failing tests while we continue investigation into the reasons these tests failed and to be able to backtrack these tests once they're resolved. A separate task to investigate the failed Tier 2 a11y_checks will be filed for an appropriate component.
Tests affected:
FAIL browser/components/search/test/browser/browser_hiddenOneOffs_diacritics.js Interactive elements must be labeled searchbar searchbar
FAIL browser/components/search/test/browser/browser_oneOffContextMenu_setDefault.js Interactive elements must be labeled searchbar searchbar
FAIL browser/components/search/test/browser/browser_oneOffContextMenu.js Interactive elements must be labeled searchbar searchbar
FAIL browser/components/search/test/browser/browser_searchbar_addEngine.js Interactive elements must be labeled searchbar searchbar
FAIL browser/components/search/test/browser/browser_searchbar_addEngine.js Node is not focusable via the accessibility API searchbar-engine-one-off-item-engine--1 button searchbar-engine-one-off-item searchbar-engine-one-off-add-engine
FAIL browser/components/search/test/browser/browser_searchbar_openpopup.js Interactive elements must be labeled searchbar searchbar
FAIL browser/components/search/test/browser/browser_searchbar_openpopup.js Node is not accessible via accessibility API html:body
FAIL browser/components/search/test/browser/browser_searchbar_smallpanel_keyboard_navigation.js Interactive elements must be labeled searchbar searchbar
FAIL browser/components/search/test/browser/telemetry/browser_search_telemetry_searchbar.js Node is not focusable via the accessibility API searchbar-engine-one-off-item-engine-0 button searchbar-engine-one-off-item
WARN browser/components/search/test/browser/telemetry/browser_search_telemetry_searchbar.js Unlabeled element removed before l10n finished richlistitem autocomplete-richlistitem -
FAIL browser/components/urlbar/tests/browser-tips/browser_picks.js Interactive elements must be labeled moz-input-box urlbar-input-box
FAIL browser/components/urlbar/tests/browser-tips/browser_searchTips_interaction.js Node is not accessible via accessibility API customizableui-special-spring1 toolbarspring chromeclass-toolbar-additional
FAIL browser/components/urlbar/tests/browser/browser_acknowledgeFeedbackAndDismissal.js Interactive elements must be labeled menuitem urlbarView-result-menuitem
WARN browser/components/urlbar/tests/browser/browser_acknowledgeFeedbackAndDismissal.js Unlabeled element removed before l10n finished menuitem urlbarView-result-menuitem -
FAIL browser/components/urlbar/tests/browser/browser_closePanelOnClick.js Node is not accessible via accessibility API li
FAIL browser/components/urlbar/tests/browser/browser_closePanelOnClick.js Node is not accessible via accessibility API scrollbox
FAIL browser/components/urlbar/tests/browser/browser_closePanelOnClick.js Node is not accessible via accessibility API customizableui-special-spring1 toolbarspring chromeclass-toolbar-additional
FAIL browser/components/urlbar/tests/browser/browser_groupLabels.js Interactive elements must be labeled moz-input-box urlbar-input-box
FAIL browser/components/urlbar/tests/browser/browser_inputHistory.js Node is not focusable via the accessibility API urlbar-engine-one-off-item-engine-6 button searchbar-engine-one-off-item
FAIL browser/components/urlbar/tests/browser/browser_locationBarExternalLoad.js Node is not accessible via accessibility API urlbar-go-button image urlbar-icon
FAIL browser/components/urlbar/tests/browser/browser_middleClick.js Node is not accessible via accessibility API scrollbox
FAIL browser/components/urlbar/tests/browser/browser_oneOffs_keyModifiers.js Node is not focusable via the accessibility API urlbar-engine-one-off-item-engine-0 button searchbar-engine-one-off-item
FAIL browser/components/urlbar/tests/browser/browser_oneOffs_keyModifiers.js Node is not focusable via the accessibility API urlbar-engine-one-off-item-bookmarks button searchbar-engine-one-off-item
FAIL browser/components/urlbar/tests/browser/browser_oneOffs_searchSuggestions.js Node is not focusable via the accessibility API urlbar-engine-one-off-item-engine-1 button searchbar-engine-one-off-item
FAIL browser/components/urlbar/tests/browser/browser_oneOffs.js Node is not focusable via the accessibility API urlbar-engine-one-off-item-tabs button searchbar-engine-one-off-item
FAIL browser/components/urlbar/tests/browser/browser_oneOffs.js Node is not focusable via the accessibility API urlbar-engine-one-off-item-history button searchbar-engine-one-off-item
FAIL browser/components/urlbar/tests/browser/browser_oneOffs.js Node is not focusable via the accessibility API urlbar-engine-one-off-item-actions button searchbar-engine-one-off-item
FAIL browser/components/urlbar/tests/browser/browser_placeholder.js Interactive elements must be labeled urlbar-search-mode-indicator-close html:div close-button
FAIL browser/components/urlbar/tests/browser/browser_placeholder.js Node is not focusable via the accessibility API urlbar-engine-one-off-item-engine-6 button searchbar-engine-one-off-item
WARN browser/components/urlbar/tests/browser/browser_placeholder.js Unlabeled element removed before l10n finished urlbar-search-mode-indicator-close html:div close-button -
FAIL browser/components/urlbar/tests/browser/browser_quickactions_screenshot.js Node is not focusable via the accessibility API urlbar-engine-one-off-item-actions button searchbar-engine-one-off-item
FAIL browser/components/urlbar/tests/browser/browser_quickactions.js Node is not focusable via the accessibility API urlbar-engine-one-off-item-engine-0 button searchbar-engine-one-off-item
FAIL browser/components/urlbar/tests/browser/browser_remove_match.js Interactive elements must be labeled urlbar-search-mode-indicator-close html:div close-button
FAIL browser/components/urlbar/tests/browser/browser_remove_match.js Node is not focusable via the accessibility API urlbar-engine-one-off-item-engine-0 button searchbar-engine-one-off-item
FAIL browser/components/urlbar/tests/browser/browser_searchMode_alias_replacement.js Interactive elements must be labeled urlbar-search-mode-indicator-close html:div close-button
FAIL browser/components/urlbar/tests/browser/browser_searchMode_autofill.js Node is not focusable via the accessibility API urlbar-engine-one-off-item-engine-0 button searchbar-engine-one-off-item
FAIL browser/components/urlbar/tests/browser/browser_searchMode_autofill.js Node is not focusable via the accessibility API urlbar-engine-one-off-item-history button searchbar-engine-one-off-item
FAIL browser/components/urlbar/tests/browser/browser_searchMode_clickLink.js Node is not focusable via the accessibility API urlbar-engine-one-off-item-bookmarks button searchbar-engine-one-off-item
FAIL browser/components/urlbar/tests/browser/browser_searchMode_engineRemoval.js Node is not focusable via the accessibility API urlbar-engine-one-off-item-engine-0 button searchbar-engine-one-off-item
FAIL browser/components/urlbar/tests/browser/browser_searchMode_excludeResults.js Node is not focusable via the accessibility API urlbar-engine-one-off-item-engine-0 button searchbar-engine-one-off-item
FAIL browser/components/urlbar/tests/browser/browser_searchMode_excludeResults.js Node is not focusable via the accessibility API urlbar-engine-one-off-item-engine-7 button searchbar-engine-one-off-item
FAIL browser/components/urlbar/tests/browser/browser_searchMode_heuristic.js Node is not focusable via the accessibility API urlbar-engine-one-off-item-bookmarks button searchbar-engine-one-off-item
FAIL browser/components/urlbar/tests/browser/browser_searchMode_heuristic.js Node is not focusable via the accessibility API urlbar-engine-one-off-item-engine-6 button searchbar-engine-one-off-item
FAIL browser/components/urlbar/tests/browser/browser_searchMode_indicator_clickthrough.js Interactive elements must be labeled urlbar-search-mode-indicator-close html:div close-button
FAIL browser/components/urlbar/tests/browser/browser_searchMode_indicator_clickthrough.js Node is not accessible via accessibility API urlbar-input-container hbox
FAIL browser/components/urlbar/tests/browser/browser_searchMode_indicator_clickthrough.js Node is not focusable via the accessibility API urlbar-engine-one-off-item-engine-0 button searchbar-engine-one-off-item
FAIL browser/components/urlbar/tests/browser/browser_searchMode_indicator.js Interactive elements must be labeled urlbar-search-mode-indicator-close html:div close-button
FAIL browser/components/urlbar/tests/browser/browser_searchMode_indicator.js Node is not focusable via the accessibility API urlbar-engine-one-off-item-engine-0 button searchbar-engine-one-off-item
WARN browser/components/urlbar/tests/browser/browser_searchMode_indicator.js Unlabeled element removed before l10n finished urlbar-search-mode-indicator-close html:div close-button -
FAIL browser/components/urlbar/tests/browser/browser_searchMode_oneOffButton.js Node is not focusable via the accessibility API urlbar-engine-one-off-item-engine-6 button searchbar-engine-one-off-item
WARN browser/components/urlbar/tests/browser/browser_searchMode_oneOffButton.js Unlabeled element removed before l10n finished urlbar-search-mode-indicator-close html:div close-button -
FAIL browser/components/urlbar/tests/browser/browser_searchMode_pickResult.js Node is not focusable via the accessibility API urlbar-engine-one-off-item-bookmarks button searchbar-engine-one-off-item
FAIL browser/components/urlbar/tests/browser/browser_searchMode_preview.js Node is not focusable via the accessibility API urlbar-engine-one-off-item-engine-0 button searchbar-engine-one-off-item
FAIL browser/components/urlbar/tests/browser/browser_searchMode_preview.js Node is not focusable via the accessibility API urlbar-engine-one-off-item-history button searchbar-engine-one-off-item
WARN browser/components/urlbar/tests/browser/browser_searchMode_preview.js Unlabeled element removed before l10n finished urlbar-search-mode-indicator-close html:div close-button -
FAIL browser/components/urlbar/tests/browser/browser_searchMode_sessionStore.js Node is not focusable via the accessibility API urlbar-engine-one-off-item-bookmarks button searchbar-engine-one-off-item
FAIL browser/components/urlbar/tests/browser/browser_searchMode_setURI.js Node is not focusable via the accessibility API urlbar-engine-one-off-item-bookmarks button searchbar-engine-one-off-item
FAIL browser/components/urlbar/tests/browser/browser_searchMode_suggestions.js Interactive elements must be labeled urlbar-search-mode-indicator-close html:div close-button
FAIL browser/components/urlbar/tests/browser/browser_searchMode_suggestions.js Node is not focusable via the accessibility API urlbar-engine-one-off-item-engine-0 button searchbar-engine-one-off-item
FAIL browser/components/urlbar/tests/browser/browser_searchMode_suggestions.js Node is not focusable via the accessibility API urlbar-engine-one-off-item-history button searchbar-engine-one-off-item
FAIL browser/components/urlbar/tests/browser/browser_searchMode_suggestions.js Node is not focusable via the accessibility API urlbar-engine-one-off-item-engine-8 button searchbar-engine-one-off-item
WARN browser/components/urlbar/tests/browser/browser_searchMode_suggestions.js Unlabeled element removed before l10n finished urlbar-search-mode-indicator-close html:div close-button -
FAIL browser/components/urlbar/tests/browser/browser_searchMode_switchTabs.js Interactive elements must be labeled urlbar-search-mode-indicator-close html:div close-button
FAIL browser/components/urlbar/tests/browser/browser_searchMode_switchTabs.js Node is not focusable via the accessibility API urlbar-engine-one-off-item-bookmarks button searchbar-engine-one-off-item
FAIL browser/components/urlbar/tests/browser/browser_searchMode_switchTabs.js Node is not focusable via the accessibility API urlbar-engine-one-off-item-tabs button searchbar-engine-one-off-item
FAIL browser/components/urlbar/tests/browser/browser_searchMode_switchTabs.js Node is not focusable via the accessibility API urlbar-engine-one-off-item-engine-6 button searchbar-engine-one-off-item
FAIL browser/components/urlbar/tests/browser/browser_searchSettings.js Node is not focusable via the accessibility API urlbar-anon-search-settings button searchbar-engine-one-off-item search-setting-button
FAIL browser/components/urlbar/tests/browser/browser_shortcuts_add_search_engine.js Node is not focusable via the accessibility API urlbar-engine-one-off-item-engine--1 button searchbar-engine-one-off-item searchbar-engine-one-off-add-engine
FAIL browser/components/urlbar/tests/browser/browser_switchToTab_chiclet.js Node is not focusable via the accessibility API urlbar-engine-one-off-item-tabs button searchbar-engine-one-off-item
FAIL browser/components/urlbar/tests/browser/browser_tabKeyBehavior.js Node is not focusable via the accessibility API urlbar-engine-one-off-item-history button searchbar-engine-one-off-item
FAIL browser/components/urlbar/tests/browser/browser_tokenAlias.js Node is not focusable via the accessibility API urlbar-engine-one-off-item-bookmarks button searchbar-engine-one-off-item
FAIL browser/components/urlbar/tests/browser/browser_tokenAlias.js Node is not focusable via the accessibility API urlbarView-row-5291-inner span urlbarView-row-inner
FAIL browser/components/urlbar/tests/browser/browser_tokenAlias.js Node is not focusable via the accessibility API urlbarView-row-5356-inner span urlbarView-row-inner
FAIL browser/components/urlbar/tests/browser/browser_top_sites.js Node is not focusable via the accessibility API urlbarView-row-5370-inner span urlbarView-row-inner
WARN browser/components/urlbar/tests/browser/browser_top_sites.js Unlabeled element removed before l10n finished urlbarView-row-5370-inner span urlbarView-row-inner -
FAIL browser/components/urlbar/tests/browser/browser_urlbar_telemetry_searchmode.js Node is not focusable via the accessibility API urlbar-engine-one-off-item-engine-0 button searchbar-engine-one-off-item
FAIL browser/components/urlbar/tests/browser/browser_urlbar_telemetry_searchmode.js Node is not focusable via the accessibility API urlbar-engine-one-off-item-history button searchbar-engine-one-off-item
FAIL browser/components/urlbar/tests/browser/browser_urlbar_telemetry_searchmode.js Node is not focusable via the accessibility API urlbar-engine-one-off-item-engine-2 button searchbar-engine-one-off-item
FAIL browser/components/urlbar/tests/browser/browser_urlbar_telemetry_searchmode.js Node is not focusable via the accessibility API urlbar-engine-one-off-item-engine-6 button searchbar-engine-one-off-item
FAIL browser/components/urlbar/tests/browser/browser_urlbar_telemetry_searchmode.js Node is not focusable via the accessibility API urlbarView-row-5865-inner span urlbarView-row-inner
WARN browser/components/urlbar/tests/browser/browser_urlbar_telemetry_searchmode.js Unlabeled element removed before l10n finished urlbarView-row-5865-inner span urlbarView-row-inner -
FAIL browser/components/urlbar/tests/browser/browser_urlbar_telemetry.js Node is not focusable via the accessibility API urlbar-engine-one-off-item-engine-0 button searchbar-engine-one-off-item
FAIL browser/components/urlbar/tests/browser/browser_urlbar_telemetry.js Node is not focusable via the accessibility API urlbar-engine-one-off-item-engine-7 button searchbar-engine-one-off-item
FAIL browser/components/urlbar/tests/browser/browser_UrlbarInput_searchTerms_searchMode.js Node is not focusable via the accessibility API urlbar-engine-one-off-item-engine-6 button searchbar-engine-one-off-item
FAIL browser/components/urlbar/tests/browser/browser_view_selectionByMouse.js Interactive elements must be labeled moz-input-box urlbar-input-box
FAIL browser/components/urlbar/tests/browser/browser_view_selectionByMouse.js Interactive elements must be labeled moz-input-box urlbar-input-box
FAIL browser/components/urlbar/tests/browser/browser_view_selectionByMouse.js Node is not accessible via accessibility API html:body
FAIL browser/components/urlbar/tests/browser/browser_view_selectionByMouse.js Node is not accessible via accessibility API browser
FAIL browser/components/urlbar/tests/engagementTelemetry/browser/browser_glean_telemetry_abandonment_search_mode.js Node is not focusable via the accessibility API urlbar-engine-one-off-item-engine-0 button searchbar-engine-one-off-item
FAIL browser/components/urlbar/tests/engagementTelemetry/browser/browser_glean_telemetry_abandonment_search_mode.js Node is not focusable via the accessibility API urlbar-engine-one-off-item-bookmarks button searchbar-engine-one-off-item
FAIL browser/components/urlbar/tests/engagementTelemetry/browser/browser_glean_telemetry_abandonment_search_mode.js Node is not focusable via the accessibility API urlbar-engine-one-off-item-history button searchbar-engine-one-off-item
FAIL browser/components/urlbar/tests/engagementTelemetry/browser/browser_glean_telemetry_abandonment_search_mode.js Node is not focusable via the accessibility API urlbar-engine-one-off-item-tabs button searchbar-engine-one-off-item
FAIL browser/components/urlbar/tests/engagementTelemetry/browser/browser_glean_telemetry_abandonment_search_mode.js Node is not focusable via the accessibility API urlbar-engine-one-off-item-actions button searchbar-engine-one-off-item
FAIL browser/components/urlbar/tests/engagementTelemetry/browser/browser_glean_telemetry_abandonment_tips.js Node is not accessible via accessibility API browser
FAIL browser/components/urlbar/tests/engagementTelemetry/browser/browser_glean_telemetry_engagement_edge_cases.js Node is not accessible via accessibility API customizableui-special-spring2 toolbarspring chromeclass-toolbar-additional
FAIL browser/components/urlbar/tests/engagementTelemetry/browser/browser_glean_telemetry_engagement_search_mode.js Node is not focusable via the accessibility API urlbar-engine-one-off-item-engine-0 button searchbar-engine-one-off-item
FAIL browser/components/urlbar/tests/engagementTelemetry/browser/browser_glean_telemetry_engagement_search_mode.js Node is not focusable via the accessibility API urlbar-engine-one-off-item-bookmarks button searchbar-engine-one-off-item
FAIL browser/components/urlbar/tests/engagementTelemetry/browser/browser_glean_telemetry_engagement_search_mode.js Node is not focusable via the accessibility API urlbar-engine-one-off-item-history button searchbar-engine-one-off-item
FAIL browser/components/urlbar/tests/engagementTelemetry/browser/browser_glean_telemetry_engagement_search_mode.js Node is not focusable via the accessibility API urlbar-engine-one-off-item-tabs button searchbar-engine-one-off-item
FAIL browser/components/urlbar/tests/engagementTelemetry/browser/browser_glean_telemetry_engagement_search_mode.js Node is not focusable via the accessibility API urlbar-engine-one-off-item-actions button searchbar-engine-one-off-item
FAIL browser/components/urlbar/tests/engagementTelemetry/browser/browser_glean_telemetry_engagement_type.js Node is not accessible via accessibility API urlbar-go-button image urlbar-icon
FAIL browser/components/urlbar/tests/engagementTelemetry/browser/browser_glean_telemetry_impression_search_mode.js Node is not focusable via the accessibility API urlbar-engine-one-off-item-engine-0 button searchbar-engine-one-off-item
FAIL browser/components/urlbar/tests/engagementTelemetry/browser/browser_glean_telemetry_impression_search_mode.js Node is not focusable via the accessibility API urlbar-engine-one-off-item-bookmarks button searchbar-engine-one-off-item
FAIL browser/components/urlbar/tests/engagementTelemetry/browser/browser_glean_telemetry_impression_search_mode.js Node is not focusable via the accessibility API urlbar-engine-one-off-item-history button searchbar-engine-one-off-item
FAIL browser/components/urlbar/tests/engagementTelemetry/browser/browser_glean_telemetry_impression_search_mode.js Node is not focusable via the accessibility API urlbar-engine-one-off-item-tabs button searchbar-engine-one-off-item
FAIL browser/components/urlbar/tests/engagementTelemetry/browser/browser_glean_telemetry_impression_search_mode.js Node is not focusable via the accessibility API urlbar-engine-one-off-item-actions button searchbar-engine-one-off-item
Some of similar test failures were reported in [Session Restore](bug 1854529) and [Tabbed Browser](bug 1854507) components as well.
Jobs affected:
test-linux1804-64-qr/opt-mochitest-browser-chrome-swr-a11y-checks-
${1-7} (Try run)
Some Tier 2 accessibility checks for click events fired on controls that should be keyboard accessible and have valid labels were failing for this component. They were captured by testing/mochitest/tests/SimpleTest/AccessibilityUtils.js
via bug 1692110. These failing tests should be temporarily skipped in the directory's browser.ini
file while we investigate these failures. For all confirmed bugs individual defects should be filed.
When the individual test failures are resolved and the individual bugs are closed, remove the fail-if
condition for a11y_checks
from the appropriate files/sections to ensure better test coverage and to avoid regressions in a11y of these components.
Assignee | ||
Comment 1•2 years ago
|
||
We are working on enabling Tier 2 a11y-checks to ensure our products have basic accessibility built-in (bug 1692110) and before that, we need to prepare the existing code base. While we triage and investigate existent tests that would fail once the a11y-checks are enabled in the CI, we are going to skip the failing tests altogether and then handle each one of them individually.
This task is to skip with fail-if
condition of the failing tests while we continue investigation into the reasons these tests failed and to be able to backtrack these tests once they're resolved.
Related bugs:
- Disabling tests: meta bug 1848402
- Investigation and remediation for components with disabled tests: meta bugs 1848394 and 1854227, Search and Address Bar-specific meta bug 1854661
- Explore capturing click events for a11y_checks: bug 1692110
Depends on D188992
Assignee | ||
Comment 2•2 years ago
|
||
The following tests failed today on Try (job):
FAIL browser/components/search/test/browser/browser_oneOffContextMenu.js Interactive elements must be labeled searchbar searchbar
FAIL browser/components/search/test/browser/browser_searchbar_openpopup.js Node is not accessible via accessibility API html:body
FAIL browser/components/urlbar/tests/browser/browser_groupLabels.js Interactive elements must be labeled moz-input-box urlbar-input-box
Updated•2 years ago
|
Updated•2 years ago
|
Comment 4•2 years ago
|
||
bugherder |
Comment 5•2 years ago
|
||
Backed out for causing multiple tier 2 failures on a11y-checks.
https://bugzilla.mozilla.org/show_bug.cgi?id=1857672
https://bugzilla.mozilla.org/show_bug.cgi?id=1857666
https://bugzilla.mozilla.org/show_bug.cgi?id=1857664
https://bugzilla.mozilla.org/show_bug.cgi?id=1857665
https://bugzilla.mozilla.org/show_bug.cgi?id=1857707
Updated•2 years ago
|
Comment 7•2 years ago
|
||
bugherder |
Description
•