1667998 - (a11y-checks) [meta] Automated accessibility checking for Firefox desktop.

Status: NEW

(Testing :: General, task, P3)

Description

[Yura Zenevich [:yzen] (please use "needinfo")]

We are looking into integrating accessibility checks into our browser-chrome testing framework whenever users perform various interactions (click for example).

This work includes several steps/stages:

• enabling accessibility service while browser chrome tests are run
• creating a module similar to EventUtils that can contains various accessibility checks that can either be used on its own or integrated into other utilities such as EventUtils
• ...

Comment 1

[Yura Zenevich [:yzen] (please use "needinfo")]

Attachment: Bug 1667998 - running tests with accessiblity enabled.

Comment 2

[Yura Zenevich [:yzen] (please use "needinfo")]

Attachment: Bug 1667998 - running tests with accessiblity enabled.

Comment 3

[Yura Zenevich [:yzen] (please use "needinfo")]

Attachment: Bug 1667998 - run browser tests with accessibility service enabled, unless "testing.accessibility.disable_by_default" pref is set

Comment 4

[Yura Zenevich [:yzen] (please use "needinfo")]

Attachment: Bug 1667998 - add AccessibilityUtils for testing accessibility related issues and use it to check accessibility when EventUtils.sendMouseEvent is used to make a click

Depends on D96876

Comment 5

[Yura Zenevich [:yzen] (please use "needinfo")]

Attachment: Bug 1667998 - disable accessibility by default in some accessible browser-chrome tests.

Depends on D96877

Comment 6

[Yura Zenevich [:yzen] (please use "needinfo")]

Attachment: Bug 1667998 - move accessibility indicator test into its separate directory.

Depends on D96878

Comment 7

[Yura Zenevich [:yzen] (please use "needinfo")]

Attachment: Bug 1667998 - disable accessibility by default when running devtools accessibility and server tests.

Depends on D96879

Comment 8

[Yura Zenevich [:yzen] (please use "needinfo")]

Attachment: Bug 1667998 - converted calls to EventUtils.sendMouseEvent to be async. r=jmaher

Depends on D96880

Comment 9

[Yura Zenevich [:yzen] (please use "needinfo")]

Attachment: Bug 1667998 - pass accessibility checks for tests that need a11y exceptions. r=jmaher

Depends on D97323

Comment 10

[Yura Zenevich [:yzen] (please use "needinfo")]

Attachment: Bug 1667998 - wip for allowing test failures when performing a11y checks.

Depends on D97324

Comment 11

[Yura Zenevich [:yzen] (please use "needinfo")]

Attachment: Bug 1667998 - add AccessibilityUtils for testing accessibility related issues and use it to check accessibility when EventUtils.sendMouseEvent is used to make a click. r=jmaher

Comment 12

[Yura Zenevich [:yzen] (please use "needinfo")]

Attachment: Bug 1667998 - [ci] Create a new 'a11y-checks' test variant. r=jmaher

Depends on D97324

Comment 13

[Yura Zenevich [:yzen] (please use "needinfo")]

Attachment: Bug 1667998 - skip some accessible related browser-chrome tests when running a11y-checks. r=jmaher

Depends on D100840

Comment 14

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/6d187c01fcb3
add AccessibilityUtils for testing accessibility related issues and use it to check accessibility when EventUtils.sendMouseEvent is used to make a click. r=jmaher
https://hg.mozilla.org/integration/autoland/rev/615b74c822ec
converted calls to EventUtils.sendMouseEvent to be async. r=jmaher
https://hg.mozilla.org/integration/autoland/rev/501c54cf59be
pass accessibility checks for tests that need a11y exceptions. r=jmaher
https://hg.mozilla.org/integration/autoland/rev/c9838cdb1303
[ci] Create a new 'a11y-checks' test variant. r=jmaher
https://hg.mozilla.org/integration/autoland/rev/633dda979b22
skip some accessible related browser-chrome tests when running a11y-checks. r=jmaher

Comment 15

[Noemi Erli[:noemi_erli]]

Backed out 5 changesets (Bug 1667998) for breaking a11y checks CLOSED TREE

Push with failures: https://treeherder.mozilla.org/jobs?repo=autoland&group_state=expanded&collapsedPushes=842382&resultStatus=testfailed%2Cbusted%2Cexception%2Cretry%2Cusercancel&classifiedState=classified&revision=633dda979b228d254d874059f9d239bc61760201&selectedTaskRun=XJgb5DhuQo-5zT7ZNsMZ_w.0

Failure log: https://treeherder.mozilla.org/logviewer?job_id=326847444&repo=autoland

Backout: https://hg.mozilla.org/integration/autoland/rev/0695ec653f99879731861e8760894f83c192a229

Comment 16

[Yura Zenevich [:yzen] (please use "needinfo")]

Attachment: Bug 1667998 - [ci] Add cli flag in desktop_unittest.py. Ensure M-a11y-checks runs on Linux x64 opt build. r=jmaher

Comment 17

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/86e35a80ef9d
add AccessibilityUtils for testing accessibility related issues and use it to check accessibility when EventUtils.sendMouseEvent is used to make a click. r=jmaher
https://hg.mozilla.org/integration/autoland/rev/c34b10f55f82
converted calls to EventUtils.sendMouseEvent to be async. r=jmaher
https://hg.mozilla.org/integration/autoland/rev/174b0c3c5cd4
pass accessibility checks for tests that need a11y exceptions. r=jmaher
https://hg.mozilla.org/integration/autoland/rev/5972a5903534
[ci] Create a new 'a11y-checks' test variant. r=jmaher
https://hg.mozilla.org/integration/autoland/rev/ef419d7bc4a5
skip some accessible related browser-chrome tests when running a11y-checks. r=jmaher
https://hg.mozilla.org/integration/autoland/rev/8886702307bd
[ci] Add cli flag in desktop_unittest.py. Ensure M-a11y-checks runs on Linux x64 opt build. r=jmaher

Comment 18

[Natalia Csoregi [:nataliaCs]]

https://hg.mozilla.org/mozilla-central/rev/86e35a80ef9d
https://hg.mozilla.org/mozilla-central/rev/c34b10f55f82
https://hg.mozilla.org/mozilla-central/rev/174b0c3c5cd4
https://hg.mozilla.org/mozilla-central/rev/5972a5903534
https://hg.mozilla.org/mozilla-central/rev/ef419d7bc4a5
https://hg.mozilla.org/mozilla-central/rev/8886702307bd