1888310 - VoiceOver fails to announce announcements from A11yUtils.announce

Status: VERIFIED FIXED

(Core :: Disability Access APIs, defect, P3)

Description

[Morgan Reschenberg [:morgan]]

STR:

1. Activate VO
2. Focus the url bar and type "g" or "am"

Expected:
VO reads "Press tab to search with google" or "Press tab to search with amazon" depending on the prefix typed

Actual:
VO reads some version of "g" or "a, m" or tells you about the text autofilled in the URL bar

This is due to a bug in the a11yUtils.announce function we use to communicate this info. We aren't sending AXAnnouncementRequested notifications and the AXLiveRegionChanged notifications aren't working (though we expect that here, due to VO's various quirks)

Comment 1

[Morgan Reschenberg [:morgan]]

Attachment: Bug 1888310: Verify a11y-announcement is a child of ChildView instead of mozRootAccessible r?eeejay

Previously we checked if the announcement was a child of the root acc directly, but it's not clear this worked reliably since we've done GetObjectOrRepresentedView for a while, which (should) return the view instead of the root acc. This patch also:

• Dispatches the announcement from NSApp instead of NSWindow, since per chrome and safari notifs fired on non-main windows get dropped
• Modifies the announcement priority from medium to high, so VO interrupts itself to speak this message (this makes the UX more consistent, since the text-inserted/text-deleted notifs from the URL bar seem to occasionally bookend the announcement)
• Updates the browser_app.js test to NOT run in headless mode, since in headless mode NSApp isn't rendered to dispatch the notification. This test contains a task for AXAnnouncementRequested via a11yUtils.announce

Comment 2

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/c9a46c22d5ff
Verify a11y-announcement is a child of ChildView instead of mozRootAccessible r=eeejay

Comment 3

[Cristian Tuns]

Backed out for causing build bustages in mozAccessible.mm

• Backout link
• Push with failures
• Failure Log
• Failure line: /builds/worker/checkouts/gecko/accessible/mac/mozAccessible.mm:886:34: error: '&&' within '||' [-Werror,-Wlogical-op-parentheses]

Comment 4

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/8c8df548d15d
Verify a11y-announcement is a child of ChildView instead of mozRootAccessible r=eeejay

Comment 5

[Natalia Csoregi [:nataliaCs]]

Backed out with Bug 1884625 for causing failures on /browser_app.js

• backout: https://hg.mozilla.org/integration/autoland/rev/7ed2595827888ad532d19d2cf3ae701994f83d6c
• push: https://treeherder.mozilla.org/jobs?repo=autoland&group_state=expanded&selectedTaskRun=ZJmMX9MvQP-kH6G4U6yvvA.0&revision=8036a50c07513fcac811cd76f460cbcd2fc08dcf
• failure log: https://treeherder.mozilla.org/logviewer?job_id=454888645&repo=autoland&lineNumber=1416

[task 2024-04-17T19:28:47.740Z] 19:28:47     INFO - TEST-PASS | accessible/tests/browser/mac/browser_app.js | Found alert to send announcements - 
[task 2024-04-17T19:28:47.740Z] 19:28:47     INFO - Buffered messages finished
[task 2024-04-17T19:28:47.741Z] 19:28:47     INFO - TEST-UNEXPECTED-FAIL | accessible/tests/browser/mac/browser_app.js | Test timed out - 
[task 2024-04-17T19:28:47.741Z] 19:28:47     INFO - GECKO(827) | Completed ShutdownLeaks collections in process 827

Comment 6

[Eitan Isaacson [:eeejay]]

Added suggestions in the abandoned revision.

Comment 7

[Eitan Isaacson [:eeejay]]

I tried the STR above with the changes I suggested, and the a11y event is sent to MacOS, but all i hear is "selection replaced". seems like the text editing/value changes get priority? not sure what do do with that.

Comment 8

[Morgan Reschenberg [:morgan]]

(In reply to Eitan Isaacson [:eeejay] from comment #7)

I tried the STR above with the changes I suggested, and the a11y event is sent to MacOS, but all i hear is "selection replaced". seems like the text editing/value changes get priority? not sure what do do with that.

I think the event only fires after the first characters that uniquely identify the search to fx, so if you type more than "g" or "am", VO speaks over it.
I get VO announcing "Press tab to search with google" or "press tab to search with amazon" routinely with that STR.

Comment 9

[Morgan Reschenberg [:morgan]]

Updated patch, pushed a new try run here https://treeherder.mozilla.org/jobs?repo=try&revision=1fc56e75432e00e97039f2221f4d0b9cf4c5a3ab

Comment 10

[Morgan Reschenberg [:morgan]]

https://treeherder.mozilla.org/jobs?repo=try&revision=0bac2b76f495f7bf12753634b1101f88d91ae561

Comment 11

[Morgan Reschenberg [:morgan]]

try looks good, attempting to land this again

Comment 12

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/29e984633f5e
Verify a11y-announcement is a child of ChildView instead of mozRootAccessible r=eeejay

Comment 13

[Sandor Molnar[:smolnar]]

Backed out for causing mochitest failures @ accessible/tests/browser/mac/browser_app.js

Backout link: https://hg.mozilla.org/integration/autoland/rev/b442a3f071ed3cd0497fab7b87bbea0874039c8f

Push with failures

Failure log -> TEST-UNEXPECTED-FAIL | accessible/tests/browser/mac/browser_app.js | Test timed out

Comment 14

[Morgan Reschenberg [:morgan]]

attempting to fire the test event on the root instead of firing it on nsapp -- https://treeherder.mozilla.org/jobs?repo=try&revision=7d4cd8730c6f3d0eb95c856036dd98323b10a9d6

Comment 15

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/718208a40b89
Verify a11y-announcement is a child of ChildView instead of mozRootAccessible r=eeejay

Comment 16

[Pulsebot]

Backout by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/a849f33583e5
Backed out changeset 718208a40b89 for causing mochitest failures @ browser_app.js CLOSED TREE

Comment 17

[Sandor Molnar[:smolnar]]

Backed out for causing mochitest failures @ browser_app.js

Backout link: https://hg.mozilla.org/integration/autoland/rev/a849f33583e596613504ab75fe5817e39cc9558b

Push with failures

Failure log -> TEST-UNEXPECTED-FAIL | accessible/tests/browser/mac/browser_app.js | Test timed out

Comment 18

[BugBot [:suhaib / :marco]]

There is an r+ patch which didn't land and no activity in this bug for 2 weeks.
:morgan, could you have a look please?
If you still have some work to do, you can add an action "Plan Changes" in Phabricator.
For more information, please visit BugBot documentation.

Comment 19

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/759bf0712a40
Verify a11y-announcement is a child of ChildView instead of mozRootAccessible r=eeejay

Comment 20

[Cristian Tuns]

https://hg.mozilla.org/mozilla-central/rev/759bf0712a40

Comment 21

[Simona Badau, Desktop Test Engineering]

I can confirm that when typing "g" in the URL bar, VoiceOver reads "Press tab to search with Google" - verified using Firefox 129.0b2 and the latest Nightly 130.0a1 on macOS 13.

However, the old behavior is encountered when typing "am" in the URL bar.
@Morgan, is this expected behavior? Are there any additional settings needed here?

Comment 22

[Morgan Reschenberg [:morgan]]

(In reply to Simona Badau, Desktop QA from comment #21)

I can confirm that when typing "g" in the URL bar, VoiceOver reads "Press tab to search with Google" - verified using Firefox 129.0b2 and the latest Nightly 130.0a1 on macOS 13.

However, the old behavior is encountered when typing "am" in the URL bar.
@Morgan, is this expected behavior? Are there any additional settings needed here?

Does it work if you just type "a"? I think it may depend on your top site/most visited, and I forget if we pre-populate any other "a" sites apart from amazon :)

Comment 23

[Simona Badau, Desktop Test Engineering]

(In reply to Morgan Reschenberg [:morgan] from comment #22)

Does it work if you just type "a"? I think it may depend on your top site/most visited, and I forget if we pre-populate any other "a" sites apart from amazon :)

It does work with "a" but only after setting the preference "browser.search.region" to US in about:config. I suppose it's intended since when the preference "browser.search.region" is set to RO, I don't even see Amazon among the available search engines (typing "b" for Bing and "d" for DuckDuckGo works properly). Verified using the Firefox 129.0b4 and Nightly 130.a1 on macOS 13.

Morgan, I also noticed that VoiceOver reads "Press Tab to search Google" and not "Press Tab to search with Google". Is that okay?

Comment 24

[Morgan Reschenberg [:morgan]]

(In reply to Simona Badau, Desktop QA from comment #23)

(In reply to Morgan Reschenberg [:morgan] from comment #22)

Does it work if you just type "a"? I think it may depend on your top site/most visited, and I forget if we pre-populate any other "a" sites apart from amazon :)

It does work with "a" but only after setting the preference "browser.search.region" to US in about:config. I suppose it's intended since when the preference "browser.search.region" is set to RO, I don't even see Amazon among the available search engines (typing "b" for Bing and "d" for DuckDuckGo works properly). Verified using the Firefox 129.0b4 and Nightly 130.a1 on macOS 13.

Morgan, I also noticed that VoiceOver reads "Press Tab to search Google" and not "Press Tab to search with Google". Is that okay?

Oh my mistake, I didn't realise the amazon pre-populated result was region specific. Thanks for bringing that to my attention :)

Yep! I must've misheard the string. Thanks for your patience!

Comment 25

[Simona Badau, Desktop Test Engineering]

Thanks Morgan!

Based on Comment 23 and Comment 24, setting this as Verified.