1870281 - The NVDA will read the cards behind the Onboarding card as if the user is opted in after Turning it OFF from Settings

Status: ASSIGNED

(Firefox :: Shopping, defect, P3)

Description

[Rares Doghi, Desktop QA]

Found in

• Nightly 122.0a1 (2023-12-15)

Affected versions

• Nightly 122.0a1 (2023-12-15)

Affected platforms

• windows

Preconditions:
Set the browser.shopping.experience2023.enabled - TRUE
Set the browser.shopping.experience2023.optedIn - 0

Enable NVDA.

Steps to reproduce

1. Reach Amazon.com/p
2. Select the first available item in order to reach the product details page.
3. Use Keyboard navigation in order to reach the X Close button from the Review Checker.
4. Hit Down arrow on the keyboard in order to read the Opt-in card.
5. Use the Tab key to reach the Yes Try it ! button and select it with Enter
6. Use Keyboard navigation in order to Reach the Settings and Turn off the Review Checker using ENTER.
7. Open the Review Checker from the Address bar.
8. Reach the X Close button from the Review Checker.
9. Hit the Down arrow button on the keyboard in order to read the Onboarding card

Expected result

• The Content from the Onboarding card should be read out loud by NVDA.

Actual result

• The Information from the already opted in Cards and Check now button is read by NVDA.

Regression range
19:30.51 INFO: Last good revision: 3152110c63b5c99048b0651867ecf7723aa5ddf8
19:30.51 INFO: First bad revision: 31a6430ad25b20a75a4b299efea5d8ca94c2dfe9
19:30.51 INFO: Pushlog:
https://hg.mozilla.org/mozilla-central/pushloghtml?fromchange=3152110c63b5c99048b0651867ecf7723aa5ddf8&tochange=31a6430ad25b20a75a4b299efea5d8ca94c2dfe9

Comment 1

[BugBot [:suhaib / :marco/ :calixte]]

:bobowen, since you are the author of the regressor, bug 1863914, could you take a look?

For more information, please visit BugBot documentation.

Comment 2

[Bob Owen (:bobowen)]

Hi Rares, are you sure about the regressions range here, because that changeset is for the backing out of my patch?
It landed again on Sat, 09 Dec 2023 09:33:28.

Comment 3

[Rares Doghi, Desktop QA]

I redid the regression range for this issue and I got these results this time:
41:57.92 INFO: Last good revision: c148a4f270cac68e05cbb78152305e9279c16f6a
41:57.93 INFO: First bad revision: f7b6b2768fa6493e56a566d4e8e6bc934afc8f75
41:57.93 INFO: Pushlog:
https://hg.mozilla.org/integration/autoland/pushloghtml?fromchange=c148a4f270cac68e05cbb78152305e9279c16f6a&tochange=f7b6b2768fa6493e56a566d4e8e6bc934afc8f75

It seems that Bug 1869195 might be the issue causing it.

Comment 4

[BugBot [:suhaib / :marco/ :calixte]]

Set release status flags based on info from the regressing bug 1869195

Comment 5

[Donal Meehan [:dmeehan]]

Adding a ni to the regressor assignee for investigation, :mkmelin see Comment 3.

:jhirsch could this be triaged in terms of impact on Fx122?
The pref mentioned in Comment 0 is disabled, though it might be needed for an experiment. Wondering if this needs a fix in Fx122 or can be dealt with later?

Comment 6

[Magnus Melin [:mkmelin]]

Are you sure about this regression range? The patch in 1869195 would only change behavior in that it would not crash. That could of course mean some issues come up elsewhere - but it would have crashed before the patch - not worked.

Comment 7

[Jared Hirsch [:jhirsch] (he/him) (Needinfo please)]

Redirecting the ni? to :ayeddi for accessibility triage

Comment 8

[Jared Hirsch [:jhirsch] (he/him) (Needinfo please)]

The pref mentioned in Comment 0 is disabled, though it might be needed for an experiment

Yes, those prefs are flipped by the shopping experiments to enable the feature, and we aren't enabling this feature beyond a few percent of en-US users in 122.

Comment 9

[Rares Doghi, Desktop QA]

I did other 2 regression ranges and I got different results, I will just leave the exact steps it happens with in our latest Nightly build. I think it might happen at random in other builds I have no idea.

Have NVDA enabled.

1. Launch Firefox.
2. Reach about:config and enabled browser.shopping.experience2023.enabled - true.
3. Open a New Tab and reach amazon.com/p
4. Left Mouse click the Product.
5. Wait a second and Hit Shift + Tab in order to reach the bottom of the page.
6. Use Tab until you reach the X Close button.
7. Hit Down arrow while on the Close button in order to read the onboarding card text.
8. Use tab to Reach the Yes Try it button and select it using Enter.
9. Use Tab key to reach the Settings card and Enter to expand it.
10. Hit Enter on the Turn off Review Checker button.
11. Hit Enter again to Dismiss the Callout.
12. Use Tab in order to reach the Review Checker in the Address bar and Hit Enter to enable it.
13. Use Shift + Tab until you reach the X Close button from the Review Checker.
14. Hit the Down arrow on the Keyboard in order to read the Onboarding Card (NVDA will read out loud the Adjusted rating or any other card from underneath)

Sometimes if the page does not go fully down the issue does not happen.
Sometimes if we dont read the Onboarding card with the Down arrow before opting in, the issue does not happen.
it might just be totally random I think it hates me

I will add a screen recording with the exact steps.

Removing the Regressing issue since I got different results each time, its hard to get an accurate range since it happens randomly in older builds.

Comment 10

[Rares Doghi, Desktop QA]

Attachment: 2023-12-20_11h10_16.mp4

Comment 12

[Anna Yeddi [:ayeddi]]

Attachment: 2024-01-09_still_reproduced.png

I was able to reproduce the bug in the most resent Nightly. As Rares mentioned, it is not consistent, but seems to be more catch-able if you press Tab+Shift quickly after the PDP starts loading - at that moment visually the Review Checker would already be visible, but it is yet to be announced by a screen reader (maybe, because the PDP titles are always long on Amazon, for instance?) and the Tab+Shift would move the focus to the end of the amazon.com PDP (as shown on a screenshot).

Then, the issue would appear at the end of STR, as described: the NVDA announces the Fakespot rating, i.e. C Mix of reliable and unreliable reviews for that product. Pressing Down further would not announce anything (the screen reader would beep communicating that the end of the document is reached and there is nothing after this element - which is not correct).

User would have to guess that it is not true (non-sighted users and users with cognitive difficulties would be the most affected) and press Tab to move to the OMC card.

My theory is that we do not properly communicate that the entire sidebar is changed: only the opt-in part of the Checker, the OMC card has role of alert which is then dynamically removed and may be causing issues for a screen reader, because the live region (which is an alert, in this case) is expected to be in the DOM before and stay after the dynamic changes are happening - this is how it keeps track of changes and communicate them to assistive technology via Accessibility API. Since we remove the alert, it is not announcing any changes for a user and this is where something may stuck in the A11y tree and this is why users would have to Tab to the OMC card.

Maybe we could move the alert role to the checker document itself, so then when its children OMC card is replaced with the checker itself and back all changes are exposed to a11y tree accordingly?

Jared, do you think this could be possible to do?

I'll mark it as access-s3, while it does create a higher severity issue for users who would not know what is, in fact, shown on their screens, but this use case is not as straightforward and the default behavior for re-opting in users seems to work as expected.

Comment 13

[Jared Hirsch [:jhirsch] (he/him) (Needinfo please)]

Thanks so much for investigating, :ayeddi. We have one small round of work ahead, and I'll add this bug to the upcoming metabug.

Comment 14

[Niklas Baumgardner [:niklas]]

Attachment: Bug 1870281 - Move aria live to shopping container. r=#shopping-reviewers,ayeddi