1804699 - [MacOS] Unable to reach and navigate between Datepicker controls using Voice Over commands

Status: VERIFIED FIXED

(Firefox :: Disability Access, defect)

Description

[Rares Doghi, Desktop QA]

Found in

• 109.0a1 (2022-12-07)

Affected versions

• Firefox Nightly 109.0a1 (2022-12-07)

Affected platforms

• Mac

Preconditions

• Enable Voice over - Mac

Steps to reproduce

1. Reach url: data:text/html,<input type='datetime-local'>
2. Use the Voice over commands to reach the Calendar button from the Datepicker.

Expected result

• VO Shift Down/UP, Vo Left/Right arrows and VO Space should work with the Datepicker controls.
• Reaching the Month/Day/Year/Hour/Minute/PM/AM fields should allow the user to change their values with the Up/Down arrow keys, or VO UP/Down ? maybe.

Actual result

• Unable to reach the Datepicker fields or the Calendar button using Voice Over commands.
• Jumping inside the Datepicker using the Tab key will not allow the user to change its Values using Up/Down arrow keys.
• Left/Right arrow keys also do not work when Voice Over is enabled and the user is unable to switch between Month/Day/Year with them.

Regression range
15:44.81 INFO: Last good revision: ab4472f175ecce33ae405f205ac40ef64fbe4a5d
15:44.81 INFO: First bad revision: 5fae4f191812aa1cd68b66e4e5d2e88c1725969e
15:44.81 INFO: Pushlog:
https://hg.mozilla.org/integration/autoland/pushloghtml?fromchange=ab4472f175ecce33ae405f205ac40ef64fbe4a5d&tochange=5fae4f191812aa1cd68b66e4e5d2e88c1725969e

It seems that bug 1798438 is the cause of this issue.

Comment 1

[Rares Doghi, Desktop QA]

Hi @Anna can you take a look at this issue it seems that bug 1798438 is whats causing this.

Comment 2

[BugBot [:suhaib / :marco/ :calixte]]

Set release status flags based on info from the regressing bug 1798438

Comment 3

[Anna Yeddi [:ayeddi]]

Attachment: 2022-12-13_Stable_107.0.1_VoiceOver.mov

The new behavior of the VoiceOver for the Date/Time/Datetime-local inputs are the same as it is in stable release 107.0.1 (refer to the screen recording attached) and it follows the Safari and Chrome on macOS too: while VO+Arrows do not work (the input is referred as text which is the fallback type for datetime inputs), but the same keyboard shortcuts works as for the keyboard-only users: Left and Right moves the focus between the inner spin buttons, similar to the Tab and Shift+Tab and the Up and Down are changing the value of the focused spin button, as expected.

I think we can close this ticket, unless we want to improve the behavior, but then it should be changed to an enhancement. What do you think, :Jamie?

Comment 4

[James Teh [:Jamie]]

It seems like comment 0 and comment 3 disagree on what the current behaviour is. It'd be good to understand whether that's a testing misunderstanding/error or whether there's a real discrepancy there.

I would expect that VO+shift+downArrow would interact with the datetime input and then VO+left/rightArrows should move between the individual controls. The arrow keys should adjust the currently focused control, which comment 3 suggests is already working.

Comment 5

[Rares Doghi, Desktop QA]

Attachment: OlderBehavior.mp4

In previous versions of Firefox we were able to get inside the Datepicker using Vo Shift Down and then move through the fields using Vo Right / Left arrows, in our latest versions of Firefox we can only go through them using Tab / Shift+Tab, if this is intended we can definitely close this one.

I will attach a screen recording of how It worked before in previous versions.

Comment 6

[Anna Yeddi [:ayeddi]]

[Updated with comparison with Fx 107 results]

I retested the behavior with the following results:

System info:

OS:

Mac with MacOS 13.1 Ventura
VO key used on my machine is Ctrl+Option/alt

Browser versions compared:

1. Firefox Release 107.0.1 (64-bit) - the original behavior, before the HTML Markup map bug 1798438 landed - will be referred to as 107
2. Firefox Nightly 111.0a1 (2022-01-23) (64-bit) - the current behavior with the datepicker bugwork landed - will be referred to as 111

Prerequisites:

1. Turn on VoiceOver
2. open a browser on data:text/html,<input type=datetime-local aria-label="First"><input type=date aria-label="Second">

Results:

Regression is confirmed with the script below:

VO+Shift+Down arrow

From the web content frame to move the VO focus on the datetime-local input field. It is announced as follows:

1. 107: announces: 50%, Month, stepper, You are currently on a stepper. To interact with this stepper..., visually the VO focus moves from the "First" datetime input itself to the first spinbutton "mm" inside of this datetime-local input
2. 111: text then, full announcement: First, First, text. First, You are currently on a text, inside of a web content. To enter this date field press...

VO+Right/Left arrow

1. 107: announces: 50%, Month, stepper, You are currently on a stepper. To begin interacting with this stepper, enter this date field..., visually the VO focus moves from the "First" datetime input itself to the first spinbutton "mm" inside of this datetime-local input
2. 111: announces: Second/First, Text. First, You are currently on a text, inside of a web content. To enter this date field..., visually the VO focus moves from the "First" datetime input to the "Second" date input

VO+Shift+Down arrow

1. 107: announces: In stepper. You are currently in a stepper, to decrease its value, press Control-Option-Down Arrow, to increase..., visually the VO focus remains on the "mm" field within the input.
2. 111: announces: In text, visually the VO focus remains on the input field

VO+Right arrow

1. 107: announces: 50, visually the VO focus remains on the "mm" field within the input
   1. For the sake of testing, if exited the stepper by pressing VO+Shift+Up arrow, then the VO+Right Arrow would navigate the VO and its focus to the /, clickable separator
2. 111: announces: End of text, visually the VO focus remains on the input field

VO+Left arrow

1. 107: announces nothing, visually the VO focus remains on the "mm" field within the input
2. 111: announces: Beginning of text visually the VO focus remains on the input field

Right arrow or Left arrow

1. 107: announces: 50%, Day, stepper, You are currently on a stepper. To begin interacting with this stepper, enter this date field..., visually the VO focus moves from first spinbutton "mm" inside of this datetime-local input to the second spinbutton "dd" inside of this datetime-local input
   1. This behavior follows the Tab navigation within the field
2. 111: Nothing happens, visually the VO focus remains on the input field

VO+Shift+Up arrow

1. 107: announces: First, group. You are currently on a group inside of web content..., visually the VO focus remains on the "dd" field within the input field
2. 111: announces: Out of text. First, You are currently on a text..., visually the VO focus remains on the input field
   Vo Left/Right arrows and VO Space should work with the Datepicker controls.

Reaching the Month/Day/Year/Hour/Minute/PM/AM fields

1. 107: By pressing Tab, Right and Left arrows (the keyboard focus moves alongside the VO focus), as well as with VO+Left/Right arrows (in this case the separators are also focused with the VO focus and are announced)
2. 111: Only by pressing Tab while the date input is focused, then all arrows work as expected (without VO)

Up/Down arrow keys, or VO + Up/Down arrow keys

1. 107:
   1. Up/Down arrows alone change the value of spinbuttons, visually the VO focus remains on the input field
   2. [VO-specific bug] VO+Up/Down arrows do not change the value. Even when entering the stepper with VO+Shift+Down arrow and hearing the prompt to use VO+Up/Down arrow keys, this does not change the spinbutton value.
2. 111: Nothing happens, visually the VO focus remains on the input field

Comment 7

[Anna Yeddi [:ayeddi]]

My apologies, I misread the regression and tested with the buggy Fx build - I fixed this now, the comment above is updated with the proper comparison - version 107 vs 111. The bug is confirmed.

Also there is a possible new bug candidate (but existent behavior on 107):

• When a spinner is focused, VO+Up/Down arrows do not change the value. Even when entering the stepper with VO+Shift+Down arrow and hearing the prompt to use VO+Up/Down arrow keys, this does not change the spinbutton value.

Comment 8

[Rares Doghi, Desktop QA]

Hi @Anna, I was able to reproduce the issue mentioned in comment 7 as well but we thought its normal Voice Over behavior since we can reach the Up and Down arrows using VO Controls and then clicking them using VO + Space in order to select a different value from the spinner.

Please let me know if we should log a separate issue for this behavior:

• When a spinner is focused, VO+Up/Down arrows do not change the value. Even when entering the stepper with VO+Shift+Down arrow and hearing the prompt to use VO+Up/Down arrow keys, this does not change the spinbutton value.

Comment 9

[Anna Yeddi [:ayeddi]]

(In reply to Rares Doghi from comment #8)

Hi @Anna, I was able to reproduce the issue mentioned in comment 7 as well but we thought its normal Voice Over behavior since we can reach the Up and Down arrows using VO Controls and then clicking them using VO + Space in order to select a different value from the spinner.

Please let me know if we should log a separate issue for this behavior:

• When a spinner is focused, VO+Up/Down arrows do not change the value. Even when entering the stepper with VO+Shift+Down arrow and hearing the prompt to use VO+Up/Down arrow keys, this does not change the spinbutton value.

@Rares, thank you for confirming the issue! You are right, it is a VO buggy behavior. Just in case, I tested the VO behavior on a date input in Chrome and Safari, and confirmed none of them follow the VO spoken instructions - if you press VO+Up/Down while a spinner is focused within a date/datetime field, nothing happens, only arrows by themselves work on other browsers too. This is a VO bug and we do not need a ticket for that.

I also made a typo in one of the results (thank you, @Jamie, for catching it up!): I updated the comment #6 from:

VO+Shift+Down arrow

From the web content frame to move the VO focus on the datetime-local input field. It is announced as follows:

1. 107: announces: 50%, Month, stepper, You are currently on a text, inside of a web content. To enter this date field..., visually the VO focus moves from the "First" datetime input itself to the first spinbutton "mm" inside of this datetime-local input

To the following (no date field is mentioned by the VO):

VO+Shift+Down arrow

From the web content frame to move the VO focus on the datetime-local input field. It is announced as follows:

1. 107: announces: 50%, Month, stepper, You are currently on a stepper. To interact with this stepper..., visually the VO focus moves from the "First" datetime input itself to the first spinbutton "mm" inside of this datetime-local input

And marked the last item, the one from the comment 7 as a VO-specific bug

Comment 10

[Anna Yeddi [:ayeddi]]

Attachment: Bug 1804699 - Allow VoiceOver to navigate between fields within a datetime-local inputs. r=morgan

Update a DATE_EDITOR role to treat date inputs (<input type=date> and <input type=datetime-local>) as a NSAccessibilityGroupRole to allow VoiceOver scren reader to navigate within these fields using VO+Left/Right Arrow keys.

Note: VO+Up/Down Arrow keys do not change a spinner value currently because of the VoiceOver behavior (which is consistent across browsers) and it is not covered and should not be tested with.

Comment 11

[Anna Yeddi [:ayeddi]]

Morgan did a huge research on the source of the unexpected behavior and has provided the following feedback on the now-obsolete patch in Phab:

• Changing this to a group role has a few unintended consequences -- mainly that we no longer announce this as a "date time field" and instead we call it a "group". This is because we no longer get the role description for free. We can override that and call into the date time role description with something like this. We'd need to create a new accessible class (moxDateTimeAccessible probably) which inherits from mozAccessible and overrides moxRoleDescription. FWIW I'm pretty sure this is what safari does, since they expose date time inputs as AXTextFields but still get the role description right.
• VO is pretty picky when it comes to tree structure, so it's possible the problem isn't so much the role but the way we've structured the mac a11y tree inside elements with AXDateField as their role. In Safari, for ex, the controls are rendered in a group element instead of as direct children of the input itself. It's possible this role/tree structure combo is necessary for VO to be able to get to elements within an input field with an AXTextField role.
• Considering these two solutions, the latter is probably more correct but we'd have to do a decent amount of testing and tree manipulation to verify the tree structure is even a factor. I'm happy with using a group role and overriding role description for now.

Thank you very much for researching this issue, @morgan!

It looks like the core of the issue does fall into the platform side of code which is outside of my expertise and experience. I'm unassigning myself from the bug but including the feedback from Phab here for future reference for a future assignee.

Comment 12

[Morgan Reschenberg [:morgan]]

Attachment: Bug 1804699: Assign date fields an AXGroup role, spoof actual role via AXTitle r?eeejay

Comment 13

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/ba5c608ca0b6
Assign date fields an AXGroup role, spoof actual role via AXTitle r=Jamie

Comment 14

[Sandor Molnar[:smolnar]]

https://hg.mozilla.org/mozilla-central/rev/ba5c608ca0b6

Comment 15

[BugBot [:suhaib / :marco/ :calixte]]

Since nightly and release are affected, beta will likely be affected too.
For more information, please visit auto_nag documentation.

Comment 16

[Rares Doghi, Desktop QA]

Verified as fixed in our latest Nightly build 113.0a1 (2023-03-26) .

Comment 17

[BugBot [:suhaib / :marco/ :calixte]]

The patch landed in nightly and beta is affected.
:morgan, is this bug important enough to require an uplift?

• If yes, please nominate the patch for beta approval.
• If no, please set status-firefox112 to wontfix.

For more information, please visit auto_nag documentation.

Comment 18

[James Teh [:Jamie]]

I don't think we should uplift this, as it has l10n strings.

Comment 19

[Morgan Reschenberg [:morgan]]

Sounds good to me