1919104 - [a11y] TalkBack backtracks from the selected item in the dropdown and focuses on something that shouldn't be focusable

Status: NEW

(Firefox for Android :: Design System and Theming, defect, P2)

Description

[Olivia Hall [:olivia]]

Steps to reproduce

0. Ensure TalkBack is running on an Android device.
1. If the device is in English, navigate to a non-English page that the translations engine supports. For example, www.mozilla.org/fr
2. Open the translations dialog on the toolbar. Labled as "Translate Page".
3. Navigate to the dropdowns.
4. Observe "Translate from" and "Translate to" are prepopulated with a language. For example, "French" and "English".
5. Open the dropdowns and navigate the list. Notice that occasionally the "Selected" item won't report its label on the one before the selected item.

Expected behavior

The selected item should always report its label.

Actual behavior

The selected item does not always report its label.

Comment 1

[Anna Yeddi [:ayeddi]]

While missing out on the content is severe, since the issue is intermittent and there is an alternative way to review the selected option (when the dropdown is closed), I'm triaging it as access-S3, but if the issue would be permanent, the accessibility severity should be increased to s2

Comment 2

[Olivia Hall [:olivia]]

Seems to be happening consistently now when testing on the Pixel 4a Android 14.

What is happening is that TalkBack backtracks to the item before the selected item both in the current version (Nightly 135) and proposed version in bug 1927716 D230604 Diff 9.

For example, open the language dropdown, TalkBack correctly opens to the checked item, perform the TalkBack gesture for next item, focus ring moves above the checked item and says unselected, next gesture continues below the checked item and performs as expected.

The bug is that there is an unexpected unlabled focus movement above the checked item to a non-painted element on the screen using the next gesture on the dropdown.

Comment 3

[Olivia Hall [:olivia]]

Attachment: Nightly-135-01-06-25.mp4

Added a video for context since it is a bit difficult to describe.

Video starts with the translations dropdown open, correctly goes to the selected language of "English".

• Left to right swipe (TalkBack next item gesture) moves focus above the selected item without moving the view to "Not selected. Button."
• Next left to right swipe moves to what was originally expected to "Estonian".

Comment 4

[Olivia Hall [:olivia]]

It seems overall that the "next item" and "previous item" gesture response could be improved here as well. The gesture seems to want to scroll a full four languages up and then navigate through them instead of a singular item. Not a very smooth experience.

I'm moving this to s2 since this now seems to be happening consistently and the experience isn't smooth overall.

Comment 5

[Olivia Hall [:olivia]]

Just adding a summary to this bug because the issue has evolved some since originally reported:

• Primary issue is now that a next item gesture (left to right on TalkBack) when starting from the checked selected item will advance before the checked item instead of below it. (See Comment 3 for example.)
• The item it advances to incorrectly is unlabled. Presumably the node TalkBack focus on is either the language before the selected item and isn't reporting the label because it isn't painted on screen. Or possibly some node or view used in displaying the dropdown that needs to be unnavigable.
  • From what I recall in the past, this unlabled element would sometimes show up, but not consistently as now.
• There is some general unsmooth behavior around using next item (left to right gesture) in the dropdown that could be improved.

Ideas to pursue to fix:

• Try to identify what the element is that TalkBack is focusing on. It might be as simple as ignoring that node if it is a view or something just for layout purposes.
• When TalkBack does advance to the next set of languages appropriately, it seems to advance too many at a time. Ensure it only advances one language at a time might also be a solution that improves fluidity. It almost looks like focus could be catching the corner of the language before the selected item.
• It might be an issue in the focus order of the language elements.
• Should likely audit other uses of the dropdown selector during testing too (in case this isn't just the translations instance).

Should wait for bug 1927716 to complete before pursuing this bug because that bug changes the dropdown implementation. We need to confirm the issue still fully persists after that change. However, from what I could tell when reviewing that patch (as of diff 9), the same issues will persist in the new implementation. So, the core issue seems to likely be translations dropdown usage specific.

Comment 6

[Segun Famisa [:segun]]

Attachment: preview

So I have spent some time looking at this bug, but it seems really non-trivial to solve. As I understand, we want to
1. We want to automatically focus on the currently selected item
2. We want to retain natural order of scrolling (upwards or downwards) whenever the user starts interacting with the list.

From my investigation, this has proven to be non trivial, because even when we force the traversal order to prioritize the selected item, talkback still focuses on the first item in the list.

I tested this outside of Fenix, using the [MenuWithScrollStateSample](https://cs.android.com/androidx/platform/frameworks/support/+/androidx-main:compose/material/material/samples/src/main/java/androidx/compose/material/samples/MenuSamples.kt;l=59-84?q=MenuWithScrollState&ss=androidx%2Fplatform%2Fframeworks%2Fsupport) in Andoid source. I adapted it to look similar to what we are doing - forcing the traversal order, and pre-selecting an item somewhere in the middle, and talkback always stayed on the first visible item in the list.

The only hack that seemed to work intermittently was adding a focus requester on the selected item, and then add an arbitrary delay (1 second in my case) before scrolling to the selected item, and requesting focus. This is definitely something that I do not recommend but I wanted to demonstrate that this seems to be a platform problem with the way talkback works.

```
@Preview
@Composable
fun Test() {
    val selectedIndex = 18
    val focusRequester = FocusRequester()
    var expanded by remember { mutableStateOf(false) }
    val scrollState = rememberScrollState()
    Box(
        modifier = Modifier
            .fillMaxSize()
            .wrapContentSize(Alignment.TopStart)
    ) {
        IconButton(onClick = { expanded = true }) {
            Icon(Icons.Default.MoreVert, contentDescription = "Localized description")
        }
        DropdownMenu(
            modifier = Modifier.requiredSize(width = 300.dp, height = 400.dp),
            expanded = expanded,
            onDismissRequest = { expanded = false },
            scrollState = scrollState
        ) {
            repeat(30) { index ->
                val isSelected = index == selectedIndex
                DropdownMenuItem(
                    modifier = Modifier
                        .then(
                            if (isSelected) {
                                Modifier.focusRequester(focusRequester)
                            } else {
                                Modifier
                            }
                        )
                        .focusable(enabled = true)
                        .selectable(
                            selected = isSelected,
                            onClick = {}
                        ),
                    onClick = { /* Handle item! */ },
                    text = { Text("Item $index") }
                )
            }
        }
        LaunchedEffect(expanded) {
            if (expanded) {
                // Scroll to the selected index
                scrollState.scrollTo(selectedIndex)
                delay(1000) // no please!
                focusRequester.requestFocus()
            }
        }
    }
}
```

Comment 7

[Segun Famisa [:segun]]

I had a look at a bunch of apps, and I don't quite have any app that does this. Most of the apps I went through list the selected item first on the list, and then create a section for the "rest of the list".

I wonder what the thoughts are on this

In addition, I am curious to know - how big a problem this is from the perspective of someone who uses talkback, especially since it seems to be a system behavior? Maybe it is something that is expected from talkback users.

Comment 8

[Olivia Hall [:olivia]]

Thanks so much for the thorough investigation, Segun!

How big a problem this is from the perspective of someone who uses talkback, especially since it seems to be a system behavior? Maybe it is something that is expected from talkback users.

I agree that this issue overall feels like a TalkBack bug. I wouldn't consider it expected behavior, though. Getting the focus order just right is super important and is even a WCAG guideline and the unlabeled mistaken focus is breaking the experience. It just lacks fluidity overall.

To get a sense of how fluid a good experience is in a Desktop paradigm go to this combobox example and turn on VoiceOver on Mac or install NVDA on PC and navigate through only using keyboard commands. PC and Android have different screen reader conventions, but it gives a feel for what the overall fluid user experience should be like. (TalkBack's next item gestures are essentially the equivalent to using tab / arrow keys on Desktop.)

I like the idea of trying a new implementation where it is moved to the front of the dropdown. I bet it will be possible to fix this way and be a good TalkBack user experience. I'll leave that UX assessment to Dasha!

:ayeddi, what are your thoughts? This implementation overall lacks fluidity and has some outright buggy behavior. Comment 5 outlines the current state and Segun has a thorough report in Comment 6.