1496851 - Track length on native HTML5 audio player is not verbalized, only current position.

Status: RESOLVED FIXED

(Toolkit :: Video/Audio Controls, defect, P3)

Description

[charleyroy]

User Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_13_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/69.0.3497.100 Safari/537.36

Steps to reproduce:

Pre-step: Enable screen reader such as NVDA.

Step 1: Go to http://jsfiddle.net/sqdkxy4z/ (or set up a native HTML5 audio with controls)
Step 2: Click the play button to start the audio.
Step 3: Click / focus on the progress bar.



Actual results:

The current progress is verbalized. The total track length is not verbalized.


Expected results:

Both should be verbalized. The track length should at least be verbalized somewhere within the player.

Comment 1

[:Gijs (he/him)]

Marco, can you elaborate on what the ideal state here should be? I'd be very happy to submit a patch, but unsure of what the best case here would end up being - I assume marking the progress bar as labelled by the entire text ("0:05 / 1:45") isn't going to be nice because it'd change all the time, and also because something like "5 seconds into 1 minute 45 second clip" might be nicer (though updating that every second also seems... suboptimal). Then there's the fact that for very short clips, the actual scrubber is much more granular than the "number of seconds elapsed". Should that be reflected in the accessible text? How do we decide when to do that?

I'm also a bit surprised we don't do anything with the value and max value of the html <progress> element in the first place. That is, why isn't NVDA giving the HTML information on the min/max/value of the scrubber? Is that a (separate) bug?

Comment 2

[Marco Zehe (:MarcoZ)]

OK, first, the elapsed time is not the label of the progress bar, but its current value. A label would be something like "Scrubber" or something like this, e. g. stating the purpose of the control. The actual value is the elapsed time. The total length would be the max value, and the min value would be 0 seconds. The value text, however, could be something like 0.5 seconds of 3 minutes 50 seconds elapsed". However, and here's the thing, hearing that over and over in a screen reader kills the audio/video experience, since the screen reader would mercilessly babble over the audio with every update this thing makes, as long as the scrubber has focus. Outside of that, e. g. when a different control has focus, the screen reader might still indicate progress, for example by ascending progress beeps like NVDA does, or at certain intervals repeat the current value, like JAWS does. This is one of those cases where a correct implementation can severely interfere with actual usability.

I am not currently sure why in this case we don't expose the min/max values. I am pretty certain we have tests for regular progress meters. It may also be that NVDA simply doesn't use these values when interacting with progress meters.

I'm NI'ing Jamie who worked on NVDA previously to shed some more light on how NVDA interacts with this type of control by default.

Comment 3

[James Teh [:Jamie]]

TL;DR: For now, I think we should at least expose a proper name on the scrubber (like Scrubber) and value text like "0:05 / 1:45" or "5 seconds of 1 minute 45 seconds". The verbosity problem is probably not something we can fix just yet, but it's also not a regression.

There are a few pieces to the puzzle here.

1. From what I can see, this player actually has two controls which show the position in the media. One has id progressBar, is an HTML <progress> and has a role of progress bar. The other has id scrubber, is an HTMl <input> (probably type="range") and has a role of slider. Which one are we talking about here? And why are there two? What is the difference visually? I would have thought the scrubber would be enough to indicate relative position. I'm going to assume we're talking about #scrubber henceforth.

2. We do expose minimum and maximum values for these controls. However, they're not particularly useful. For progressBar, minimum is 0.0 and maximum is 100.0. For scrubber, minimum is 0.0 and maximum is 45327.0.

3. Accessibility APIs can only expose floating point numbers for minimum, current and maximum values. That essentially means that nothing we expose for those will be useful in this instance.

4. NVDA ignores these numeric values anyway, precisely because they're usually not useful (compared to value text). (The only exception to this in future might be progress bars, where the numeric values might be used to do progress beeps in order to handle the case where value text has been overridden.)

5. So, that means the only relevant thing here is the value text, for the scrubber at least.

6. Now we're getting somewhere. :) That leads us to the question of what to expose. I'd suggest that "0:05 / 1:45" is okay. We could try to format that into pronounceable wording (5 seconds...), but that isn't ideal for braille, for example. There are advantages to both approaches, though, and I'm not personally going to push back on friendly, pronounceable text if that's the consensus. We don't yet have the ability to provide pronunciation hints via accessibility APIs, so that's unfortunately out of the question.

7. Finally, there's the problem of excessive verbosity as this updates.
a) Right now, if scrubber is focused, you get arbitrary numbers speaking continually. So, IMO, changing the value text to something meaningful (as above) is not a *regression* and should be done regardless.
b) We could work around the verbosity problem by suppressing value change events for this control or not updating the value text as it changes while the control is focused. Both of these suck, though. If you don't update the value text, querying the current value will report outdated information. For either approach, braille will always report outdated information, and there's no reason for braille not to update here.
c) Arguably, screen readers shouldn't speak changes to sliders that weren't triggered by the user. That would solve the verbosity problem. We should consider discussing this with screen reader vendors.
d) #progressBar still concerns me, as does #bufferBar. Screen readers report progress bars even if they're not focused,since that information is usually relevant to the user. In this case, though, it's just downright annoying. We probably need some way of indicating that updates to a progress bar are not worthy of the user's immediate attention, but we don't have anything like that yet. We could probably apply that same thing to sliders as alternative solution to c).

Comment 4

[Asa Dotzler [:asa]]

Updating the Accessibility Team's impact assessment to conform with the new triage guidelines. See https://wiki.mozilla.org/Accessibility/Triage for descriptions of these whiteboard flags.

Comment 5

[James Teh [:Jamie]]

Note that the duration/total time is already exposed as text (e.g. 0:00 / 1:10) in bdi.positionDurationBox. That is currently aria-hidden due to bug 1271765 comment 97, under the assumption that this info was already exposed by the scrubber. Unfortunately, the info isn't exposed by the scrubber. We're probably going to need to duplicate the content of bdi.positionDurationBox into the aria-valuetext attribute of the scrubber.

Comment 6

[James Teh [:Jamie]]

Attachment: Bug 1496851: Improve a11y of the position scrubber in the video controls.

1. Label it.

2. Expose the position and duration as its value text.
   The raw numeric value exposed previously wasn't meaningful to users.

3. Hide the progressBar for a11y.
   While the scrubber and progressBar are visually combined, they are exposed as entirely separate controls for a11y.
   As well as duplicating information, progress bars are automatically reported by screen readers even if they aren't focused, which intrudes on the audio being played.

Comment 7

[James Teh [:Jamie]]

Attachment: Bug 1496851 part 3: Don't expose the buffering bar as a progress bar for a11y.

Comment 8

[James Teh [:Jamie]]

Attachment: Bug 1496851 part 1: Enable videocontrols to use Fluent.

Comment 9

[James Teh [:Jamie]]

Attachment: Bug 1496851 part 2: Improve a11y of the position scrubber in the video controls.

1. Label it.

2. Expose the position and duration as its value text.
   The raw numeric value exposed previously wasn't meaningful to users.

3. Hide the progressBar for a11y.
   While the scrubber and progressBar are visually combined, they are exposed as entirely separate controls for a11y.
   As well as duplicating information, progress bars are automatically reported by screen readers even if they aren't focused, which intrudes on the audio being played.

Comment 10

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/0aa1e618b7a3
part 1: Enable videocontrols to use Fluent. r=fluent-reviewers,Gijs
https://hg.mozilla.org/integration/autoland/rev/c109733bb089
part 2: Improve a11y of the position scrubber in the video controls. r=MarcoZ,fluent-reviewers,Gijs
https://hg.mozilla.org/integration/autoland/rev/b86f18e1e4db
part 3: Don't expose the buffering bar as a progress bar for a11y. r=MarcoZ,fluent-reviewers,Gijs

Comment 11

[Atila Butkovits]

https://hg.mozilla.org/mozilla-central/rev/0aa1e618b7a3
https://hg.mozilla.org/mozilla-central/rev/c109733bb089
https://hg.mozilla.org/mozilla-central/rev/b86f18e1e4db