Note: There are a few cases of duplicates in user autocompletion which are being worked on.

[Mac] Firefox doesn't tell VoiceOver when a page has finished loading.

RESOLVED FIXED in Firefox 44

Status

()

Core
Disability Access APIs
P1
normal
RESOLVED FIXED
6 years ago
2 years ago

People

(Reporter: MarcoZ, Assigned: fredw)

Tracking

(Blocks: 1 bug)

Trunk
mozilla44
x86_64
Mac OS X
Points:
---

Firefox Tracking Flags

(firefox44 fixed)

Details

Attachments

(1 attachment, 3 obsolete attachments)

(Reporter)

Description

6 years ago
STR:
1. Run accessibility-enabled build of Firefox on OS X with VoiceOver.
2. Open a page.
Expected: When the page finishes loading, you should hear an ascending three tone sound effect, and VoiceOver should start reading.
Actual: VoiceOver remains silent, and only by navigation do you notice that the new content is available.

Safari somehow notifies VoiceOver when it has finished loading the page, similar to events of doc load complete etc. on Windows.
(Reporter)

Comment 1

6 years ago
This is very important that we notify VoiceOver that it can update its view of things. Otherwise users will never know when a page is loaded.
Priority: -- → P1

Updated

6 years ago
Assignee: nobody → hub
Created attachment 613830 [details] [diff] [review]
Notify that the document is loaded. r=
Suggested patch. We get the notification from Gecko, but then there is no output in VoiceOver. Chrome doesn't either so I'm not sure what's appropriate here. Attaching for now.
(Reporter)

Comment 4

5 years ago
Comment on attachment 613830 [details] [diff] [review]
Notify that the document is loaded. r=

I built with this patch locally, and am sorry to report that there's no difference to previous behavior. VoiceOver still doesn't get notified correctly that a new document loaded.
Attachment #613830 - Flags: feedback-
That's what I said. VoiceOver does nothing. Not sure of the reason. I'll ask the Mac a11y list.
According to this reply in the mac accessibility mailing list, the chime in Safari is actually performed by Safari by playing a sound back.

http://prod.lists.apple.com/archives/accessibility-dev/2012/Apr/msg00007.html
(Reporter)

Comment 7

5 years ago
(In reply to Hub Figuiere [:hub] from comment #6)
> According to this reply in the mac accessibility mailing list, the chime in
> Safari is actually performed by Safari by playing a sound back.
> 
> http://prod.lists.apple.com/archives/accessibility-dev/2012/Apr/msg00007.html

Yes, but there are more things that happen once this sound is played. If set in the VoiceOver settings (on by default), VoiceOver then focuses the web area and automatically starts reading the page. Perhaps it takes the sound as a queue to do that, but I would think it would rather react to that notification postmessage....

Updated

4 years ago
Assignee: hub → nobody
(Assignee)

Comment 8

2 years ago
Created attachment 8666747 [details] [diff] [review]
Patch

Just updating Hubert's patch.

It seems that notifications sent by WebKit has changed:
http://trac.webkit.org/browser/trunk/Source/WebCore/dom/Document.cpp#L2697
(Assignee)

Comment 9

2 years ago
@Marco: can you please try this one (I added the AXLayoutComplete notification):

http://ftp.mozilla.org/pub/mozilla.org/firefox/try-builds/fred.wang@free.fr-e55a7bf751a3/try-macosx64/firefox-44.0a1.en-US.mac.dmg
Flags: needinfo?(mzehe)
(Reporter)

Comment 10

2 years ago
I'm not seeing any difference in behavior with this try build unfortunately. :(
Flags: needinfo?(mzehe)
(Assignee)

Comment 11

2 years ago
What about that one:

http://ftp.mozilla.org/pub/mozilla.org/firefox/try-builds/fred.wang@free.fr-de96135319b4/try-macosx64/firefox-44.0a1.en-US.mac.dmg

I notify NSAccessibilityFocusedUIElementChangedNotification on the document root, hopefully that will help to get focus.

Other public notifications on https://developer.apple.com/library/prerelease/mac/documentation/AppKit/Reference/NSAccessibility_Protocol_Reference/index.html seem irrelevant or are not used in WebKit code.
Flags: needinfo?(mzehe)
(Reporter)

Comment 12

2 years ago
Nope, this is only slightly better in that it now sets focus to the document, but when activating a link, I still don't hear when the new page has finished loading. What about that LayoutChangedNotification that you mentioned? Are we firing that or not?
Flags: needinfo?(mzehe)
(Assignee)

Comment 13

2 years ago
(In reply to Marco Zehe (:MarcoZ) from comment #12)
> Nope, this is only slightly better in that it now sets focus to the
> document,

Yes, that's what I wanted to do with this latest change.

> but when activating a link, I still don't hear when the new page
> has finished loading.

IIUC the answer by Apple dev (comment 6), we'll have to play the sound ourselves rather than sending a notification to VoiceOver to ask it to play the sound ; I guess this is doable. However, I'm not sure how we can do the "automatically starts reading the page" mentioned in comment 7 without explicitly sending a notification to VoiceOver.

> What about that LayoutChangedNotification that you
> mentioned? Are we firing that or not?

This is what is fired in the latest version:
NSAccessibilityPostNotification(realSelf, @"AXLoadComplete");
NSAccessibilityPostNotification(realSelf, @"AXLayoutComplete");
NSAccessibilityPostNotification(realSelf, NSAccessibilityFocusedUIElementChangedNotification);
(Reporter)

Comment 14

2 years ago
(In reply to Frédéric Wang (:fredw) from comment #13)
> > but when activating a link, I still don't hear when the new page
> > has finished loading.
> 
> IIUC the answer by Apple dev (comment 6), we'll have to play the sound
> ourselves rather than sending a notification to VoiceOver to ask it to play
> the sound ; I guess this is doable. However, I'm not sure how we can do the
> "automatically starts reading the page" mentioned in comment 7 without
> explicitly sending a notification to VoiceOver.

I have a feeling the answer in comment #6 isn't the whole story. Could we take a peek at what Chromium does, since that is a separate project and has different code? It does work, so we should be able to get it to work, too.
(Assignee)

Comment 15

2 years ago
(In reply to Marco Zehe (:MarcoZ) from comment #14)
> I have a feeling the answer in comment #6 isn't the whole story. Could we
> take a peek at what Chromium does, since that is a separate project and has
> different code? It does work, so we should be able to get it to work, too.

Checking the Chromium code didn't help much. I essentially found the same notifications.

I just wrote a mail on the WebKit mailing list:
https://lists.webkit.org/pipermail/webkit-dev/2015-October/027703.html
(Assignee)

Comment 16

2 years ago
So if I understand correctly the answer from Apple developers, this won't work until Firefox is white listed as a "web browser" by VoiceOver. So what about just taking attachment 8666747 [details] [diff] [review] now and test that later if more work is needed.

Apple developers only mentioned "AXLoadComplete", but I guess it is safe to send more notifications than needed. I propose the following order:

1) NSAccessibilityFocusedUIElementChangedNotification
2) "AXLoadComplete"
3) "AXLayoutComplete"

So that VoiceOver will first focus the document (after NSAccessibilityFocusedUIElementChangedNotification) and then we will tell it to play the sound and read the previously focused document (after "AXLoadComplete" and "AXLayoutComplete").
Flags: needinfo?(mzehe)
(Assignee)

Comment 17

2 years ago
Created attachment 8668907 [details] [diff] [review]
Patch V2
Assignee: nobody → fred.wang
Status: NEW → ASSIGNED
(Reporter)

Comment 18

2 years ago
Yup, let's do that!
Flags: needinfo?(mzehe)
(Assignee)

Updated

2 years ago
Attachment #613830 - Attachment is obsolete: true
(Assignee)

Comment 19

2 years ago
Created attachment 8668915 [details] [diff] [review]
Modify Mac Bundle id to claim being Safari so that VoiceOver knows we are a web browser.
(Reporter)

Comment 20

2 years ago
OK, so without the bundle modifications, the patch still doesn't cause VoiceOver to recognize DocumentLoadComplete, but I really suspect this might have something to do with the whitelisting. I'd say let's get this reviewed and landed.
(Assignee)

Updated

2 years ago
Attachment #8668907 - Flags: review?(surkov.alexander)

Comment 21

2 years ago
Comment on attachment 8668907 [details] [diff] [review]
Patch V2

Review of attachment 8668907 [details] [diff] [review]:
-----------------------------------------------------------------

::: accessible/mac/AccessibleWrap.mm
@@ +100,5 @@
>  
>    uint32_t eventType = aEvent->GetEventType();
>  
> +  // ignore everything but focus-changed, value-changed, caret, selection
> +  // and document load complete events for now.

eventually this comment will get huge :)
Attachment #8668907 - Flags: review?(surkov.alexander) → review+
(Assignee)

Updated

2 years ago
Attachment #8666747 - Attachment is obsolete: true
(Assignee)

Updated

2 years ago
Attachment #8668915 - Attachment is obsolete: true
(Assignee)

Comment 22

2 years ago
https://treeherder.mozilla.org/#/jobs?repo=try&revision=39335b8bc2b8
(Assignee)

Updated

2 years ago
Keywords: checkin-needed

Comment 23

2 years ago
https://hg.mozilla.org/integration/mozilla-inbound/rev/2a6d058f82d1
Keywords: checkin-needed
https://hg.mozilla.org/mozilla-central/rev/2a6d058f82d1
Status: ASSIGNED → RESOLVED
Last Resolved: 2 years ago
status-firefox44: --- → fixed
Resolution: --- → FIXED
Target Milestone: --- → mozilla44
You need to log in before you can comment on or make changes to this bug.