[Mac] Firefox doesn't tell VoiceOver when a page has finished loading.

RESOLVED FIXED in Firefox 44

Status

()

defect
P1
normal
RESOLVED FIXED
8 years ago
4 years ago

People

(Reporter: MarcoZ, Assigned: fredw)

Tracking

(Blocks 1 bug)

Trunk
mozilla44
x86_64
macOS
Points:
---
Dependency tree / graph

Firefox Tracking Flags

(firefox44 fixed)

Details

Attachments

(1 attachment, 3 obsolete attachments)

STR:
1. Run accessibility-enabled build of Firefox on OS X with VoiceOver.
2. Open a page.
Expected: When the page finishes loading, you should hear an ascending three tone sound effect, and VoiceOver should start reading.
Actual: VoiceOver remains silent, and only by navigation do you notice that the new content is available.

Safari somehow notifies VoiceOver when it has finished loading the page, similar to events of doc load complete etc. on Windows.
This is very important that we notify VoiceOver that it can update its view of things. Otherwise users will never know when a page is loaded.
Priority: -- → P1
Assignee: nobody → hub
Suggested patch. We get the notification from Gecko, but then there is no output in VoiceOver. Chrome doesn't either so I'm not sure what's appropriate here. Attaching for now.
Comment on attachment 613830 [details] [diff] [review]
Notify that the document is loaded. r=

I built with this patch locally, and am sorry to report that there's no difference to previous behavior. VoiceOver still doesn't get notified correctly that a new document loaded.
Attachment #613830 - Flags: feedback-
That's what I said. VoiceOver does nothing. Not sure of the reason. I'll ask the Mac a11y list.
According to this reply in the mac accessibility mailing list, the chime in Safari is actually performed by Safari by playing a sound back.

http://prod.lists.apple.com/archives/accessibility-dev/2012/Apr/msg00007.html
(In reply to Hub Figuiere [:hub] from comment #6)
> According to this reply in the mac accessibility mailing list, the chime in
> Safari is actually performed by Safari by playing a sound back.
> 
> http://prod.lists.apple.com/archives/accessibility-dev/2012/Apr/msg00007.html

Yes, but there are more things that happen once this sound is played. If set in the VoiceOver settings (on by default), VoiceOver then focuses the web area and automatically starts reading the page. Perhaps it takes the sound as a queue to do that, but I would think it would rather react to that notification postmessage....
Assignee: hub → nobody
Posted patch Patch (obsolete) — Splinter Review
Just updating Hubert's patch.

It seems that notifications sent by WebKit has changed:
http://trac.webkit.org/browser/trunk/Source/WebCore/dom/Document.cpp#L2697
@Marco: can you please try this one (I added the AXLayoutComplete notification):

http://ftp.mozilla.org/pub/mozilla.org/firefox/try-builds/fred.wang@free.fr-e55a7bf751a3/try-macosx64/firefox-44.0a1.en-US.mac.dmg
Flags: needinfo?(mzehe)
I'm not seeing any difference in behavior with this try build unfortunately. :(
Flags: needinfo?(mzehe)
What about that one:

http://ftp.mozilla.org/pub/mozilla.org/firefox/try-builds/fred.wang@free.fr-de96135319b4/try-macosx64/firefox-44.0a1.en-US.mac.dmg

I notify NSAccessibilityFocusedUIElementChangedNotification on the document root, hopefully that will help to get focus.

Other public notifications on https://developer.apple.com/library/prerelease/mac/documentation/AppKit/Reference/NSAccessibility_Protocol_Reference/index.html seem irrelevant or are not used in WebKit code.
Flags: needinfo?(mzehe)
Nope, this is only slightly better in that it now sets focus to the document, but when activating a link, I still don't hear when the new page has finished loading. What about that LayoutChangedNotification that you mentioned? Are we firing that or not?
Flags: needinfo?(mzehe)
(In reply to Marco Zehe (:MarcoZ) from comment #12)
> Nope, this is only slightly better in that it now sets focus to the
> document,

Yes, that's what I wanted to do with this latest change.

> but when activating a link, I still don't hear when the new page
> has finished loading.

IIUC the answer by Apple dev (comment 6), we'll have to play the sound ourselves rather than sending a notification to VoiceOver to ask it to play the sound ; I guess this is doable. However, I'm not sure how we can do the "automatically starts reading the page" mentioned in comment 7 without explicitly sending a notification to VoiceOver.

> What about that LayoutChangedNotification that you
> mentioned? Are we firing that or not?

This is what is fired in the latest version:
NSAccessibilityPostNotification(realSelf, @"AXLoadComplete");
NSAccessibilityPostNotification(realSelf, @"AXLayoutComplete");
NSAccessibilityPostNotification(realSelf, NSAccessibilityFocusedUIElementChangedNotification);
(In reply to Frédéric Wang (:fredw) from comment #13)
> > but when activating a link, I still don't hear when the new page
> > has finished loading.
> 
> IIUC the answer by Apple dev (comment 6), we'll have to play the sound
> ourselves rather than sending a notification to VoiceOver to ask it to play
> the sound ; I guess this is doable. However, I'm not sure how we can do the
> "automatically starts reading the page" mentioned in comment 7 without
> explicitly sending a notification to VoiceOver.

I have a feeling the answer in comment #6 isn't the whole story. Could we take a peek at what Chromium does, since that is a separate project and has different code? It does work, so we should be able to get it to work, too.
(In reply to Marco Zehe (:MarcoZ) from comment #14)
> I have a feeling the answer in comment #6 isn't the whole story. Could we
> take a peek at what Chromium does, since that is a separate project and has
> different code? It does work, so we should be able to get it to work, too.

Checking the Chromium code didn't help much. I essentially found the same notifications.

I just wrote a mail on the WebKit mailing list:
https://lists.webkit.org/pipermail/webkit-dev/2015-October/027703.html
So if I understand correctly the answer from Apple developers, this won't work until Firefox is white listed as a "web browser" by VoiceOver. So what about just taking attachment 8666747 [details] [diff] [review] now and test that later if more work is needed.

Apple developers only mentioned "AXLoadComplete", but I guess it is safe to send more notifications than needed. I propose the following order:

1) NSAccessibilityFocusedUIElementChangedNotification
2) "AXLoadComplete"
3) "AXLayoutComplete"

So that VoiceOver will first focus the document (after NSAccessibilityFocusedUIElementChangedNotification) and then we will tell it to play the sound and read the previously focused document (after "AXLoadComplete" and "AXLayoutComplete").
Flags: needinfo?(mzehe)
Posted patch Patch V2Splinter Review
Assignee: nobody → fred.wang
Status: NEW → ASSIGNED
Yup, let's do that!
Flags: needinfo?(mzehe)
Attachment #613830 - Attachment is obsolete: true
OK, so without the bundle modifications, the patch still doesn't cause VoiceOver to recognize DocumentLoadComplete, but I really suspect this might have something to do with the whitelisting. I'd say let's get this reviewed and landed.
Attachment #8668907 - Flags: review?(surkov.alexander)
Comment on attachment 8668907 [details] [diff] [review]
Patch V2

Review of attachment 8668907 [details] [diff] [review]:
-----------------------------------------------------------------

::: accessible/mac/AccessibleWrap.mm
@@ +100,5 @@
>  
>    uint32_t eventType = aEvent->GetEventType();
>  
> +  // ignore everything but focus-changed, value-changed, caret, selection
> +  // and document load complete events for now.

eventually this comment will get huge :)
Attachment #8668907 - Flags: review?(surkov.alexander) → review+
Attachment #8666747 - Attachment is obsolete: true
Attachment #8668915 - Attachment is obsolete: true
Keywords: checkin-needed
https://hg.mozilla.org/mozilla-central/rev/2a6d058f82d1
Status: ASSIGNED → RESOLVED
Closed: 4 years ago
Resolution: --- → FIXED
Target Milestone: --- → mozilla44
You need to log in before you can comment on or make changes to this bug.