Closed Bug 1828295 Opened 1 year ago Closed 11 months ago

Latest release (111.0.1 (64-bit)) no longer works correctly with Windows Magnifier (Windows 10)

Categories

(Core :: Disability Access APIs, defect, P2)

Firefox 111
defect

Tracking

()

RESOLVED FIXED
115 Branch
Tracking Status
relnote-firefox --- 115+
firefox115 --- fixed

People

(Reporter: wordwizardwizard, Assigned: Jamie)

Details

Attachments

(3 files)

User Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:109.0) Gecko/20100101 Firefox/111.0

Steps to reproduce:

I am severely visually impaired and use the Windows magnifier in docked mode. Since the most recent update of Firefox, the text insertion point is no longer centred in an edit box. In this current edit box, for instance, the insertion point is centred, but slightly off the top of the magnifier screen, so I am currently reading about 3/4 of the height of the letters. In the search box of the Google main page, the magnifier no longer tracks the insertion point until you type more than one line of text. The behaviour is still correct in Chrome and Edge, so it is not an OS issue.

The same is true of the address box, which no longer tracks the insertion point.
I have used the magnifier with Firefox for many years (since the early days of Windows 7), and this new and frustrating behaviour is new to the latest update.

Actual results:

Windows Magnifier failed to track the insertion point correctly in Firefox.

Expected results:

With my magnifier settings, the insertion point (text cursor) should always be centred in the docked magnifier.

The Bugbug bot thinks this bug should belong to the 'Core::Widget: Win32' component, and is moving the bug to that component. Please correct in case you think the bot is wrong.

Component: Untriaged → Widget: Win32
Product: Firefox → Core

Thanks for the response. The driver for the onboard Intel video adapter is fully up to date.
I am fully aware of the Windows magnifier settings as I have used the magnifier for many years. I spent several hours trying different settings before I filed a bug report.
I am unsure what you mean by "contact the support team of the software you are using". I am using the built-in Windows magnifier and Firefox. Nothing else. The magnifier works perfectly well, as it always has done, with all other software, including Word, Notepad, other browsers such as Chrome and Edge, all Windows OS functions and dialogs, etc. The issue is only with Firefox and only since the recent update.
Unfortunately, it renders Firefox virtually useless to me for filling in forms or even logging in to sites with a user name and password. I am currently typing this effectively blind, although I shall listen to it for typos when I finish.

I can only repeat that Firefox has worked perfectly well with the magnifier for ten years or more. It ceased to do so at the last update (or at least at a very recent update), and I have had no issues with any other software.

To try to reproduce the problem, try the following steps (I am running Windows 10):

  • Start the Windows Magnifier (Win key plus numeric +)
  • In the Magnifier settings, ensure that you are using docked mode and that you have checked all 4 options under "Enable magnifier to follow" (if you want, you can only check "Text cursor". This is the one which no longer works with Firefox.)
  • Now open something like Notepad and type some text. The text insertion point remains centred vertically and horizontally in the magnifier window.
  • Now open Firefox and go to a page with a box of any kind for entering text - the main Google search page will do.
    If you try to enter text, the Magnifier no longer tracks the insertion point at all or (as is the case as I am typing now), the insertion point is centred horizontally, but is at the upper margin of the magnifier window, with only the bottom half of the letters showing.
  • If you try the same thing with Chrome or Edge, the behaviour is correct.

Thank you for your time.

I now have some further information that may help you localize the problem. I discovered an old bug report from 12 years ago detailing a very similar issue (Bug 656089). The issue then appeared to be related to whether or not the menu bar was displayed.

So I tried hiding various toolbars, but none of that helped. However, when I enabled Full Screen mode, the insertion point is correctly centred in the magnifier, including in the text box into which I am now typing.
However, the issue is only resolved in some text boxes. The Search text box on the main Google page, for instance, still fails to track the insertion point entirely. The same is true of the search text box on the main Bing page and of the address box in Firefox itself (when it pops up in Full Screen mode).
Most other form fields on other sites appear to work correctly when Firefox is in full-screen mode, so I have a workaround. But the issue remains with the main search boxes in Bing and Google, for instance, and in the address bar in FF. There is a common feature to all of these boxes in that they have autosuggest lists that appear as you type. This has never been an issue in the past, but appears to be one now.

Could you run mozregression to see when this started happening?

A number of Firefox versions will open in succession to narrow down when this started occurring. Simply answer "good" or "bad" based on whether or not a build reproduces the bug. Once finished, please post the output from the last run. It should give a last good and first bad revision as well as a link to look at the changesets in that range. Thank you!

Flags: needinfo?(wordwizardwizard)

Will try. I shall be out for a few hours, though.

The regression test was frustrating.
Initially, I could find no bad version at all, including the most recent releases.
Then I realized that when Firefox starts in the regression tester, it starts with default settings. For me, this means that the title bar is disabled. With my visual impairment, I always use the title bar to more easily identify where windows are on screen. I have overridden this default setting since it was introduced many years ago.
By enabling the title bar, the problem is reproducible in the regression tester. However, the problem shows up in all versions right back to May 2022. On my live system, however, the problem only arose with a recent update (a few weeks ago at most). Firefox is set to update automatically, so is constantly up to date.
So the situation is now as follows:

  • When the title bar is enabled, the insertion point does not centre correctly vertically in edit boxes. When the title bar is disabled, vertical centring is correct.
  • On my live system, this behaviour only started very recently, but the regression test shows it as early as May 2022 (quite possibly earlier as well, but I didn't go back any further).
  • My title bar has always been enabled. This is not a new setting for me.
  • As far as the failure to track the insertion point at all in some edit boxes is concerned (e.g. in the main search box on the main Google search page), this shows up in all regression tests back to May 2022, although the problem only started on my live system a few weeks ago. As I have said, this issue does not occur in other browsers such as Chrome or Edge.
  • I mentioned that the URL address box was also affected. Here, the behaviour is actually a little different. If you start to enter a URL that is in your history and overtype exactly what comes up in the box, the insertion point is not tracked. As soon as you type a character that is no longer in the history, the cursor is tracked again. This behaviour is acceptable and may not be new. It may be a change to my browsing behaviour with a couple of sites over the past couple of weeks, where I have been regularly returning to certain pages from the same website. The address box is not affected by the vertical centring issue.
    At least I seem to have narrowed down that the vertical centring issue is related to whether the title bar is displayed or not. It just so happens that the vertical offset caused by showing the title bar corresponds exactly to slightly more than one third of the vertical height of my magnifier window, giving the impression that the insertion point is aligned at the top of the window. It is not. If I reduce the height of my magnifier window, the insertion point is not shown in the window at all.

What I do not understand is that the regression tests suggest that both issues (vertical centring and failure to track in some boxes) are long-standing issues, but I have only very recently experienced them, although I use Firefox and the magnifier on a regular basis for many hours a day. I shall have a close look at other display settings, although I do not believe that I have deliberately changed anything over the past months (or, indeed, years).
Thank you again for your time and patience.

Flags: needinfo?(wordwizardwizard)

Thanks for trying to track this down. Although you said that this doesn't affect other browsers, it is possible that there was a different change to your system, such as a Windows update that may have affected Firefox in ways that didn't affect other browsers.

Severity: -- → S3
Priority: -- → P3

Yes. That is entirely possible. It also crossed my mind. The .NET framework has been updated a couple of times in the relevant period, and there have been a couple of regular updates as well. I was considering rolling Windows back, but that gives me the screaming heebie jeebies.

I do have another machine that I haven't switched on for about a month, so there is an outside chance that I could prove (but not necessarily disprove, that theory. I'll have a go later.

I just started up the other machine. Windows was last updated on 23 February 2023 and Firefox 110.0 was running. Both described issues were present. However, they did not disturb me as much.
The reason for this is that, on my main working machine, I have started using a significantly greater magnification (1000 %) as my vision has deteriorated somewhat in the past months. The second machine was set to a lower magnification. This means that, although the insertion point does not centre correctly in the magnifier window, it does not become partially obscured as it partially disappears out of the magnifier window, i.e. it is still perfectly usable at a magnification of 800%. I used to use 600 or 700%. The issue with some combobox type text boxes such as the Google main search is also less annoying, as more of the text I type is visible in the magnifier at a lower magnification.

This explains why I had not realized there was a bug until recently.
But there are still two bugs in Firefox that do not apply in other browsers:

  1. The insertion point does not centre correctly vertically in the Magnifier when the title bar is visible. The insertion point is offset upwards by the corresponding height of the title bar.
  2. The Magnifier does not track the insertion point at all in some combobox-type fields (e.g. the main search boxes on the Google and Bing home pages).

I thank you again for your help and patience. I shall continue to monitor this bug, but I don't think I can contribute any more unless you explicitly ask me for information.

Thank you for isolating this further.

This is most likely a bug in the way the fake caret rectangle is drawn by the accessibility engine (AccessibleWrap::UpdateSystemCaretFor and friends).

Severity: S3 → S2
Component: Widget: Win32 → Disability Access APIs
Keywords: access
Priority: P3 → P2

:wordwizardwizard, can you please go to about:support in the address bar, copy the information under the Accessibility heading and paste it here?

Flags: needinfo?(wordwizardwizard)

Confirmed. The fake caret rect is wrong when the title bar is enabled. For the address bar, it's even worse: it's just all 0s.

Note to self: to get the Windows caret rect location in the NVDA Python console:

wx.CallLater(2000, lambda: speech.speakMessage(repr(oleacc.AccessibleObjectFromEvent(focus.windowHandle, winUser.OBJID_CARET, 0)[0].accLocation())))

Status: UNCONFIRMED → NEW
Ever confirmed: true

Implementation thought: maybe we need to use ClientToScreen with (0, 0) instead of GetWindowRect? I think SetCaretPos takes client coordinates, but we're currently subtracting window rect coordinates from screen coordinates. The window rect includes the non-client area, which includes the title bar.

I don't know what could be causing the issue with the Google and Bing search boxes, though. From what I can tell, the caret is being updated correctly there.

:wordwizardwizard, in addition to my other question, do the Google/Bing search boxes work correctly when the title bar is disabled or do they fail then too?

(I'm totally blind, which is why I didn't just check this myself.)

:jamie
This is the information under Accessibility:
Activated true
Prevent Accessibility 0
Accessible Handler Used true
Accessibility Instantiator UIAUTOMATION|

And in answer to your second question, the behaviour of the Google and Bing search boxes seems to be unrelated. It is present whether the title bar is on or off.

Hope that helps.

Flags: needinfo?(wordwizardwizard)

Thank you. In the Google/Bing search box, does caret tracking work if you press left arrow to move back through characters you've typed previously?

Flags: needinfo?(wordwizardwizard)

Investigation notes: ClientToScreen helps, but it isn't enough by itself. There's also another bug which seems to make GetCaretRect return incorrect coordinates (negative height!) for Accessibles in the parent process (e.g. the address bar, the search box in Settings). As for the Google search box, my guess is that this is because of the aria-owned list box consuming the final character in the text box, which clobbers the insertion point at the end and results in us returning a 0 rect. We may need to fix bug 1280188 to deal with that, though maybe I can come up with an interim fix.

(In reply to James Teh [:Jamie] from comment #18)

Thank you. In the Google/Bing search box, does caret tracking work if you press left arrow to move back through characters you've typed previously?

Well done! Yes, that works correctly, and If I move back with the left arrow and insert text, the caret also tracks correctly. So as a workaround, I can type a character, move back and insert what I want to search for, finally deleting the superfluous character I typed. This also works if I type a punctuation mark, but not if I type a space.

I don't quite know which part of this was breaking, but the coordinates were quite wrong in the parent process.
They were wrong enough that when we adjusted for the character height, we ended up with a negative height in the resulting rect.
This code is now more similar to how we do this in LocalAccessible::BoundsInAppUnits.
This should also take CSS transforms into account in most cases where we weren't previously, though perhaps not for some iframe transforms.

Previously, we were making the screen coordinates relative to the window rect.
However, the window rect includes the non-client area of the window, which includes the title bar.
SetCaretPos expects client coordinates.
In Firefox, this normally doesn't matter because the title bar is disabled, so there is no non-client area.
However, when the title bar is disabled, the coordinates were previously incorrect.

Assignee: nobody → jteh
Status: NEW → ASSIGNED

:wordwizardwizard, would you mind trying out this build of Firefox and letting me know whether the caret is tracked correctly in most text boxes (including the browser address bar) when the title bar is enabled? Note that the Google and Bing search boxes are still broken. I'll need to deal with those separately, as the root cause is very different.
Here's a zip archive which you can extract somewhere without installing, or here's a full installer if you prefer.

:Jamie That looks good to me. I checked against four or five known culprits and the caret tracks correctly.
The address bar also tracks correctly vertically. It tracks correctly horizontally unless you are typing a known address, in which case the field is autofilled and the insertion point is marked by highlighting of the remaining text. This has always been the case, and I have never mentioned it, as it never presents a problem as it is unlikely that I will type more than five or six characters of a known address before selecting from the drop-down. I would not see this as a bug as such. I merely mention it for the sake of completeness.
The Bing and Google search boxes now track correctly vertically, as expected, but not horizontally, as you said.

Flags: needinfo?(wordwizardwizard)
Attachment #9330555 - Attachment description: Bug 1828295 part 1 WIP: Fix HyperTextAccessible::GetCaretRect when the title bar is enabled. → Bug 1828295 part 1: Fix HyperTextAccessible::GetCaretRect when the title bar is enabled.

This required exposing GetCaretRect via XPCOM.

Pushed by jteh@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/4e8ff9c439ef
part 1: Fix HyperTextAccessible::GetCaretRect when the title bar is enabled. r=morgan
https://hg.mozilla.org/integration/autoland/rev/6e6ee5fe337f
part 2: When setting the fake a11y Windows caret, convert the screen coordinates to client coordinates. r=nlapre
https://hg.mozilla.org/integration/autoland/rev/52b9bf7e6344
part 3: Add tests for GetCaretRect. r=morgan

Backed out for causing failures on browser_caret_rect.js

[task 2023-05-31T05:22:16.707Z] 05:22:16     INFO - TEST-PASS | accessible/tests/browser/bounds/browser_caret_rect.js | Doc has same width after title bar change - 
[task 2023-05-31T05:22:16.707Z] 05:22:16     INFO - Buffered messages finished
[task 2023-05-31T05:22:16.714Z] 05:22:16     INFO - TEST-UNEXPECTED-FAIL | accessible/tests/browser/bounds/browser_caret_rect.js | Doc has smaller height after title bar change - 866 < 866 - {"filename":"chrome://mochitests/content/browser/accessible/tests/browser/bounds/browser_caret_rect.js","name":null,"sourceId":615,"lineNumber":145,"columnNumber":14,"sourceLine":"","asyncCause":null,"asyncCaller":{"filename":"chrome://mochitests/content/browser/accessible/tests/browser/shared-head.js","name":"accessibleTask/</<","sourceId":583,"lineNumber":548,"columnNumber":15,"sourceLine":"","asyncCause":null,"asyncCaller":{"filename":"resource://testing-common/BrowserTestUtils.sys.mjs","name":"withNewTab","sourceId":566,"lineNumber":149,"columnNumber":22,"sourceLine":"","asyncCause":null,"asyncCaller":{"filename":"chrome://mochitests/content/browser/accessible/tests/browser/shared-head.js","name":"accessibleTask/<","sourceId":583,"lineNumber":467,"columnNumber":28,"sourceLine":"","asyncCause":null,"asyncCaller":null,"caller":{"filename":"chrome://mochikit/content/browser-test.js","name":"handleTask","sourceId":534,"lineNumber":1133,"columnNumber":26,"sourceLine":"","asyncCause":null,"asyncCaller":null,"caller":{"filename":"chrome://mochikit/content/browser-test.js","name":"_runTaskBasedTest","sourceId":534,"lineNumber":1205,"columnNumber":18,"sourceLine":"","asyncCause":null,"asyncCaller":{"filename":"chrome://mochikit/content/browser-test.js","name":"Tester_execTest","sourceId":534,"lineNumber":1347,"columnNumber":14,"sourceLine":"","asyncCause":null,"asyncCaller":null,"caller":{"filename":"chrome://mochikit/content/browser-test.js","name":"nextTest/<","sourceId":534,"lineNumber":1122,"columnNumber":14,"sourceLine":"","asyncCause":null,"asyncCaller":null,"caller":{"filename":"chrome://mochikit/content/tests/SimpleTest/SimpleTest.js","name":"SimpleTest.waitForFocus/<","sourceId":560,"lineNumber":1056,"columnNumber":13,"sourceLine":"","asyncCause":null,"asyncCaller":null,"caller":null,"formattedStack":"SimpleTest.waitForFocus/<@chrome://mochikit/content/tests/SimpleTest/SimpleTest.js:1056:13\n","nativeSavedFrame":{}},"formattedStack":"nextTest/<@chrome://mochikit/content/browser-test.js:1122:14\nSimpleTest.waitForFocus/<@chrome://mochikit/content/tests/SimpleTest/SimpleTest.js:1056:13\n","nativeSavedFrame":{}},"formattedStack":"async*Tester_execTest@chrome://mochikit/content/browser-test.js:1347:14\nnextTest/<@chrome://mochikit/content/browser-test.js:1122:14\nSimpleTest.waitForFocus/<@chrome://mochikit/content/tests/SimpleTest/SimpleTest.js:1056:13\n","nativeSavedFrame":{}},"caller":null,"formattedStack":"_runTaskBasedTest@chrome://mochikit/content/browser-test.js:1205:18\nasync*Tester_execTest@chrome://mochikit/content/browser-test.js:1347:14\nnextTest/<@chrome://mochikit/content/browser-test.js:1122:14\nSimpleTest.waitForFocus/<@chrome://mochikit/content/tests/SimpleTest/SimpleTest.js:1056:13\n","nativeSavedFrame":{}},"formattedStack":"handleTask@chrome://mochikit/content/browser-test.js:1133:26\n_runTaskBasedTest@chrome://mochikit/content/browser-test.js:1205:18\nasync*Tester_execTest@chrome://mochikit/content/browser-test.js:1347:14\nnextTest/<@chrome://mochikit/content/browser-test.js:1122:14\nSimpleTest.waitForFocus/<@chrome://mochikit/content/tests/SimpleTest/SimpleTest.js:1056:13\n","nativeSavedFrame":{}},"formattedStack":"async*accessibleTask/<@chrome://mochitests/content/browser/accessible/tests/browser/shared-head.js:467:28\nhandleTask@chrome://mochikit/content/browser-test.js:1133:26\n_runTaskBasedTest@chrome://mochikit/content/browser-test.js:1205:18\nasync*Tester_execTest@chrome://mochikit/content/browser-test.js:1347:14\nnextTest/<@chrome://mochikit/content/browser-test.js:1122:14\nSimpleTest.waitForFocus/<@chrome://mochikit/content/tests/SimpleTest/SimpleTest.js:1056:13\n","nativeSavedFrame":{}},"caller":null,"formattedStack":"async*withNewTab@resource://testing-common/BrowserTestUtils.sys.mjs:149:22\nasync*accessibleTask/<@chrome://mochitests/content/browser/accessible/tests/browser/shared-head.js:467:28\nhandleTask@chrome://mochikit/content/browser-test.js:1133:26\n_runTaskBasedTest@chrome://mochikit/content/browser-test.js:1205:18\nasync*Tester_execTest@chrome://mochikit/content/browser-test.js:1347:14\nnextTest/<@chrome://mochikit/content/browser-test.js:1122:14\nSimpleTest.waitForFocus/<@chrome://mochikit/content/tests/SimpleTest/SimpleTest.js:1056:13\n","nativeSavedFrame":{}},"caller":null,"formattedStack":"async*accessibleTask/</<@chrome://mochitests/content/browser/accessible/tests/browser/shared-head.js:548:15\nasync*withNewTab@resource://testing-common/BrowserTestUtils.sys.mjs:149:22\nasync*accessibleTask/<@chrome://mochitests/content/browser/accessible/tests/browser/shared-head.js:467:28\nhandleTask@chrome://mochikit/content/browser-test.js:1133:26\n_runTaskBasedTest@chrome://mochikit/content/browser-test.js:1205:18\nasync*Tester_execTest@chrome://mochikit/content/browser-test.js:1347:14\nnextTest/<@chrome://mochikit/content/browser-test.js:1122:14\nSimpleTest.waitForFocus/<@chrome://mochikit/content/tests/SimpleTest/SimpleTest.js:1056:13\n","nativeSavedFrame":{}},"caller":null,"formattedStack":"@chrome://mochitests/content/browser/accessible/tests/browser/bounds/browser_caret_rect.js:145:14\nasync*accessibleTask/</<@chrome://mochitests/content/browser/accessible/tests/browser/shared-head.js:548:15\nasync*withNewTab@resource://testing-common/BrowserTestUtils.sys.mjs:149:22\nasync*accessibleTask/<@chrome://mochitests/content/browser/accessible/tests/browser/shared-head.js:467:28\nhandleTask@chrome://mochikit/content/browser-test.js:1133:26\n_runTaskBasedTest@chrome://mochikit/content/browser-test.js:1205:18\nasync*Tester_execTest@chrome://mochikit/content/browser-test.js:1347:14\nnextTest/<@chrome://mochikit/content/browser-test.js:1122:14\nSimpleTest.waitForFocus/<@chrome://mochikit/content/tests/SimpleTest/SimpleTest.js:1056:13\n","nativeSavedFrame":{}}
[task 2023-05-31T05:22:16.714Z] 05:22:16     INFO - Stack trace:
[task 2023-05-31T05:22:16.714Z] 05:22:16     INFO - chrome://mochitests/content/browser/accessible/tests/browser/bounds/browser_caret_rect.js:null:145
[task 2023-05-31T05:22:16.714Z] 05:22:16     INFO - chrome://mochitests/content/browser/accessible/tests/browser/shared-head.js:accessibleTask/</<:548
[task 2023-05-31T05:22:16.714Z] 05:22:16     INFO - resource://testing-common/BrowserTestUtils.sys.mjs:withNewTab:149
[task 2023-05-31T05:22:16.714Z] 05:22:16     INFO - chrome://mochitests/content/browser/accessible/tests/browser/shared-head.js:accessibleTask/<:467
[task 2023-05-31T05:22:16.714Z] 05:22:16     INFO - chrome://mochikit/content/browser-test.js:handleTask:1133
[task 2023-05-31T05:22:16.714Z] 05:22:16     INFO - chrome://mochikit/content/browser-test.js:_runTaskBasedTest:1205
[task 2023-05-31T05:22:16.714Z] 05:22:16     INFO - chrome://mochikit/content/browser-test.js:Tester_execTest:1347
[task 2023-05-31T05:22:16.714Z] 05:22:16     INFO - chrome://mochikit/content/browser-test.js:nextTest/<:1122
[task 2023-05-31T05:22:16.714Z] 05:22:16     INFO - chrome://mochikit/content/tests/SimpleTest/SimpleTest.js:SimpleTest.waitForFocus/<:1056
[task 2023-05-31T05:22:16.715Z] 05:22:16     INFO - Focusing input
[task 2023-05-31T05:22:16.715Z] 05:22:16     INFO - TEST-PASS | accessible/tests/browser/bounds/browser_caret_rect.js | Recieved text caret moved event - 
Flags: needinfo?(jteh)
Flags: needinfo?(jteh)
Pushed by jteh@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/06187ba37ad9
part 1: Fix HyperTextAccessible::GetCaretRect when the title bar is enabled. r=morgan
https://hg.mozilla.org/integration/autoland/rev/34823d243452
part 2: When setting the fake a11y Windows caret, convert the screen coordinates to client coordinates. r=nlapre
https://hg.mozilla.org/integration/autoland/rev/4ae88349315c
part 3: Add tests for GetCaretRect. r=morgan
Status: ASSIGNED → RESOLVED
Closed: 11 months ago
Resolution: --- → FIXED
Target Milestone: --- → 115 Branch

Release Note Request (optional, but appreciated)
[Why is this notable]: We've fixed a bug impacting Windows Magnifier users.
[Affects Firefox for Android]: No.
[Suggested wording]: Windows Magnifier now follows the text cursor correctly when the Firefox title bar is visible.
[Links (documentation, blog post, etc)]: None.

relnote-firefox: --- → ?

Thanks to everyone involved in resolving this issue. I look forward to the release.

Thanks, added to the Fx115 Nightly release notes. Keeping the relnote? flag open to keep it on the radar for inclusion in our final release notes.

You need to log in before you can comment on or make changes to this bug.

Attachment

General

Created:
Updated:
Size: