"Scroll to hide toolbar" causes errant scroll and zoom on input focus
Categories
(Core :: Panning and Zooming, defect, P3)
Tracking
()
| Tracking | Status | |
|---|---|---|
| firefox120 | --- | fixed |
People
(Reporter: matt.tingen, Assigned: ajakobi)
References
Details
Attachments
(6 files, 2 obsolete files)
Steps to reproduce:
- Ensure "Scroll to hide toolbar" is enabled
- Adjust the
margin-topin the attached document such that the input would partially intersect the onscreen keyboard - Tap the input to focus it
(Tested on Pixel 4a on 113.0b2 Build #2015944243)
Actual results:
- The page zooms slightly
- The input ends up underneath the keyboard such that it is not visible
Expected results:
- The page should not zoom
- The page should scroll such that the input is above the onscreen keyboard
| Reporter | ||
Comment 1•3 years ago
|
||
Possibly related: bug 1746126
Comment 2•3 years ago
|
||
Confirmed. The fact that the page zooms suggests that the mechanism added in bug 1746126 to prevent zoom-to-focused-input on elements inside a touch-action: none region is not working for some reason in this case.
| Assignee | ||
Updated•2 years ago
|
| Assignee | ||
Comment 3•2 years ago
•
|
||
I investigated different values inside of nsLayoutUtils::GetContentViewerSize.
The use cases Overlap, High and Low refer to the input field placement.
Overlap places the input field such that the dynamic toolbar will overlap with the input field when activated.
High will keep the input field visible and above the dynamic toolbar
Low the input field is placed so low such that it is placed under the dynamic toolbar and hidden by the keyboard.
I have used these pages to reproduce on a Samsung Galaxy A54 5G phone. You might have to adjust the input field placement according to your device.
The most interesting finding being the DynamicToolbarHeight being 0 in the overlap case, after tapping into the input field, minimizing the keyboard, then scrolling down a bit to minimize the dynamic toolbar and then tapping to bring the toolbar back up.
Overlap
aPresContext->HasDynamicToolbar() always returns true, because GetDynamicToolbarMaxHeight() always returns 158.
When the toolbar is shown, with or without keyboard, GetDynamicToolbarHeight() reports 158 as well.
After clicking into the input field, the keyboard and the toolbar are shown. When scrolling down a little, the toolbar
disappears and GetDynamicToolbarHeight() reports 0.
Then minimizing the keyboard again, the toolbar is shown and GetDynamicToolbarHeight() reports 158 again.
Before - keyboard collapsed, toolbar shown
RootContentDocumentCrossProcess = true
HasDynamicToolbar = true
DynamicToolbarHeight = 158
DynamicToolbarHeight MAX = 158
bounds height total = 2218
bounds height w/o toolbar = 2060
aSubtractDynamicToolbar = Yes
After - keyboard shown, toolbar shown
RootContentDocumentCrossProcess = true
HasDynamicToolbar = true
DynamicToolbarHeight = 158
DynamicToolbarHeight MAX = 158
aSubtractDynamicToolbar = No
bounds height total = 1252
bounds height w/o toolbar = 1094
After - keyboard shown, toolbar collapsed (after scroll)
RootContentDocumentCrossProcess = true
HasDynamicToolbar = true
DynamicToolbarHeight = 0
DynamicToolbarHeight MAX = 158
bounds height total = 1252
bounds height w/o toolbar = 1094
aSubtractDynamicToolbar = No
After - keyboard collapsed, toolbar scrolled away, then shown again through tap
RootContentDocumentCrossProcess = true
HasDynamicToolbar = true
DynamicToolbarHeight = 0
DynamicToolbarHeight MAX = 158
bounds height total = 2218
bounds height w/o toolbar = 2060
aSubtractDynamicToolbar = No
High
aPresContext->HasDynamicToolbar() always returns true, because GetDynamicToolbarMaxHeight() always returns 158.
The toolbar is always shown, therefore GetDynamicToolbarHeight() reports 158 as well.
Scrolling is not possible.
Before - keyboard collapsed, toolbar shown
RootContentDocumentCrossProcess = true
HasDynamicToolbar = true
DynamicToolbarHeight = 158
DynamicToolbarHeight MAX = 158
bounds height total = 2218
bounds height w/o toolbar = 2060
aSubtractDynamicToolbar = Yes
After - keyboard shown, toolbar shown
RootContentDocumentCrossProcess = true
HasDynamicToolbar = true
DynamicToolbarHeight = 158
DynamicToolbarHeight MAX = 158
bounds height total = 1252
bounds height w/o toolbar = 1094
aSubtractDynamicToolbar = Yes
Low
Mixture of "Overlap" and "High" case: GetDynamicToolbarMaxHeight() always returns 158 because the toolbar is shown.
Unless you scroll down a bit after the keyboard appears, because the dynamic toolbar disappears and GetDynamicToolbarHeight() reports 0.
Before - keyboard collapsed, toolbar shown
RootContentDocumentCrossProcess = true
HasDynamicToolbar = true
DynamicToolbarHeight = 158
DynamicToolbarHeight MAX = 158
bounds height total = 2218
bounds height w/o toolbar = 2060
aSubtractDynamicToolbar = Yes
After - keyboard shown, toolbar shown
RootContentDocumentCrossProcess = true
HasDynamicToolbar = true
DynamicToolbarHeight = 158
DynamicToolbarHeight MAX = 158
bounds height total = 1252
bounds height w/o toolbar = 1094
aSubtractDynamicToolbar = No
After - keyboard shown, toolbar collapsed (after scroll)
RootContentDocumentCrossProcess = true
HasDynamicToolbar = true
DynamicToolbarHeight = 0
DynamicToolbarHeight MAX = 158
bounds height total = 1252
bounds height w/o toolbar = 1094
aSubtractDynamicToolbar = No
ICBSize.height
Logged from nsHTMLScrollFrame::UpdateMinimumScaleSize.
Value were the same across all three use cases (Overlap, High, Low).
No keyboard: 43260
Keyboard: 22974
Comment 4•2 years ago
|
||
Thanks for the update!
Here are some conclusions we can draw from this info:
1. The ICB size indeed consistently excludes the dynamic toolbar height.
The logged values are in app units, we can convert them to screen pixels by diving by 60 (number of app units per CSS pixel) and multiplying by the zoom (number of Screen pixels per CSS pixel).
The zoom consists of the device scale and the resolution; we're interested in how the ICB size compares to the content viewer size when the page loads, and the page loads with initial-scale=1 (i.e. a resolution of 1), so the zoom is just the device scale, which on your device we observed to be ~2.857.
So we have:
- no keyboard: 43260 / 60 * 2.857 = 2060
- keyboard: 22974 / 60 * 2.857 = 1094
2. My theory about the page being in a "static toolbar" state (HasDynamicToolbar() == false) when the toolbar is always shown ("high" testcase) was incorrect
Question: on the "low" testcase, can the toolbar be hidden before tapping the input field? If not, i.e. if that's also a "toolbar is always shown" scenario, then we are consistently seeing aSubtractDynamicToolbar = Yes in "toolbar is always shown" scenarios, which would provide an alternative explanation for some of the metrics we were seeing (e.g. composition bounds).
Next steps
I think a good next step is to also log in the various scenarios the composition bounds height and scrollable rect height.
Previously we were logging these in ZoomToRect(), but that's only applicable to the "keyboard shown" case. A good place to check them on page load is this existing log statement in NotifyLayersUpdated(); it's a bit noisy, but we can just check the most recently logged value after loading the page.
Comment 5•2 years ago
•
|
||
I'm going to make some predictions:
| Keyboard hidden | Keyboard shown | |
|---|---|---|
| Overlap | cb: 2060, sr: 721 | cb: 1252, 382 < sr < 438 |
| High | cb: 2060, sr: 721 | cb: 1094, sr: 382 |
| Low | cb: 2060 (?), sr: 721 (?) | cb: 1252, sr >= 438 |
- "cb" is the composition bounds height, in Screen pixels (
Metrics().GetCompositionBounds().height) - "sr" is the scrollable rect height, in CSS pixels (
Metrics().GetScrollableRect().height) - The sr values of 721, 438, and 382 are just 2060, 1252, and 1094 divided by the device scale of 2.857
- The most interesting part of the hypothesis is that in the overlap case, and only in the overlap case, we have
382 < sr < 438when the keyboard is shown
| Assignee | ||
Comment 6•2 years ago
•
|
||
Question: on the "low" testcase, can the toolbar be hidden before tapping the input field? If not, i.e. if that's also a "toolbar is always shown" scenario, then we are consistently seeing
aSubtractDynamicToolbar = Yesin "toolbar is always shown" scenarios, which would provide an alternative explanation for some of the metrics we were seeing (e.g. composition bounds).
The toolbar cannot be hidden. If I'm trying to scroll down after initially loading the "Low" test page, I can see aSubtractDynamicToolbar = No as well.
Next steps
I think a good next step is to also log in the various scenarios the composition bounds height and scrollable rect height.
Your predictions were spot on, here are the logged values:
| Keyboard hidden | Keyboard shown | |
|---|---|---|
| Overlap | cb: 2060, sr: 721 | cb: 1252, 382 < sr = 397.95 < 438 |
| High | cb: 2060, sr: 721 | cb: 1094, sr: 382(.89) |
| Low | cb: 2060, sr: 721 | cb: 1252, sr = 452.89 >= 438 |
Comment 7•2 years ago
|
||
(In reply to Alex Jakobi from comment #6)
Your predictions were spot on, here are the logged values:
Keyboard hidden Keyboard shown Overlap cb: 2060, sr: 721 cb: 1252, 382 < sr = 397.95 < 438 High cb: 2060, sr: 721 cb: 1094, sr: 382(.89) Low cb: 2060 (?), sr: 721 (?) cb: 1252, sr = 452.89 >= 438
Thanks. Based on this, here's what I believe is going on:
- The initial height of the composition bounds is equal to the content viewer height excluding the dynamic toolbar (2060 with the keyboard hidden, 1094 with the keyboard shown).
- If the scrollable rect height, after conversion to Screen pixels, is no larger than this, the composition bounds remains at this height.
- This is the case with the keyboard hidden on all pages, and with the keyboard shown on the "High" page.
- In this case, the ratio
compositionBounds.Height() / cssPageRect.Height()here, is equal to the device scale (2.857), and thus the minimum zoom ends up being 2.857 as well. (The width ratio is 2.857 in all scenarios on this page.)
- If the scrollable rect height is larger than this, the composition bounds height gets expanded to the content viewer height including the dynamic toolbar (1252 with the keyboard shown).
- I believe the check for this is here; if the branch is taken, we pass
SubtractDynamicToolbar::Noto the call toGetContentViewerSize()whose result we'll use for the composition bounds size. - If the scrollable rect height is greater than or equal to this expanded height (i.e. >= 438 CSS pixels), then all is well -- the ratio
compositionBounds.Height() / cssPageRect.Height()will be <= 2.857, and so the minimum zoom will remain 2.857. - But if the scrollable rect height is less than this expanded height, i.e. the "382 < sr < 438" case, now the ratio
compositionBounds.Height() / cssPageRect.Height()will be > 2.857 (in this case, we have 1252 / 397.95 = 3.14), and so we compute a minimum zoom (3.14) which is higher than the actual current zoom of 2.857.
- I believe the check for this is here; if the branch is taken, we pass
One way to characterize the problem is:
- We expect the scrollable rect always fills the composition bounds
- But the composition bounds can be expanded to include the toolbar height, without the scrollable rect height being expanded by that same amount
Which leads me to consider the following potential solution: if we expand the composition bounds to include the toolbar height, maybe we should also expand the scrollable rect to be at least this tall as well?
Hiro, what do you think about this solution approach -- does it sound feasible? Do you foresee any roadblocks to it?
Comment 8•2 years ago
|
||
(In reply to Botond Ballo [:botond] from comment #7)
One way to characterize the problem is:
- We expect the scrollable rect always fills the composition bounds
- But the composition bounds can be expanded to include the toolbar height, without the scrollable rect height being expanded by that same amount
Which leads me to consider the following potential solution: if we expand the composition bounds to include the toolbar height, maybe we should also expand the scrollable rect to be at least this tall as well?
Hiro, what do you think about this solution approach -- does it sound feasible? Do you foresee any roadblocks to it?
Though I don't yet understand fully what the underlying problem causing this bug is, we've already done such kind of expanding the scrollable rect by this GetExpandedScrollableRect?
Comment 9•2 years ago
•
|
||
(In reply to Hiroyuki Ikezoe (:hiro) from comment #8)
Though I don't yet understand fully what the underlying problem causing this bug is, we've already done such kind of expanding the scrollable rect by this GetExpandedScrollableRect?
Good point!
I initially envisioned expanding the scrollable rect in a layout sense (such that e.g. documentElement.scrollHeight will report the expanded height), but that's probably unnecessarily complicated; it's probably sufficient to just expand the scrollable rect used by APZ, and like you say we already have GetExpandedScrollableRect which returns a rect that covers the composition bounds.
The place where we compute the minimum zoom for zoom-to-focused-input does not use GetExpandedScrollableRect -- maybe fixing this is as simple as using it there in place of GetScrollableRect?
Comment 10•2 years ago
|
||
Oh it's in ZoomToRect's restriction? Then using the expanded scrollable rect should just work. I think.
| Assignee | ||
Updated•2 years ago
|
| Assignee | ||
Comment 11•2 years ago
|
||
| Assignee | ||
Comment 12•2 years ago
•
|
||
Thanks for all your help, folks!
Oh it's in ZoomToRect's restriction? Then using the expanded scrollable rect should just work. I think.
It works, no unwanted zooming in the Overlap case. However, two other problems emerged.
Overlap
See attached screenshot.
Reproduction
Steps
- Open Overlap test page
- Tap into the input field
- Scroll down in order to hide the dynamic toolbar
Observed result
The bottom of the input field that was previously hidden by the dynamic toolbar is not shown.
Expected result
The bottom of the input field should be visible.
Low
Reproduction
- Open Low test page
- Tap into input field
- Scroll down in order to hide the dynamic toolbar
- Close the keyboard
Observed result
The input field jumps down
Expected result
The input field should remain at its position.
| Assignee | ||
Comment 13•2 years ago
|
||
| Assignee | ||
Comment 14•2 years ago
|
||
Given we seem to be back into ZoomToRect, should we reconsider using the initial fix Dan and I found, which was to put the SetZoom call inside an if statement to prevent it being called when aFlags contains PAN_INTO_VIEW_ONLY, or should we keep working on resolving the issues described above?
Comment 15•2 years ago
|
||
(In reply to Alex Jakobi from comment #12)
It works, no unwanted zooming in the Overlap case. However, two other problems emerged.
Does the patch that you tried change all usages of cssPageRect in ZoomToRect() to be the expanded scrollable rect, or just the one in the localMinZoom computation?
If it's the former, it's worth checking whether limiting the change to just the localMinZoom computation avoids these issues.
Comment 16•2 years ago
•
|
||
(In reply to Alex Jakobi from comment #14)
should we reconsider using the initial fix Dan and I found, which was to put the SetZoom call inside an if statement to prevent it being called when
aFlagscontainsPAN_INTO_VIEW_ONLY, or should we keep working on resolving the issues described above?
If the suggestion in comment 15 doesn't avoid the issues you found, but this alternative fix does, then I think it makes sense to go with this alternative fix for now, as it seems pretty harmless.
However, we should file a follow-up bug to continue exploring the original fix, because I think that computing a minimum zoom which is higher than the current zoom can lead to other problems which fall outside of the PAN_INTO_VIEW_ONLY case. A couple of specific scenarios that come to mind are:
- A page that doesn't use
touch-action: none, and where the input element is the full width of the viewport (or close to it). In such cases, we don't want to zoom in (since the input element would overflow the viewport horizontally), but the incorrect min-zoom will cause us to anyways. - Double-tap zoom, which also uses ZoomToRect. The first double-tap zooms in, a second double-tap should zoom back out to the original zoom level. An incorrect min-zoom will prevent us from zooming all the way out.
Comment 17•2 years ago
|
||
Yeah, the alternative will introduce other problems, for example if the body height is calc(50vh - 20px).
The problem in comment 12 is what I had in my mind when I saw this bug. The root problem is that our scroll-into-view machinery doesn't care the dynamic toolbar (since the area covered by the dynamic toolbar is not considered as scrollable).
So, I believe here need two different changes. One is the approach what Botond suggested. The other one is tweak scroll-into-view machinery.
If we do a follow-up issue, the other part should be a follow-up. I don't think skipping SetZoom in the case of PAN_INTO_VIEW_ONLY is the right thing to do.
Comment 18•2 years ago
|
||
I guess just doing the other part fixes this bug entirely?
Comment 19•2 years ago
|
||
(In reply to Hiroyuki Ikezoe (:hiro) from comment #17)
the alternative will introduce other problems, for example if the body height is
calc(50vh - 20px).
[...]
I don't think skippingSetZoomin the case ofPAN_INTO_VIEW_ONLYis the right thing to do.
Could you elaborate on what would go wrong? It's not clear to me.
Comment 20•2 years ago
|
||
(In reply to Botond Ballo [:botond] from comment #19)
(In reply to Hiroyuki Ikezoe (:hiro) from comment #17)
the alternative will introduce other problems, for example if the body height is
calc(50vh - 20px).
[...]
I don't think skippingSetZoomin the case ofPAN_INTO_VIEW_ONLYis the right thing to do.Could you elaborate on what would go wrong? It's not clear to me.
Without zooming we can NOT have enough room to scroll at the bottom of the scroll container unfortunately.
For example, given that the toolbar height is 50px, and if an input element is positioned 25px from the bottom edge of the page, if we zoom the contents 2x, we can have 50px (in the screen units) room at the bottom of the element, thus the element can be positioned above the toolbar.
Comment 21•2 years ago
|
||
After looking at the test case in comment 0, now I believe the case can not be visible unless we forcibly hide the toolbar. That's the only one option we can do.
Comment 22•2 years ago
|
||
I am attaching a modified test case which doesn't need to change anything.
Comment 23•2 years ago
|
||
(In reply to Hiroyuki Ikezoe (:hiro) from comment #20)
Without zooming we can NOT have enough room to scroll at the bottom of the scroll container unfortunately.
For example, given that the toolbar height is 50px, and if an input element is positioned 25px from the bottom edge of the page, if we zoom the contents 2x, we can have 50px (in the screen units) room at the bottom of the element, thus the element can be positioned above the toolbar.
I don't think we want to do zooming for the purpose of increasing the amount of room we have to scroll, for a couple of reasons:
- It doesn't scale (e.g. if the element were positioned 5px from the bottom of the page, we'd have to zoom by 10x to create 50px of room to scroll; if the element is right at the bottom, the strategy doesn't work at all)
- It's not consistent with the intent behind
PAN_INTO_VIEW_ONLY, which is to avoid zooming (hence the "only")
I think an explicit check to avoid zooming in PAN_INTO_VIEW_ONLY mode is fine for now as a way to address the zooming-related part of this bug.
For the scroll-into-view part of this bug, I suggest spinning it out into a separate bug. A couple of options for how we could fix that part are:
- Hide the toolbar if necessary to reveal the input element (as suggested in comment 21)
- Allow scrolling the content out of bounds to show the input element above the toolbar. (We may need to do something similar for overlays-content in the future; perhaps the same mechanism can be re-used for both.)
Comment 24•2 years ago
|
||
(In reply to Botond Ballo [:botond] from comment #23)
For the scroll-into-view part of this bug, I suggest spinning it out into a separate bug. A couple of options for how we could fix that part are:
- Hide the toolbar if necessary to reveal the input element (as suggested in comment 21)
- Allow scrolling the content out of bounds to show the input element above the toolbar. (We may need to do something similar for overlays-content in the future; perhaps the same mechanism can be re-used for both.)
I don't think any feature of the interactive-widget can be reused for the out of bounds scrolling. It's totally a new concept. And honestly it's quite uncertain how much needs to be involved.
Comment 25•2 years ago
|
||
Note that hiding the toolbar should not be hard. There are three approaches;
- Hide the toolbar from GeckoView
1-1. ZoomToFocusedInput is triggered in GeckoView side thus we can easily invoke GeckoView APIs there to hide the toolbar
1-2. We need to add an API to tell whether the root content height is greater thanheight: 100svh. - Hide the toolbar from the content process
2-1.nsDOMWindowUtils::ZoomToFocusedInputis invoked in each content process (e.g. it's invoked in an OOP iframe process), thus it's a bit hard to tell the root content heigtht is greater thanheight: 100svh, we need to add an IPC call in PBrowserBridge.ipdl to propagate the information
2-2. We also need to add an API to hide the toolbar - Hide the toolbar from APZ
3-1. In APZ we have enough information to tell whether the root content height is greater thanheight: 100svh(I may be wrong), so we can just add an IPC call PAPZ.ipdl to hide the toolbar
Apparently 3) is the easiest approach I think.
| Assignee | ||
Comment 26•2 years ago
|
||
Unfixed state, showing Overlap, High and Low test pages.
| Assignee | ||
Comment 27•2 years ago
|
||
GetExpandedScrollableRect - Showing Overlap, High and Low case using the following fix:
const CSSRect cssExpandedPageRect = Metrics().GetExpandedScrollableRect();
CSSToParentLayerScale localMinZoom(
std::max(compositionBounds.Width() / cssExpandedPageRect.Width(),
compositionBounds.Height() / cssExpandedPageRect.Height()));
| Assignee | ||
Comment 28•2 years ago
|
||
Exlcuding PAN_INTO_VIEW_ONLY - Showing Overlap, High and Low case. Code:
if (!(aFlags & PAN_INTO_VIEW_ONLY)) {
endZoomToMetrics.SetZoom(CSSToParentLayerScale(targetZoom));
}
| Assignee | ||
Comment 29•2 years ago
|
||
I have added screen recordings for the states Unfixed, GetExpandedScrollableRect and PAN_INTO_VIEW_ONLY.
The cut-off bottom of the input field is shown for both fixes. From my understanding it seems a bit cleaner to go with the GetExpandedScrollabeRect approach, because with this we don't calculate a too large localMinZoom.
So I would suggest to proceed like this:
- Implement the fix using
GetExpandedScrollableRect - Write a mochitest
- File follow-ups
What do you think?
Comment 30•2 years ago
•
|
||
(In reply to Alex Jakobi from comment #29)
I have added screen recordings for the states Unfixed,
GetExpandedScrollableRectandPAN_INTO_VIEW_ONLY.
The cut-off bottom of the input field is shown for both fixes. From my understanding it seems a bit cleaner to go with theGetExpandedScrollabeRectapproach, because with this we don't calculate a too largelocalMinZoom.So I would suggest to proceed like this:
- Implement the fix using
GetExpandedScrollableRect
I believe Botond and I have agreed on this. :)
- Write a mochitest
- File follow-ups
What Botond and I have discussed is how we address the remaining issue. Though I don't have strong opinion whether it should be done in follow-ups or not, given that we haven't yet concluded the approach as of now, doing 1 and 2 in this bug is quite reasonable for me.
Comment 31•2 years ago
|
||
(In reply to Alex Jakobi from comment #29)
So I would suggest to proceed like this:
- Implement the fix using
GetExpandedScrollableRect- Write a mochitest
- File follow-ups
What do you think?
Yep, sounds reasonable to me.
I would also encourage you to post a draft of your fix to Phabricator, and push it to Try, in parallel with working on the mochitest.
| Assignee | ||
Comment 32•2 years ago
|
||
Comment 33•2 years ago
|
||
(In reply to Botond Ballo [:botond] from comment #23)
For the scroll-into-view part of this bug, I suggest spinning it out into a separate bug. A couple of options for how we could fix that part are:
- Hide the toolbar if necessary to reveal the input element (as suggested in comment 21)
- Allow scrolling the content out of bounds to show the input element above the toolbar. (We may need to do something similar for overlays-content in the future; perhaps the same mechanism can be re-used for both.)
I checked the behaviour in Chrome, and Chrome just seems to use a static toolbar in this case (i.e. the toolbar cannot hide, and there is no website content under the toolbar in the first place). That would be another option.
Comment 34•2 years ago
|
||
Comment 35•2 years ago
|
||
| bugherder | ||
| Assignee | ||
Comment 36•2 years ago
|
||
The implemented fix prevents the unwanted zooming. The desired scrolling into view of the input field is now tracked in a separate Bug 1855990.
The zooming was caused by a faulty calculation of the minimal zoom, which didn't account for the dynamic toolbar.
Updated•2 years ago
|
Description
•