1587921 - Context menu covers blue carets when the input field is near the top of the screen

Status: RESOLVED WORKSFORME

(GeckoView :: IME, defect, P3)

Description

[Ting-Yu Lin [:TYLin] (PST, UTC-8)]

Attachment: Screenshot_20191010-123217.png

Filed this per discussion in bug 1581007 comment 8.

STR

1. Open Wikipedia: https://en.m.wikipedia.org.
2. Enter some words, such as "Mozilla Firefox" in the "Search Wikipedia" text field.
3. Try to select an individual word by long pressing it.

Expected Result

The context menu doesn't cover the blue carets.

Actual Result

The context menu covers the blue carets. Users are unable to move blue carets.

Comment 1

[Daniel Holbert [:dholbert]]

TYLin, what's your hardware & Android version?

I'm using a Pixel 3 with Android 10 (Q), and I can't reproduce this bug here (my just-fine screenshot: attachment 9100316 [details]). I'm guessing our differing behavior might be due to a pixel-density difference, or an Android-API difference.

Comment 2

[Ting-Yu Lin [:TYLin] (PST, UTC-8)]

I'm using a Pixel 2 (not XL) with Android 10, and I can still reproduce the issue on Firefox Preview Nightly version 191011 13:15 (Build #12841339).

Comment 3

[Ting-Yu Lin [:TYLin] (PST, UTC-8)]

Attachment: Screenshot_20191011-110957.png

This is another screenshot on my device that the context menu is too far away from the blue carets.

Comment 4

[Chris Peterson [:cpeterson]]

IIUC, dholbert's bug 1581007 comment 15 says Fenix issue https://github.com/mozilla-mobile/fenix/issues/2038 can be worked around if this caret bug is fixed.

Comment 5

[Chris Peterson [:cpeterson]]

The app is responsible for drawing the menu. This bug might caused by Gecko giving GV wrong page coordinates, GV incorrectly mapping page coordinates to screen coordinates, and/or Fenix incorrectly positioning the menu's screen coordinates.

GV engineers are unable to reproduce in Fenix Release or Nightly.

P3

Comment 6

[Ting-Yu Lin [:TYLin] (PST, UTC-8)]

I can take a look later since I can reproduce this bug.

Comment 7

[Ting-Yu Lin [:TYLin] (PST, UTC-8)]

If I set layout.css.devPixelsPerPx=1, the context menu position looks correct to me. However its position shifts as soon as I pinch-zoom in, so I guess there's something wrong when GV mapping page coordinates (or CSS coordinates) to screen coordinates. On Fennec, I don't see this issue.

My normal window.devicePixelRatio is 2.61 (by looking at the MDN cod example), but no matter what layout.css.devPixelsPerPx I set, I always see the same resolution and displaySize in geckoview_example log like

GeckoViewActivity: New Session state: {"scrolldata":{"zoom":{"resolution":1,"displaySize":{"height":1584,"width":1080}}}

But if I pinch-zoom in, the value of resolution does become larger than 1.

Botond, any idea offhand where else I can look? (Not sure why this bug happens only on my Pixel 2 ...)

Comment 8

[Botond Ballo [:botond]]

I also cannot reproduce the issue.

(In reply to Ting-Yu Lin [:TYLin] (UTC-7) from comment #7)

My normal window.devicePixelRatio is 2.61 (by looking at the MDN cod example), but no matter what layout.css.devPixelsPerPx I set, I always see the same resolution and displaySize in geckoview_example log like

GeckoViewActivity: New Session state: {"scrolldata":{"zoom":{"resolution":1,"displaySize":{"height":1584,"width":1080}}}

But if I pinch-zoom in, the value of resolution does become larger than 1.

I think this part is expected. The resolution includes the pinch zoom scale but not the device scale, and the display size is likely in screen coordinates.

Botond, any idea offhand where else I can look? (Not sure why this bug happens only on my Pixel 2 ...)

I did some investigation of the positioning of this menu in bug 1484597 comment 2, although that was for Fennec.

Assuming GeckoView still uses that same code, I would suggest logging the bounding client rect in the places linked to from that comment, and seeing if the values are reasonable.

If GeckoView does not use that code, we would need to find the corresponding code that GeckoView uses to position the menu.

Comment 9

[Ting-Yu Lin [:TYLin] (PST, UTC-8)]

GeckoView uses onGetContentRect in Callback2Wrapper to provide the coordinate for the ActionMode, i.e. the "copy/select all/..." menu.

It calls onGetContentRect in BasicSelectionActionDelegate, which further calls GeckoSession's getClientToScreenMatrix. What's interesting is that the client coordinate from Accessiblecaret is in CSS pixels, but the Screen coordinate should be something similar to device pixels. I cannot find anything in GeckoSession.java that translates between these two coordinates.

Comment 10

[Ting-Yu Lin [:TYLin] (PST, UTC-8)]

Not sure what had been changed, but I cannot reproduce this on fenix nightly with my Pixel 2.

Nightly 200114 18:01 (Build #20141812)
28.0.0, f4ddf4e26
GV: 74.0a1-20200113093823

Comment 11

[Botond Ballo [:botond]]

(In reply to Ting-Yu Lin [:TYLin] (UTC-8) from comment #10)

Not sure what had been changed

It might have been fixed by bug 1600652.

Comment 12

[Chris Peterson [:cpeterson]]

Moving some input bugs to the new GeckoView::IME component.