1240917 - Consider action bar behavior with native text selection

Status: RESOLVED FIXED

(Firefox for Android Graveyard :: Text Selection, defect)

Description

[James Willcox (:snorp) ([e-mail]) (he/him)]

Right now on Nightly, we show the action bar (copy/paste actions, etc) when you focus a text input field. After 3 seconds, the action bar is hidden. This behavior was mostly carried over from the B2G implementation, so we need to decide if this is right for us or not.

Comment 1

[James Willcox (:snorp) ([e-mail]) (he/him)]

Anthony, please advise.

Comment 2

[Anthony Lam (:antlam) Find me, @antlam on Slack!]

(In reply to James Willcox (:snorp) ([e-mail]) from comment #0)
> Right now on Nightly, we show the action bar (copy/paste actions, etc) when
> you focus a text input field. After 3 seconds, the action bar is hidden.
> This behavior was mostly carried over from the B2G implementation, so we
> need to decide if this is right for us or not.

No. :)

Thanks for filing this. I actually thought this was broken. So, I've been meaning to get around to this. 

We should be consistent with the platform unless there's a really strong reason otherwise.

Comment 3

[Mark Capella [:capella]]

drive by, and forgive me if this is way obvious / SDLC 101 type stuff...

From my perspective, this is an analysis-only / discussion bug so far, which is fine for planning.

I mention this, as I'm happy to help, but if I'm asked to dev code / provide a patch, I need clearly defined / discrete / actionable bugs with set deliverables.

For any requirements driven out of analysis, please file bugs / code change requests with descriptions like:
"Change mobile caret timeout from default 3 seconds to /never/".

Please indicate a desired completion date for proper scoping / planning. You'll notice I resist strongly attempts at open-ended or circular dependancies   :-)

Comment 4

[Ting-Yu Lin [:TYLin] (PDT, UTC-7)]

It's easy to disable the 3 seconds timeout by setting "layout.accessiblecaret.timeout_ms" to 0 [1].

[1] https://dxr.mozilla.org/mozilla-central/rev/b67316254602a63bf4e568198a5c7d3288a9db27/modules/libpref/init/all.js#4915

Comment 5

[James Willcox (:snorp) ([e-mail]) (he/him)]

Anthony I think we discussed not showing the action bar at all, instead of removing the 3 second timeout which hides it, right? The comment in #2 is a little unclear.

The change needed would be to never show the action bar with selection length of zero.

Comment 6

[Mark Capella [:capella]]

Um, but we just did Bug 1230582

Comment 7

[Mark Capella [:capella]]

Or do you never want to have a movable selection handle with a field of length > 0?

Comment 8

[Mark Capella [:capella]]

The next problem becomes you lose the ability to paste into empty fields, or between characters

Comment 9

[Anthony Lam (:antlam) Find me, @antlam on Slack!]

(In reply to Mark Capella [:capella] (nearby) from comment #6)
> Um, but we just did Bug 1230582

Sorry, but this bug should have been a WONTFIX. The UX behavior described in there doesn't seem to make sense for us. Nor is it something that other browsers do AFAICT (feel free to correct me if I'm wrong).

Single pressing on an input field should NOT trigger an action bar. But long-pressing should. 

(In reply to Mark Capella [:capella] (nearby) from comment #8)
> The next problem becomes you lose the ability to paste into empty fields, or
> between characters

Actually, users use long-press for this - not single-press. So, if a user long-presses on an input field, that should place the cursor inside, and trigger the action bar.

(In reply to James Willcox (:snorp) ([e-mail]) from comment #5)
> Anthony I think we discussed not showing the action bar at all, instead of
> removing the 3 second timeout which hides it, right? The comment in #2 is a
> little unclear.
> 
> The change needed would be to never show the action bar with selection
> length of zero.

Sorry if I was unclear before. Let me try this again :P

IF the user single-presses an input field > NO ACTION BAR (the intent is to type)

IF the user long-presses an input field > SHOW ACTION BAR (the intent is to copy/paste/do some sort of action)

IF the user long-presses a word in the input field to _select_ it > HIGHLIGHT + SHOW ACTION BAR (the intent is to copy/paste/do some sort of action WITH specific text)

Finally, I'm not sure why we're doing anything time-based here at all. So, I think we should remove it.

Hope that helps :)

Comment 10

[Ting-Yu Lin [:TYLin] (PDT, UTC-7)]

Re comment 9:

Actually, Bug 1230582 detoured the original design from b2g, and make it less like what Anthony imagines. I could help re-open the bug and revert the patch once the design decision is made.

Let me summaries what b2g did for comparison since we both use the nearly the same accessiblecaret code now. Also from the implementation of view, caret code sends events to control the copy/paste menu on b2g or actionbar on Fennec. So it's important to consider the caret appearance in all circumstances.

If the user single-presses an "empty" input field > No blue caret (only the blinking cursor), no copy/paste menu (for typing).

If the user long-presses an "empty" input field > No blue caret + show copy/paste menu (for copy/paste).  Note that we can have different appearance for caret and menu. Menu disappear after 3 seconds (for typing).

If the user single-presses a word in non-empty input field > show blue caret + copy paste/menu (for moving caret or copy/paste). Caret and menu disappear after 3 seconds (for typing).

If the user long-presses a word in non-empty input field > select and highlight the word + copy/paste menu (for moving caret or copy/paste with text).

Comment 11

[Anthony Lam (:antlam) Find me, @antlam on Slack!]

(In reply to Ting-Yu Lin [:TYLin] (UTC+8) from comment #10)
> Re comment 9:
> 
> Actually, Bug 1230582 detoured the original design from b2g, and make it
> less like what Anthony imagines. I could help re-open the bug and revert the
> patch once the design decision is made.
> 
> Let me summaries what b2g did for comparison since we both use the nearly
> the same accessiblecaret code now. Also from the implementation of view,
> caret code sends events to control the copy/paste menu on b2g or actionbar
> on Fennec. So it's important to consider the caret appearance in all
> circumstances.
> 
> If the user single-presses an "empty" input field > No blue caret (only the
> blinking cursor), no copy/paste menu (for typing).
>
> If the user long-presses an "empty" input field > No blue caret + show
> copy/paste menu (for copy/paste).  Note that we can have different
> appearance for caret and menu. Menu disappear after 3 seconds (for typing).
> 
> If the user single-presses a word in non-empty input field > show blue caret
> + copy paste/menu (for moving caret or copy/paste). Caret and menu disappear
> after 3 seconds (for typing).
> 
> If the user long-presses a word in non-empty input field > select and
> highlight the word + copy/paste menu (for moving caret or copy/paste with
> text).

Thanks for the added clarity :TYLin!

While under the hood, it might be the same, I think we work on very different platforms. It's important for us to be more platform agnostic to Android. Especially for these interactions that are just so common on the platform, we don't want to create unnecessary friction for our users. 

So, our behaviour needs to match Android more.

Comment 12

[Anthony Lam (:antlam) Find me, @antlam on Slack!]

NI-ing :ahunt here since he's working on these now :)

Comment 13

[:Margaret Leibovic]

(In reply to Anthony Lam (:antlam) from comment #12)
> NI-ing :ahunt here since he's working on these now :)

I believe ahunt is just working on updating the caret images (bug 1097398). This bug sounds like more of a platform problem.

TYLin, can you help fix this bug, since it sounds like the patch in bug 1230582 is what caused this problem?

Anthony, can you re-summarize this bug to reflect what changes need to be made here? We should address this as part of shipping these new text selection carets.

Comment 14

[Ting-Yu Lin [:TYLin] (PDT, UTC-7)]

I can help on this bug, but I'll wait for Anthony to summarize what need to be done here.

I imagine what need to be done here is that selection handle and actionbar need to be persist, i.e. without the 3 seconds timeout.

Do we need to show selection handle on an empty input? If not, bug 1230582 need revert or rework.

Comment 15

[Anthony Lam (:antlam) Find me, @antlam on Slack!]

(In reply to :Margaret Leibovic from comment #13)
> I believe ahunt is just working on updating the caret images (bug 1097398).
> This bug sounds like more of a platform problem.

OK! WFM.

> TYLin, can you help fix this bug, since it sounds like the patch in bug
> 1230582 is what caused this problem?
> 
> Anthony, can you re-summarize this bug to reflect what changes need to be
> made here? We should address this as part of shipping these new text
> selection carets.

Yes we should definitely address this before shipping these new carets.

(In reply to Ting-Yu Lin [:TYLin] (UTC+8) from comment #14)
> I can help on this bug, but I'll wait for Anthony to summarize what need to
> be done here.

Thanks :TYLin!

> I imagine what need to be done here is that selection handle and actionbar
> need to be persist, i.e. without the 3 seconds timeout.
> 
> Do we need to show selection handle on an empty input? If not, bug 1230582
> need revert or rework.

Hopefully I can summarize everything here :)

If input field is empty:
 - Single press on empty input field -> 
 -- keyboard comes up
 -- single cursor line 
 -- DO NOT SHOW any action bar 
 -- DO NOT SHOW any additional handle carets 

 - Long press on empty input field -> 
 -- single cursor line
 -- SHOW action bar indefinitely
 -- SHOW single handle caret under cursor line indefinitely

If input field is filled:
 - Single press on filled input field -> 
 -- keyboard comes up
 -- single cursor line where the press occurred
 -- DO NOT SHOW any action bar
 -- SHOW single handle caret directly underneath cursor line

 - Long press on filled input field -> 
 -- keyboard comes up
 -- SHOW action bar indefinitely (with select all, cut, copy, paste, etc actions we currently support)
 -- HIGHLIGHT all text inside indefinitely
 -- SHOW two handle carets on both ends indefinitely

There should be no time based behaviour anywhere. 

Please let me know if I left anything out! thanks again!

Comment 16

[Ting-Yu Lin [:TYLin] (PDT, UTC-7)]

(In reply to Anthony Lam (:antlam) from comment #15)
> If input field is filled:
>  - Single press on filled input field -> 
>  -- keyboard comes up
>  -- single cursor line where the press occurred
>  -- DO NOT SHOW any action bar
>  -- SHOW single handle caret directly underneath cursor line

In this case, if the user copied some text somewhere, and want to paste it into the filled input field, he might be confused that there's no actionbar. On B2G or Chrome browser on my Sony Z3C, I could single tap the caret to show the paste menu (our actionbar).

Which behavior do we prefer? Show actionbar directly on the single press on filled input or require the user to single tap on the caret to show the action bar.

Comment 17

[Anthony Lam (:antlam) Find me, @antlam on Slack!]

(In reply to Ting-Yu Lin [:TYLin] (UTC+8) from comment #16)
> In this case, if the user copied some text somewhere, and want to paste it
> into the filled input field, he might be confused that there's no actionbar.
> On B2G or Chrome browser on my Sony Z3C, I could single tap the caret to
> show the paste menu (our actionbar).

I don't think we need to worry about this. The more common case is single-tapping to type, not single-tapping to paste.

A single press should not bring up the action bar right now. If the user wants to paste, they can long-press to bring up the action bar. That's a very common convention in the Android platform and it's exactly how they got to copy the text in the beginning anyways.

So, let's stick with the following:
> If input field is filled:
>  - Single press on filled input field -> 
>  -- keyboard comes up
>  -- single cursor line where the press occurred
>  -- DO NOT SHOW any action bar
>  -- SHOW single handle caret directly underneath cursor line

thanks!

Comment 18

[Ting-Yu Lin [:TYLin] (PDT, UTC-7)]

Re comment 17:

No problem. I'll make single tap on filled input without showing the ActionBar. BTW, I found it is trivial to add the support to single tap on the caret to show up the action bar, so I just implement this anyways.

One last question. The default caret implementation in gecko is designed not showing caret when the input is empty no matter the user is single tapping or long pressing on it. After all, the caret is to assist the user to move cursor easily. When the input is empty, the cursor cannot be moved anyway. Do you feel it's OK to reuse the behavior? Or is there any benefits to show the caret when the input is empty when long-pressing?

Comment 19

[Anthony Lam (:antlam) Find me, @antlam on Slack!]

(In reply to Ting-Yu Lin [:TYLin] (UTC+8) from comment #18)
> Re comment 17:
> 
> No problem. I'll make single tap on filled input without showing the
> ActionBar. BTW, I found it is trivial to add the support to single tap on
> the caret to show up the action bar, so I just implement this anyways.

Hey :TYLin can you explain what you did here? this sounds confusing. I am especially worried about users looking to move the input position being surprised with an action bar. This is not expected behaviour and the UX of it is not something I am comfortable with. 

Although it might be trivial to add support for something, if we're not comfortable with the UX it brings, we shouldn't implement it.

So, some screenshots of what you're adding here would be great. :) just so we're on the same page.

> is there any benefits to show the caret when the input is empty when long-pressing?

There is a direct relationship between the caret and the action bar so i want to maintain that expectation. If the user long presses on an empty input field, we should show the action bar indefinitely, and show the single handle caret under the cursor line indefinitely also. 

Thanks :TYLin!

Comment 20

[Ting-Yu Lin [:TYLin] (PDT, UTC-7)]

(In reply to Anthony Lam (:antlam) from comment #19)
> Hey :TYLin can you explain what you did here? this sounds confusing. I am
> especially worried about users looking to move the input position being
> surprised with an action bar. This is not expected behaviour and the UX of
> it is not something I am comfortable with. 

Sorry I cannot explain my idea with user interaction with a screenshot. Let me explain as follows.

Per comment 15, single-press on filled input field does not show actionbar by default. As you explain in comment 17, it's a rare case that user want to insert (paste) something when there's only a single caret, and user could long press to bring up the action bar. However, long press is guaranteed to select some text. We cannot select zero character. The user could never paste something without replace the existing text. 

If the user does want to insert something, it might be good to allow the user to move the caret and then he could single tap on the caret to bring up to actionbar. When the user is moving caret, he must have long-pressed on the caret, the actionbar does not show in this case. This will be a feature I'd like to see. What do you think? How does other browser handle this case? I saw this behavior on Chrome on my Android 4.4 Sony Z3C as well.

> There is a direct relationship between the caret and the action bar so i
> want to maintain that expectation. If the user long presses on an empty
> input field, we should show the action bar indefinitely, and show the single
> handle caret under the cursor line indefinitely also. 

I agree that the direct visual relationship between the caret and the actionbar is important. However, what I feel is that this expectation is not met when user single press on a filled input since the actionbar does not show and caret shows. That's why I had my question in comment 16 to make the caret and actionbar show together in this case.

From the implementation point of view, user interaction is an event to make the system state change, which might trigger other system events like zooming in to make user see the input field clearly. It's infeasible to remember the user's previous action to determine the system state since you'll never know whether another system event is coming.

One example is single tap to not show caret while long press to show the caret on an empty input. To determine the visibility of a single caret, the only stable state is whether the input is empty or not. Either show the single caret on the input (empty or filled) at all times, or only show single caret on a filled input as the current implementation. I hope I could implement all of your request, but in some cases, it is not possible to do so.

Thanks Anthony!

Comment 21

[Anthony Lam (:antlam) Find me, @antlam on Slack!]

(In reply to Ting-Yu Lin [:TYLin] (UTC+8) from comment #20)
> Sorry I cannot explain my idea with user interaction with a screenshot. Let
> me explain as follows.

No worries :) thanks for taking the time to explain!

> Per comment 15, single-press on filled input field does not show actionbar
> by default. As you explain in comment 17, it's a rare case that user want to
> insert (paste) something when there's only a single caret, and user could
> long press to bring up the action bar. However, long press is guaranteed to
> select some text. We cannot select zero character. The user could never
> paste something without replace the existing text. 
> 
> If the user does want to insert something, it might be good to allow the
> user to move the caret and then he could single tap on the caret to bring up
> to actionbar. When the user is moving caret, he must have long-pressed on
> the caret, the actionbar does not show in this case. This will be a feature
> I'd like to see. What do you think? How does other browser handle this case?
> I saw this behavior on Chrome on my Android 4.4 Sony Z3C as well.

Sorry, but I don't think we should implement this behavior. It's too complicated of a nuance to understand and can get very frustrating because it's taking over a common action (that is, the action of touching the caret). 

I'd be happy to look at alternatives to address this issue in another bug if we think its really necessary but we should not implement this.

> I agree that the direct visual relationship between the caret and the
> actionbar is important. However, what I feel is that this expectation is not
> met when user single press on a filled input since the actionbar does not
> show and caret shows. That's why I had my question in comment 16 to make the
> caret and actionbar show together in this case.
> 
> From the implementation point of view, user interaction is an event to make
> the system state change, which might trigger other system events like
> zooming in to make user see the input field clearly. It's infeasible to
> remember the user's previous action to determine the system state since
> you'll never know whether another system event is coming.

Sorry, I think we're confusing the issues here. The interaction might make sense from the implementation point of view but our users simply are not implementing this. They're just using the product and the usability and experience you're describing here just doesn't make sense to me. 

I still don't think the caret should be a button that the user presses to launch something because its purpose in Android's design guidelines is only to move the cursor, anything else would be confusing to the user without a clear value proposition.

> One example is single tap to not show caret while long press to show the
> caret on an empty input. To determine the visibility of a single caret, the
> only stable state is whether the input is empty or not. Either show the
> single caret on the input (empty or filled) at all times, or only show
> single caret on a filled input as the current implementation. I hope I could
> implement all of your request, but in some cases, it is not possible to do
> so.

I think this is getting too complicated and out of scope. These situations are edge cases that we can revisit down the line.

For the time being, I think the right UX decisions here for single taps are only the following:

If input field is empty:
 - Single press on empty input field -> 
 -- keyboard comes up
 -- single cursor line 
 -- DO NOT SHOW any action bar 
 -- DO NOT SHOW any additional handle carets 

If input field is filled:
 - Single press on filled input field -> 
 -- keyboard comes up
 -- single cursor line where the press occurred
 -- DO NOT SHOW any action bar
 -- SHOW single handle caret directly underneath cursor line (where the user tapped)

> Thanks Anthony!

Thanks :TYLin!

Comment 22

[Ting-Yu Lin [:TYLin] (PDT, UTC-7)]

(In reply to Anthony Lam (:antlam) from comment #21)
> I think this is getting too complicated and out of scope. These situations
> are edge cases that we can revisit down the line.

Understood. I'll stick to current scope.

Comment 23

[Ting-Yu Lin [:TYLin] (PDT, UTC-7)]

Attachment: MozReview Request: Bug 1240917 - Do not show caret in empty input on Fennec. r=roc

Per request in bug 1240917 comment 15, we decided not to show caret when
single press on an empty input. This effectively reverts the work in Bug
1230582.

Review commit: https://reviewboard.mozilla.org/r/33209/diff/#index_header
See other reviews: https://reviewboard.mozilla.org/r/33209/

Comment 24

[Ting-Yu Lin [:TYLin] (PDT, UTC-7)]

Attachment: MozReview Request: Bug 1240917 - Turn off caret timeout on Fennec. r=snorp

Review commit: https://reviewboard.mozilla.org/r/33211/diff/#index_header
See other reviews: https://reviewboard.mozilla.org/r/33211/

Comment 25

[Ting-Yu Lin [:TYLin] (PDT, UTC-7)]

Attachment: MozReview Request: Bug 1240917 - Tweak ActionBar behavior when selection is collapsed. r=snorp

* Long pressing on empty input -> show the ActionBar.
* Single tapping on input (either empty or non-empty) -> do not show the
  ActionBar.

Review commit: https://reviewboard.mozilla.org/r/33213/diff/#index_header
See other reviews: https://reviewboard.mozilla.org/r/33213/

Comment 26

[Robert O'Callahan (:roc) (email my personal email if necessary)]

Comment on attachment 8714663 [details]
MozReview Request: Bug 1240917 - Do not show caret in empty input on Fennec. r=roc

https://reviewboard.mozilla.org/r/33209/#review29937

Comment 27

[James Willcox (:snorp) ([e-mail]) (he/him)]

Comment on attachment 8714665 [details]
MozReview Request: Bug 1240917 - Tweak ActionBar behavior when selection is collapsed. r=snorp

https://reviewboard.mozilla.org/r/33213/#review30035

Comment 28

[James Willcox (:snorp) ([e-mail]) (he/him)]

Comment on attachment 8714664 [details]
MozReview Request: Bug 1240917 - Turn off caret timeout on Fennec. r=snorp

https://reviewboard.mozilla.org/r/33211/#review30037

Comment 29

[Pulsebot]

https://hg.mozilla.org/integration/fx-team/rev/9a9874a77aa9
https://hg.mozilla.org/integration/fx-team/rev/580790343878
https://hg.mozilla.org/integration/fx-team/rev/ce6b2fdabc3e

Comment 30

[Carsten Book [:Tomcat]]

https://hg.mozilla.org/mozilla-central/rev/9a9874a77aa9
https://hg.mozilla.org/mozilla-central/rev/580790343878
https://hg.mozilla.org/mozilla-central/rev/ce6b2fdabc3e

Comment 31

[Teodora Vermesan (:TeoVermesan)]

Tested using:
Device: Nexus 6 (Android 6.0)
Build: Firefox for Android 47.0a1 (2016-02-04)

If input field is empty:
 - Single press on empty input field -> 
 -- keyboard comes up
 -- single cursor line (disappears after a few seconds)
 -- DO NOT SHOW any action bar 
 -- DO NOT SHOW any additional handle carets 

 - Long press on empty input field -> 
 -- single cursor line (disappears after a few seconds)
 -- SHOW action bar indefinitely
 -- DOES NOT SHOW single handle caret under cursor line indefinitely
(is this expected?)

If input field is filled:
 - Single press on filled input field -> 
 -- keyboard comes up
 -- single cursor line where the press occurred (disappears after a few seconds)
 -- DO NOT SHOW any action bar
 -- SHOW single handle caret directly underneath cursor line

- Long press on filled input field -> 
 -- keyboard comes up
 -- SHOW action bar indefinitely with all options
 -- HIGHLIGHT all text inside indefinitely
 -- SHOW two handle carets on both ends indefinitely

Comment 32

[Anthony Lam (:antlam) Find me, @antlam on Slack!]

I'm testing this on today's Nightly right now, Nexus 6P but some of the behavior is missing.. 

 - Long press on empty input field -> 
 -- SHOW single handle caret under cursor line indefinitely 
 --- (I don't see the single caret underneath)

 - Long press on filled input field -> 
 -- SHOW two handle carets on both ends indefinitely 
 --- (Sometimes I see this, but sometimes the carets are missing... not sure if this is just me?)

:TYLin, do you mind taking a look at my comments (in parenthesis above) and see if you're seeing the same thing? thanks! :)

Comment 33

[Ting-Yu Lin [:TYLin] (PDT, UTC-7)]

Teodora, Anthony, thank you for the testing.

Re comment #32
>  - Long press on empty input field -> 
>  -- SHOW single handle caret under cursor line indefinitely 
>  --- (I don't see the single caret underneath)

Sorry for not being clear in previous discussion. This is expected. I haven't implement this special case in this bug. Filed bug 1246064.

>  - Long press on filled input field -> 
>  -- SHOW two handle carets on both ends indefinitely 
>  --- (Sometimes I see this, but sometimes the carets are missing... not sure
> if this is just me?)

You're not alone. Mark saw this in bug 1245754 as well, but I don't see this. It would help me debug if there's STR on certain website.