Closed Bug 501440 Opened 11 years ago Closed 11 years ago

Support selecting text in touch-enabled devices

Categories

(Core :: Widget: Win32, defect)

1.9.1 Branch
x86
Windows 7
defect
Not set

Tracking

()

RESOLVED FIXED
mozilla1.9.2a1
Tracking Status
status1.9.1 --- .4-fixed

People

(Reporter: Felipe, Assigned: Felipe)

References

(Blocks 1 open bug)

Details

(Keywords: verified1.9.2)

Attachments

(1 file)

User-Agent:       Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US; rv:1.9.1.1pre) Gecko/20090630 Shiretoko/3.5.1pre (.NET CLR 3.5.30729)
Build Identifier: Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US; rv:1.9.1.1pre) Gecko/20090630 Shiretoko/3.5.1pre (.NET CLR 3.5.30729)

Currently when using a touch tablet, it's only possible to pan the page using the 1 or 2 finger scrolling. We should be able to select text as well.

The simple case for selecting text (and fortunately the most common case I think) is when starting the selection horizontally in a page that doesn't have horizontal scrollbars.

For other cases, IE currently does some guessing, apparently based on the distance from nearby text nodes.

We should determine how to deal with both scrolling and panning together and implement it.

Reproducible: Always

Steps to Reproduce:
1. Try to select text on a touch tablet
Actual Results:  
You can't. The page always pan. If there are no scrollbars on the page, nothing happens.

Expected Results:  
User should be able to select text, both from input fields as well as regular text nodes.
Blocks: 488715
Version: unspecified → 1.9.1 Branch
Status: UNCONFIRMED → NEW
Ever confirmed: true
This will likely end up a dupe of bug 492791, but let's leave it open in case splitting the selection stuff off makes sense.
Attached patch Patch rev1Splinter Review
This patch allows to select text and drag elements through horizontal movements, as can be seen for example in http://script.aculo.us/

This behaves the same way as IE currently does.
Attachment #386634 - Flags: review?(jmathies)
well, let me just improve the information here: this is not just how IE behaves, it is the system default setting, per comment 17 of bug 485101. Horizontal single finger triggers default mouse click+hold (which allows text selection and dragging), and vertical triggers touch panning.
(In reply to comment #3)
> well, let me just improve the information here: this is not just how IE
> behaves, it is the system default setting, per comment 17 of bug 485101.
> Horizontal single finger triggers default mouse click+hold (which allows text
> selection and dragging), and vertical triggers touch panning.

If the page the user is viewing has horizontal scroll bars visible and they try and pan horizontally using a single finger, would we default to panning or text selection with this patch applied?
(In reply to comment #4)
> If the page the user is viewing has horizontal scroll bars visible and they try
> and pan horizontally using a single finger, would we default to panning or text
> selection with this patch applied?

We would default to text selection. To pan, the user would have to start the movement vertically. Once a vertical movement triggers panning, then it's possible to keep panning in any direction.
The pages I tested with the patch worked much better than without and appear to emulate the same behavior as IE. Felipe, could you test the behavior of IE when there is only a horizontal scrollbar to see if it is possible to initiate single finger panning?
(In reply to comment #6)
> Felipe, could you test the behavior of IE when
> there is only a horizontal scrollbar to see if it is possible to initiate
> single finger panning?

Just tested on IE and turns out you cannot scroll horizontally when there's no vertical scrollbar. It always starts a selection, and you can't scroll the site. An example page for testing: http://www.peter-pearson.com/

Even with two fingers it acts a little weird. It looks like the two fingers triggers a zoom even if I keep my fingers firmly aligned, and then after the zoom it is able to pan due to the zoomed page getting a scrollbar.
I'm wondering if selection should be tied to the presence of horizontal scroll. (Bug 492791 really ties into this as that information is up in the view.) In playing with it I became accustomed to moving things around with a single finger, the selection stuff seemed to "get in the way". The selection though would be nice for well formatted pages that didn't have horizontal scroll distance showing.

I really think we need to move control over this type of stuff out of widget to an area where we have more knowledge about the state of the view. (We could use events to get this information down into widget, but that seems like a hack to me.) What we could do is add an interface to the gesture stuff that you retrieve through nsWindow, then the view could control how it's configured, the feedback, etc.. I think we're going to need this anyway to fix up some of the other problems with menus and scroll bars.

All of this though really can't be done until we convert from mouse event simulation to some form of touch / gesture gui event.
Yeah I agree that we will need more fine grained control to cover well most of the use cases for webpages and our UI. But while we're not at it I believe that this change makes the situation much better because it makes it possible to select text, drag tabs and even sites like Google Maps works.  Currently none of this is possible if single finger always pan.  Horizontal scroll is not that common and if the user really needs it it can be triggered with two fingers.
building up some try builds to take a look.
Hmm, on 7100 two finger pan wasn't working with these builds. I'll get this tablet upgraded today to the RC and see if that addresses it..
Assignee: nobody → felipc
Hi Jim, for these to work on trunk you'll have to have the patch from bug 494281 applied as well.
(In reply to comment #13)
> Hi Jim, for these to work on trunk you'll have to have the patch from bug
> 494281 applied as well.

That shouldn't affect this as long as I don't close a window.

I guess 7100 is the RC, does that jibe with what you guys have? I'm updating my drivers to 2.59 now to be sure I have the latest.
Hi Jim, from my tests this did affect the touch. It looks like some sub-windows are created and destroyed during startup, so the touch never worked even in the first window.

But yes, I just checked and the windows 7 RC from here is indeed build 7100. The driver version installed here is the 2.59 so we will be with the same settings. Obs.: the pen works here after we upgraded to 2.59.
Attachment #386634 - Flags: review?(jmathies) → review+
Comment on attachment 386634 [details] [diff] [review]
Patch rev1


Ok, this seems fine with me. At first I found myself trying to drag the page around single finger and accidentally triggering selection, but after a while you get used to it.
(In reply to comment #15)
> Hi Jim, from my tests this did affect the touch. It looks like some sub-windows
> are created and destroyed during startup, so the touch never worked even in the
> first window.
> 
> But yes, I just checked and the windows 7 RC from here is indeed build 7100.
> The driver version installed here is the 2.59 so we will be with the same
> settings. Obs.: the pen works here after we upgraded to 2.59.

Ok, cool. Lets get sr's on both of these, land them on trunk and nom for approval1.9.1.1, hopfully they'll make it in.

Nice work Felipe!
Comment on attachment 386634 [details] [diff] [review]
Patch rev1

vlad, could you take a look at this patch?
Attachment #386634 - Flags: superreview?(vladimir)
Madhava - is text selection in pannable contextes something we want in Fennec too?
note: this is specific to Win7's WM_GESTURE events and GID_PAN which isn't part of mobile touch.
Yes, the code for this patch is win7 sdk based, although the interaction discussed here might be valuable for Fennec too.

Should I be using a different term instead of "touch enabled devices" to make the description clearer about that?
Attachment #386634 - Flags: superreview?(vladimir) → superreview+
Pushed to mozilla-central
http://hg.mozilla.org/mozilla-central/rev/db7c21404bb4
Status: NEW → RESOLVED
Closed: 11 years ago
Resolution: --- → FIXED
Target Milestone: --- → mozilla1.9.2a1
Summary: Support selecting text in touch enabled devices → Support selecting text in touch-enabled devices
Comment on attachment 386634 [details] [diff] [review]
Patch rev1

This is simply a configuration change that enables single finger text selection. We'd like this for 1.9.1.2.
Attachment #386634 - Flags: approval1.9.1.2?
Comment on attachment 386634 [details] [diff] [review]
Patch rev1

Jim: does this affect the mozilla-1.9.1 branch? If so, please renominate for 1.9.1.3
Attachment #386634 - Flags: approval1.9.1.2? → approval1.9.1.3?
Attachment #386634 - Flags: approval1.9.1.3?
Comment on attachment 386634 [details] [diff] [review]
Patch rev1

This does affect mozilla-1.9.1 and is a usability improvement when using touch
Attachment #386634 - Flags: approval1.9.1.3?
Comment on attachment 386634 [details] [diff] [review]
Patch rev1

Approved for 1.9.1.4, a=dveditz for release-drivers
Attachment #386634 - Flags: approval1.9.1.3? → approval1.9.1.4+
You sir, should use the ".4-fixed" option of the status1.9.1 flag to indicate this has landed.
Keywords: fixed1.9.1
bah... old habits die hard
Verified fixed on the 1.9.2 branch using Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US; rv:1.9.2b1pre) Gecko/20090923 Namoroka/3.6b1pre (.NET CLR 3.5.30729). Adding keyword.
Keywords: verified1.9.2
Blocks: 548100
You need to log in before you can comment on or make changes to this bug.