Closed Bug 777972 Opened 7 years ago Closed 6 years ago

[responsive mode] translate click events to touch events

Categories

(DevTools :: Responsive Design Mode, defect, P2)

x86
All
defect

Tracking

(Not tracked)

RESOLVED FIXED
Firefox 26

People

(Reporter: paul, Assigned: paul)

References

(Blocks 1 open bug)

Details

Attachments

(3 files, 3 obsolete files)

No description provided.
Blocks: 749628
Blocks: 778169
I'm interested in tackling this one. I've got a working prototype that is generating touchstart events on mousedown, but I have some questions before putting together a complete patch.

Do you just want touchstart and touchend events to fire along with mousedown and mouseup, or should it cancel the mouse events?

Do you want to simulate touchmove events?

If touchmove is handled, do you want to simulate page scrolling as well?

Do you want click events to happen as normal, or should I try to simulate the 300ms delay a device would have and cancel if the user drags?

Should this just be always on in responsive mode, or would something like a checkbox next to the rotate button be good?

Are you interested in simulating gesture/multitouch events for people with multitouch touchpads? (I have no idea how to implement that one)

Sorry for so many questions, just want to make sure all the bases are covered.
(In reply to cpaul37 from comment #1)
Some input below :

> I'm interested in tackling this one. I've got a working prototype that is
> generating touchstart events on mousedown, but I have some questions before

This is awesome!
 
> Do you just want touchstart and touchend events to fire along with mousedown
> and mouseup, or should it cancel the mouse events?

We can have an option somewhere (either on the toolbar of responsive mode or options panel) to choose one of them (or both). The options could be like
[ ] Emulate touch events
[ ] Don't emit mouse events while emulating touch events

> Do you want to simulate touchmove events?

If possible, yes.
 
> If touchmove is handled, do you want to simulate page scrolling as well?

Maybe yes, maybe not. Not sure how difficult it is to get into the first patch. If this is not difficult, then yes, otherwise, followups .
 
> Do you want click events to happen as normal, or should I try to simulate
> the 300ms delay a device would have and cancel if the user drags?

What is the behavior on mobile devices ? If it is the latter, then do that.

> Should this just be always on in responsive mode, or would something like a
> checkbox next to the rotate button be good?

Like I said above, some sort of checkbox should be there.

> Are you interested in simulating gesture/multitouch events for people with
> multitouch touchpads? (I have no idea how to implement that one)

We can take this up in follow ups.
Here is a partial patch implementing some really basic touch event emulation. So far you can enable touch events which will fire alongside mousedown, mousemove if you're dragging, and mouseup. You can also suppress mouse events preventing mousedown, mousemove, and mouseup while emulating touch.

On an actual device, click events are simulated after 300ms if the phone hasn't detected some other type of action happening such as a scroll. I haven't implemented this delay/cancel functionality since it requires having scrolling/gesture detection to be useful. Click events do still fire after any mouseup/touchend events, so the device's simulated click behavior is partially covered.

I didn't implement scrolling on drag because getting it to behave like actual devices seems complicated, and if it doesn't behave like a device then it's not very useful. It also interacts with the 300ms delay described above, as well as multitouch/gesture events, so I think those could all be implemented together as part of a follow up.

I also haven't finished populating the touch events with data in this patch. This is a trivial but tedious task so I skipped it in favor of getting what I have out for review.

The one major limitation I have right now is I can't generate an actual TouchEvent. Currently I create a UIEvent and set the type to touchstart/touchmove/touchend. Functionally this is exactly the same as an actual TouchEvent, it just shows up as a UIEvent if you log it to the console.

Right now if you enable touch event emulation and don't suppress mouse events the behavior appears to be the same as the touch event emulation in Chrome, with the exception of the event data not being filled and the type being UIEvent.

If I'm on the right track here I can probably have a complete patch ready sometime tomorrow.
Attachment #769466 - Flags: review?
Wow, happy to see someone looking at this!

@cpaul, some context: we don't want to perfectly emulate the touch experience. For example, we don't need multitouch recording, or kinetic scrolling. But we want basic events to be well supported.

We have been doing that in the past for desktop versions of Fennec also for the Gaia simulator.

If you can re-use existing code, that would be much better.

See here: https://github.com/mozilla-b2g/gaia/blob/master/tools/extensions/desktop-helper/content/touch-events.js

Ask me for reviews.

Thanks!
Comment on attachment 769466 [details] [diff] [review]
Simulated touch events in responsive mode

Cancelling the review.

See if you can use Gaia code (you will have to copy this file in devtools/shared). Also - we will need some mochitests.
Attachment #769466 - Flags: review?
@cpaul, should I assign this bug to you?
Component: Developer Tools → Developer Tools: Responsive Mode
Priority: -- → P2
(In reply to Paul Rouget [:paul] from comment #6)
> @cpaul, should I assign this bug to you?

Yes, please do.
Assignee: nobody → cpaul37
@cpaul, let me know if you need any help here.
@cpaul, still working on this?
I'm still working on it, I've just been really busy the last few weeks. If anyone else wants to take this that's cool, otherwise I'll have it done soon.
Attached patch patch1.diff (obsolete) — Splinter Review
Assignee: cpaul37 → paul
Attachment #769466 - Attachment is obsolete: true
Status: NEW → ASSIGNED
Comment on attachment 797251 [details] [diff] [review]
patch1.diff

Boris, in the developer tools, we want to simulate touch events. We basically translate mouse events to touch events. So we need to use `document.createEvent("touchevent")`, which is not possible if the pref `w3c_touch_events` is not set to `1`. We programmatically  switch the pref in the developer tools when the "simulate touch events" feature is turned on, but because this pref is cached in nsDOMTouchEvent.cpp, it doesn't work. We could use Preferences::RegisterCallback and re-request the pref value in nsDOMTouchEvent.cpp, but I thought it might be better to just not cache the value.

Let me know what you think.
Attachment #797251 - Flags: review?(bzbarsky)
Attached patch patch2.diff (obsolete) — Splinter Review
Attachment #797258 - Flags: review?(mratcliffe)
Attachment #797258 - Flags: review?(mratcliffe) → review+
Comment on attachment 797251 [details] [diff] [review]
patch1.diff

Olli, does this seem reasonable to you?  Note that this function is called during HTMLElement prototype setup and a few other places... but I think this should still be ok.
Attachment #797251 - Flags: review?(bzbarsky) → review?(bugs)
I don't know how slow or fast mozilla::widget::IsTouchDeviceSupportPresent() is, so its return value
should be cached. And nsContentUtils::InitializeTouchEventTable() should be called just once...
except that nsContentUtils::InitializeTouchEventTable() itself makes sure it is initialized just once.

So, cache the return value of mozilla::widget::IsTouchDeviceSupportPresent()
Attachment #797251 - Flags: review?(bugs) → review-
Attachment #797251 - Attachment is obsolete: true
Attachment #797851 - Flags: review?(bugs)
Mike, I had to change a couple of things:
- reload if needed
- new way to handle the pref
Attachment #797258 - Attachment is obsolete: true
Attachment #797852 - Flags: review?(mratcliffe)
Attachment #797852 - Flags: review?(mratcliffe) → review+
Mike, because some events might have been added before the button is clicked, the user might need to reload the page to make the listeners work. I think we should show a notification.

Can you also please check the locales are in good english?
Attachment #797920 - Flags: review?(mratcliffe)
Attachment #797851 - Flags: review?(bugs) → review+
Comment on attachment 797920 [details] [diff] [review]
Patch v1 - add notification box

Review of attachment 797920 [details] [diff] [review]:
-----------------------------------------------------------------

Simple enough
Attachment #797920 - Flags: review?(mratcliffe) → review+
Product: Firefox → DevTools
You need to log in before you can comment on or make changes to this bug.