1036722 - Emulate mouse/pointer events from devtools

Status: NEW

(DevTools :: Inspector, enhancement, P2)

Description

[Jeff Griffiths (:canuckistani) (:⚡︎)]

Not all hover effects use :hover in css, there should be a way to do this with JS-based hover effects as well, example: go to MDN, hover your mouse over 'Web Platform'. There is no :hover involved in showing the sub-menu, so selecting :hover does nothing making this sort of code difficult to debug. Emulating the presence of a pointer over a selected node might help instead?

Comment 1

[Patrick Brosset <:pbro>]

That would be a pretty awesome tool indeed. I can almost see this as a "virtual pointer" toolbar button, where you can drag and drop it over the page, wherever you want, make it simulate mouseover/move/down/...
This way it would be easy to move it so your menus and submenus expand and stay there as long as you have that tool activated. Even if you use the real pointer to do things with the devtools or on the page, that wouldn't trigger anything for as long as this second virtual pointer is here.

With some platform support, I suspect this might be feasible. We would need an API that allows to stop reacting to the actual pointer and instead let us drive it entirely programmatically.

Comment 2

[Girish Sharma [:Optimizer]]

The same is somewhat true for typeahead suggestions.

You cannot inspect those suggestions (without using super clever techniques)

The thing with typeahead suggestions is that when the suggestions box is open, you right click the node and as soon as you click on inspect node, the suggestions box disappears. Basically, as soon as the input box or the popup looses focus, it closes the suggestions box.

Comment 3

[Mike Ratcliffe [:miker] [:mratcliffe] [:mikeratcliffe]]

Why limit this to mouse events? There are a whole host of events that it would be useful to trigger: touch events, scroll etc.

I am pretty sure there is a bug somewhere to add triggering of events from the markup view context menu.

Would we disable real mouse events when this is active? Otherwise, if you emulate a mouseover by dragging a pointer then moving your real pointer out of the element the real pointer will still trigger a mouseout event.

Comment 4

[Jeff Griffiths (:canuckistani) (:⚡︎)]

(In reply to Michael Ratcliffe [:miker] [:mratcliffe] from comment #3)
> Why limit this to mouse events? There are a whole host of events that it
> would be useful to trigger: touch events, scroll etc.

I'd like to limit this bug to pointer emulation :)

> I am pretty sure there is a bug somewhere to add triggering of events from
> the markup view context menu.
> 
> Would we disable real mouse events when this is active?

Maybe for the document, but not generally ( you'd still need the mouse in the toolbox for example )

> Otherwise, if you
> emulate a mouseover by dragging a pointer then moving your real pointer out
> of the element the real pointer will still trigger a mouseout event.

The way I see it working is: a developer puts the mouse where they need the pointer to 'stick', then enables the mode somehow which puts the current document into a state where additional mouse events are ignored. The can then do their work, then 'un-stick' the mouse in some way when they're done.

Comment 5

[Stephanie Hobson [:shobson]]

I like :pbrosset's idea of dragging a virtual pointer off the tool bar and onto the page. When you're done you can drag it back to the toolbar and 'dock' it. It might also be handy to be able to place over top an element from the inspector pane using a context menu, the same way that :hover is triggered now.

Another good reason to add this feature is that it would trigger hover on all the parent elements! I like it better and better.

Comment 6

[Mike Ratcliffe [:miker] [:mratcliffe] [:mikeratcliffe]]

If Canuckistani wants to limit this to pointer events then pbrosset's draggable virtual pointer makes sense... not sure how we would choose which event it sends though.

When the virtual pointer is over an element we should disable all events on the page. I am not sure that we have a way to disable all events though and we do want our own event to fire.

Simulating mouse events is fairly simple:
function simulateClick(node) {
  var evt = document.createEvent("MouseEvents");
  evt.initMouseEvent("click", true, true, window,
    0, 0, 0, 0, 0, false, false, false, false, 0, null);
  var canceled = !node.dispatchEvent(evt);
  if(canceled) {
    // A handler called preventDefault
    alert("canceled");
  } else {
    // None of the handlers called preventDefault
    alert("not canceled");
  }
}

Comment 7

[Manish Goregaokar [:manishearth]]

One (hackish) way to do it would be to put a mouse event capturing overlay-thing (z-index: a million, transparent), and when the mouse event is triggered via dragging of pointer, it is removed when the pointer is being dragged (your mouse is on the draggable pointer at this point in time, so it can't use the lack of overlay to interfere with mousemove)

Comment 8

[Julian Descottes [:jdescottes]]

Inspector bug triage. Filter on CLIMBING SHOES.

Comment 10

[Patrick Brosset <:pbro>]

Attachment: virtual-pointer.diff

I don't have time right now to push this further, but here's an idea for how to implement this. I've demo'd it recently to the rest of the DevTools team and there was some excitement that this would be very useful.
This patch works a little bit at least, but there are many other things to fix.

Having bug 1177346 for this would be great, since it would give us a way to suppress only content events, which we would need here in order to allow clicking on virtual pointer buttons.

I'll attach a GIF that shows how far I've gone with this.

Comment 11

[Patrick Brosset <:pbro>]

Attachment: vp.gif

Comment 12

[Stephanie Hobson [:shobson]]

The GIF is amazing. As a dev I would definitely find this useful!