881898 - Create a 'heads up display' for user interactions

Status: RESOLVED INACTIVE

(Remote Protocol :: Marionette, defect, P5)

Description

[Dave Hunt [:davehunt] [he/him] ⌚BST]

It would be really awesome if we could have visual feedback for user interactions (tap, move, etc). Often times we're interacting with the wrong element or our offsets are wildly off.

If we could optionally switch on a mode where interactions result in drawing on top of the viewport (something like a red circle or cross-hair) it could help test writers to hone in on the desired target.

Comment 1

[Dave Hunt [:davehunt] [he/him] ⌚BST]

The other thing that would be useful for this mode is to highlight the boundaries of the discovered element whenever using find_element. Perhaps overlay a new element with the same location/size, or similar.

Comment 2

[Kartikaya Gupta (email:[e-mail])]

Bug 964904 is very related to this, I think. But it likewise has not gotten much traction beyond the initial phase. However, now that bug 920036 has landed and the flow of touch events is a little more sane I think it would be worth revisiting this. It would help to describe what you had in mind so that we can figure out where in the flow it fits in. When I think about this I'm picturing a setting in the developer tools that you can turn on, and then anytime you touch the screen (in any app, anywhere) you get some sort of visual feedback. Then the question is what should the visual feedback represent. Should it show the fluffing area (this is what bug 964904 was intending to show)? Should it highlight the element(s) that are receiving the events? I'm not sure what the use cases here are exactly.

Comment 3

[Naoki Hirata :nhirata (please use needinfo instead of cc)]

I think showing the fluffing area and the type of touch it detected would help.
ie a triangle for long tap, square for double tap, circle for tap and the size would be the area of tap detected. Drag, and pinch zoom would also help....

UX could hash out the icons, I was only providing example of what I would like to see.  Also a debug log output of the fluff area as a debug option for trouble shooting would help as well.

Comment 4

[Dave Hunt [:davehunt] [he/him] ⌚BST]

I had in mind a red circle that would indicate the catchment area for a tap (I suppose that's what you mean by fluffing area). For finger movements I was thinking a red line that traces the interaction and perhaps fades away after a few seconds. I also like Naoki's idea of indicating when a double tap and long press are detected. In general it sounds like we're in agreement.

It would be necessary for Marionette to be able to enable this on demand, perhaps by using a preference or setting?

Comment 5

[David Burns :automatedtester]

[Mass closure] Closing as these features are no longer needed or wanted

Comment 6

[Dave Hunt [:davehunt] [he/him] ⌚BST]

Reopening as myself and :ato agree this would be a valuable feature.

Comment 7

[David Burns :automatedtester]

realistically this is probably never going to be implemented as it is low on the priority list.