1531311 - Improve UX when picking elements on remote browsers

Status: RESOLVED FIXED

(DevTools :: about:debugging, enhancement, P3)

Description

[Hani Yacoub, Desktop QA]

[Affected versions]:
Nightly 67.0a1

[Affected platforms]:
Platforms: Windows 10 x 64, Mac OS X 10.14, and Ubuntu 16.04 x64.

[Steps to reproduce]:

1. Make sure that you are connected to a mobile device using about:debugging, following these steps:
   a. In about:config set:
   • devtools.aboutdebugging.network -> true
   • devtools.aboutdebugging.new-enabled -> true
     b. Enable remote debugging in the device -> https://developer.mozilla.org/en-US/docs/Tools/WebIDE/Troubleshooting
2. From "about:debugging" make sure to be connected to a mobile device.
3. On your mobile device open many tabs (ex. reddit.com, amazon.com) and make sure this is reflected in the Desktop UI.
4. Inspect any of the opened tabs.
5. Click on "Pick an element from the page".

[ Expected result]:
Should remove the feature.
Future plans: Show the UI of the Device in about:devtools-toolbox

[Actual result]:
It's not possible to pick an element from the page when you are connected via USB

Comment 1

[Julian Descottes [:jdescottes]]

I suppose this was discussed in the standup, but the idea for now is just to hide the feature in about:devtools-toolbox.

Comment 2

[Julian Descottes [:jdescottes]]

My plan here would be to only disable/remove the feature when the debugged target is on a remote browser. It can still be helpful when debugging a local addon for instance, so completely removing it doesn't sound great.

We should still decide whether it's best to hide the icon completely or to disable it with a tooltip (eg "Unavailable when debugging a remote browser").

Comment 3

[Julian Descottes [:jdescottes]]

Attachment: Bug 1531311 - Hide element Picker button for remote targets

Comment 4

[Julian Descottes [:jdescottes]]

Attachment: Bug 1531311 - THIS_FIREFOX connection type is not recognized;r=ladybenko

Depends on D28065

Using split("-") was clashing with the connection type "this-firefox".
Renamed to "this_firefox" for now.
Added unit test for remote-client-manager.

Comment 5

[Julian Descottes [:jdescottes]]

Attachment: Bug 1531311 - Add mochitest for element picker display with remote targets

Depends on D28066

Comment 6

[Alexandre Poirot [:ochameau]]

Is it really broken?

I've not tested with a device, but it still works when debugging a "remote" firefox.
When you click on "pick an element", we expect you to click on the element you want to inspect on the remote.
On a remote firefox, you have to click on an element in the debugged tab, you will see the highlighter working.
On a remote device, you have to touch your screen, on the debugged tab to select an element. The highlighter should display the selected element.

Now. It isn't working like chrome devtools which replicated the device screen, but it still works and is useful.

Comment 7

[Julian Descottes [:jdescottes]]

Indeed it works! Just tried with a device, and yes if you touch the screen of the device after clicking on the element picker button you will select it in the inspector.

The lack of user feedback and explanations make it feel broken to me. I wonder if this feels intuitive to others?

I won't push further for disabling the button, but I think we should have some user feedback here. Maybe a message saying "click or touch the element you want to select in your remote browser" displayed in the toolbox? Or maybe a message in the remote browser (although it may hide the element you actually want to touch so maybe not a great idea...)

I will cancel the review requests here, I will just spawn another bug to fix the remote-client-manager bug I spotted while writing the patches here.

Comment 8

[Phabricator Automation]

Comment on attachment 9059270 [details]
Bug 1531311 - THIS_FIREFOX connection type is not recognized;r=ladybenko

Revision D28066 was moved to bug 1545482. Setting attachment 9059270 [details] to obsolete.

Comment 9

[Nicolas Chevobbe [:nchevobbe]]

Attachment: node picker notice highlighter

First thing that came to my mind was that we could put an highlighter on the remote debuggee to indicate the node picker was initiated, with a way to close it in case the user wants to pick something underneath (see screenshot)

Comment 10

[Nicolas Chevobbe [:nchevobbe]]

Attachment: Bug 1531311 - [devtools] Move command-pick.svg and close.svg to devtools/shared. r=jdescottes.

The images will be used in the RemoteNodePickerNotice (later in this patch queue),
so we need those both on the client and the server.

Depends on D132224

Comment 11

[Nicolas Chevobbe [:nchevobbe]]

Attachment: Bug 1531311 - [devtools] Add more visibility for node picker when debugging remote pages. r=jdescottes.

Depends on D132704

Comment 12

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/b6344e7d67bf
[devtools] Add more visibility for node picker when debugging remote pages. r=jdescottes,devtools-backward-compat-reviewers.

Comment 13

[Alexandru Michis [:malexandru]]

https://hg.mozilla.org/mozilla-central/rev/b6344e7d67bf