Web Replay: Basic Inspector devtools support

RESOLVED FIXED in Firefox 68

Status

()

enhancement
RESOLVED FIXED
9 months ago
2 months ago

People

(Reporter: bhackett, Assigned: bhackett)

Tracking

(Blocks 1 bug)

Trunk
mozilla68
Points:
---
Dependency tree / graph

Firefox Tracking Flags

(firefox64 wontfix, firefox68 fixed)

Details

Attachments

(11 attachments, 4 obsolete attachments)

45.49 KB, patch
Details | Diff | Splinter Review
47 bytes, text/x-phabricator-request
Details | Review
47 bytes, text/x-phabricator-request
Details | Review
47 bytes, text/x-phabricator-request
Details | Review
47 bytes, text/x-phabricator-request
Details | Review
47 bytes, text/x-phabricator-request
Details | Review
47 bytes, text/x-phabricator-request
Details | Review
47 bytes, text/x-phabricator-request
Details | Review
47 bytes, text/x-phabricator-request
Details | Review
47 bytes, text/x-phabricator-request
Details | Review
47 bytes, text/x-phabricator-request
Details | Review
Assignee

Description

9 months ago
Posted patch WIP (obsolete) — Splinter Review
The attached patch adds basic support for the DOM inspector in recording/replaying tabs --- the DOM is shown, but nothing else (searching, highlighting elements on screen, etc) works.  While I don't think I'm going to work on this anymore right now, I was curious about what would be necessary to get the inspector working with the current state of Web Replay's architecture.

The main new challenge here is allowing devtools server code to inspect the DOM in the replaying process.  Since we can already use the ReplayDebugger to create debugger objects that both wrap and allow inspection of objects in the replaying process, this patch adds a ReplayInspector interface for creating proxy veneers over ReplayDebugger.Objects.  These proxies can access properties or be called like in-process objects, performing such operations by calling into the underlying ReplayDebugger.Object.  This doesn't work for things like 'for of' or logic that mixes in-process and out-of-process data like domArray.some(callback), but these should be fixable.

The main limitation of basing this on Debugger.Object is that all the Debugger.Objects are invalidated when the process unpauses.  So this can't really be used for updating the inspector contents in a live fashion as the page plays forward or rewinds.  Doing that will require some way of dealing with this object-permanence problem (having an identifier for things that persists after unpausing), either for JS objects in general or more specifically for DOM elements.  As is, it shouldn't be hard to have an inspector that is grayed out when the process unpauses and is filled in when the process pauses.
Assignee

Updated

9 months ago
Depends on: 1498012
Assignee

Comment 1

4 months ago
Posted patch WIP (obsolete) — Splinter Review

Updated WIP. The general goal of the approach here is to minimize the number of changes to the inspector. Some new logic is necessary to get a window and tree walker that operate on the child process, but actual accesses are mediated through proxies and other objects in replay/inspector.js that enable the server to operate on objects in the child process in the same way it would normally operate on objects in its own process.

Constructing the DOM tree and layout information for each node seem to be working. The main remaining steps are getting the element highlighter and object picker to work, supporting inspecting CSS information, and changing the UI so that the inspector is grayed out when the debuggee is unpaused. The child will need to be paused in order to inspect its DOM.

Assignee: nobody → bhackett1024
Attachment #9015414 - Attachment is obsolete: true
Assignee

Comment 2

4 months ago
Posted patch WIP (obsolete) — Splinter Review

Updated WIP with support for the element highlighter and object picker. Getting these to work was actually pretty easy, thanks in large part to how graphics are handled in the middleman process. The graphics data from the child process are copied into a canvas in a normal document in the middleman. The element highlighter can then do its normal thing of drawing an overlay in the current window, and everything just works. (This should also be useful for other drawing we'd like to do while replaying, like showing the original mouse cursor position.)

Attachment #9047846 - Attachment is obsolete: true
Assignee

Comment 3

4 months ago
Posted patch patchSplinter Review

Complete patch, including a test (I'll try to write some more as well). While there will still be a fair amount of followup work and fixes to get the inspector working smoothly, this patch adds basic support. Things that have been tested on a small page:

  • When the replaying process is unpaused, the inspector's markup pane is blank (eventually we'll want to show a message, but that is better for followup).

  • When the replaying process pauses, the inspector's markup pane show the current state of the page.

  • The element highlighter, picker, and eyedropper all work.

  • In the markup pane, attributes, contents etc. are shown. Event handlers should also be shown and work (I tested with a basic DOM0 handler, I'm sure there's more to do to get this working in all cases).

  • The styles pane in the middle and the computed pane on the right should work, though I only tested basic inline styles.

  • The box model in the layout pane on the right should work.

As followup, it would also be good to hide all the features which aren't supported, especially those based around being able to edit the DOM or style information. I don't think we should support this initially, for simplicity. Eventually we could, but any changes would only apply while the process was paused, and would be reverted to the original state when the process resumes, which seems like it would be confusing for users.

Attachment #9047891 - Attachment is obsolete: true

Comment 13

3 months ago
Pushed by bhackett@mozilla.com:
https://hg.mozilla.org/integration/mozilla-inbound/rev/f89cf2e076ec
Part 1 - Add isReplaying global to server bindings, r=loganfsmyth.
https://hg.mozilla.org/integration/mozilla-inbound/rev/b23b6d4643d7
Part 2 - Tweak definition of target.isReplayEnabled(), r=loganfsmyth.
https://hg.mozilla.org/integration/mozilla-inbound/rev/d0c8809342e4
Part 3 - Remove ReplayDebuggerObject.global, r=loganfsmyth.
https://hg.mozilla.org/integration/mozilla-inbound/rev/47c9d34dbef1
Part 4 - Suppress event handling at the usual times in server when replaying, r=loganfsmyth.
https://hg.mozilla.org/integration/mozilla-inbound/rev/6c051fac78f3
Part 5 - Add ReplayInspector module for accessing objects in a replaying process, r=loganfsmyth.
https://hg.mozilla.org/integration/mozilla-inbound/rev/da8ea59a87f4
Part 6 - Emit event when the debugger pauses or resumes, r=loganfsmyth.
https://hg.mozilla.org/integration/mozilla-inbound/rev/269961cd844f
Part 7 - Server side changes for inspector support while replaying, r=pbro.
https://hg.mozilla.org/integration/mozilla-inbound/rev/263c51a3b7b4
Part 8 - Client side changes for inspector support while replaying, r=pbro.
https://hg.mozilla.org/integration/mozilla-inbound/rev/46a995ea433f
Part 9 - Add test for inspector support while replaying.
Attachment #9049841 - Attachment is obsolete: true

Comment 16

3 months ago
Pushed by jlaster@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/5859134b057b
replace onPauseChange with native threadClient event. r=bhackett

Comment 17

3 months ago
Backout by nbeleuzu@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/15883cbc7171
Backed out changeset 5859134b057b for merge conflicts. CLOSED TREE
Flags: needinfo?(jlaster)

Comment 19

3 months ago
Pushed by jlaster@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/98f1b0d8ff4b
replace onPauseChange with native threadClient event.
Summary: Web Relay: Basic Inspector devtools support → Web Replay: Basic Inspector devtools support
Flags: needinfo?(jlaster)
You need to log in before you can comment on or make changes to this bug.