1731714 - Prototype a chrome API for saving a skeleton UI based on the prominent shapes on a webpage

Status: RESOLVED FIXED

(Core :: Layout, task)

Description

[Brian Grinstead [:bgrins]]

We'd like to experiment with saving a low fidelity version of a page to potentially render in place of an unloaded browser while the page loads.

One of the unknowns is the technical feasibility of grabbing a representative set of rectangles on the page that would roughly match the type of skeleton UIs used by web content. This bug is to get some sense of that feasibility and not to actually ship anything. The rough thought in mind is a function available from ChromeUtils that returns the current relevant rects (bounds, size, colors, etc) given a page.

Comment 1

[Emilio Cobos Álvarez (:emilio)]

Attachment: Bug 1731714 - Add a document.getWireframe() API that gets some relevant rects in the viewport. r=mconley!

I added an option to return also the nodes because it's very helpful for
debugging. A bunch of the "unknown" ones are just elements with
transparent backgrounds (but still hit-testable), so we probably want to
exclude them from the result list or what not to massively reduce the
size of the returned object.

But anyhow, this probably does for a prototype.

Comment 2

[Emilio Cobos Álvarez (:emilio)]

Attachment: Example result for facebook.com

What I've done to test this is basically copy(JSON.stringify(tabs[0].content.document.getWireframe())) from the inspector's console for a given PID. This is the result for facebook.com.

As mentioned in the commit message, there's a lot of "unknown" rects that should probably go away (they're mostly useful for debugging when combined with showNodes=true).

Comment 3

[Emilio Cobos Álvarez (:emilio)]

Attachment: Quick test page to test the rendering.

Comment 4

[Emilio Cobos Álvarez (:emilio)]

Attachment: Screenshot of rendered result.

We might want to use rounded corners etc by default, perhaps something else than rgba(0, 0, 0, .2) for text / images, but this looks close enough imo :-)

Comment 5

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/feea84165bf0
Add a document.getWireframe() API that gets some relevant rects in the viewport. r=mconley

Comment 6

[Brian Grinstead [:bgrins]]

Attachment: WIP: Bug 1731714 - WIP (for reference) marionette script to generate comparison screenshots and wireframes for a number of popular sites

Just for reference - don't intend to land this

Comment 7

[Norisz Fay [:noriszfay]]

https://hg.mozilla.org/mozilla-central/rev/feea84165bf0