1021547 - Screenshot size should be hardware size, not CSS size

Status: RESOLVED WONTFIX

(DevTools Graveyard :: WebIDE, defect, P3)

Description

[Jukka Jylänki]

When using the Screenshot button in App Manager to capture a screenshot of the device, the pixel dimensions of the captured screenshot are not 1:1 correct with the physical display size of the device.

Testing on the Flame device, it has dimensions of 480x854, but when capturing a screenshot, it is downscaled and only instead has dimensions 320x569.

Comment 1

[Paul Rouget [:paul]]

It's the CSS size of the window. One CSS pixel == one pixel in the image. We don't use hardware pixels.

Comment 2

[Jukka Jylänki]

When you have an image on the local harddrive outside the browser, that image of course doesn't relate in any way with CSS pixels. The device renders to a display resolution of 480x854, so it shouldn't be pulling in downscaled screenshots, this looks like a bug.

Comment 3

[Paul Rouget [:paul]]

I don't know how that would work with sub pixel rendering.

Nical, maybe you can help me:

We want to take a screenshot of the Flame. We use the canvas.drawWindow method. Basically:

    let canvas = window.document.createElementNS("http://www.w3.org/1999/xhtml", "canvas");
    let width = window.innerWidth;
    let height = window.innerHeight;
    canvas.setAttribute('width', width);
    canvas.setAttribute('height', height);
    let context = canvas.getContext('2d');
    let flags =
          context.DRAWWINDOW_DRAW_CARET |
          context.DRAWWINDOW_DRAW_VIEW |
          context.DRAWWINDOW_USE_WIDGET_LAYERS;
    context.drawWindow(window, 0, 0, width, height, 'rgb(255,255,255)', flags);

We could use window.devicePixelRatio to know the correct size of the canvas, but then I don't know how to correctly draw the window into the canvas. Do we need to use canvas.scale(ratio)? And then, would subpixels would be correctly rendered? For example, on a 2x device, would it scale the odd pixels or use the extra sub pixel?

Comment 4

[Jukka Jylänki]

To reproduce this from the command line:

1. Connect the FFOS device via USB. Make sure ADB and Devtools Developer options are enabled
2. Type

wget https://raw.githubusercontent.com/kripken/emscripten/incoming/tools/ffdb.py
chmod +x ffdb.py
./ffdb.py screenshot screen.png
open screen.png

on the Flame, this will print

Wrote 166.9KB to file 'screen.png' (320x569 pixels).

but expected that it would print

Wrote xxxKB to file 'screen.png' (480x854 pixels).

Comment 5

[Markus Stange [:mstange]]

(In reply to Paul Rouget [:paul] (slow to respond. Ping me on IRC) from comment #3)
> We could use window.devicePixelRatio to know the correct size of the canvas,
> but then I don't know how to correctly draw the window into the canvas. Do
> we need to use canvas.scale(ratio)?

Exactly. This should work correctly, and it's what we use for the history swiping snapshots (disabled by default) on Firefox Mac: http://dxr.mozilla.org/mozilla-central/source/browser/base/content/browser-gestureSupport.js#973
(Ignore the zoom part, if you didn't need that before then you won't need it now.)

Comment 6

[Jukka Jylänki]

Btw, regarding window.devicePixelRatio, I find it brittle and fishy to be used anywhere. In my view, the whole concept of using window.devicePixelRatio to deduce device pixel sizes is a bit broken. I wrote up https://github.com/KhronosGroup/WebGL/issues/587 to talk about that, since I face that issue with WebGL specifically, although it may be a more generic canvas/DOM issue. If we have skilled DOM specification people who would know a way around this issue, I'd love to hear.

Comment 7

[Markus Stange [:mstange]]

Concerning the example you gave over there, your whole problem would be solved the dips size weren't 320x533, but 320x533.333333, right? I can't test this at the moment because I don't have a device that reports such numbers, but my guess is that using document.documentElement.getBoundingClientRect().width/.height will give you those fractional numbers, whereas document.documentElement.clientWidth/Height (what quirksmode used) is an API that rounds and that loses the information you need.
In summary, I don't think the problem you think you're having is an actual problem.

Comment 8

[Jukka Jylänki]

Markus: what if I write a page with a canvas element <canvas style="width:320px; height:533px;"></canvas>? What should I feed to canvas.width and canvas.height to make the rendering pixel-perfect?

I think the answer is that if devicePixelRatio is fractional, one cannot know. We are now having a discussion at https://www.khronos.org/webgl/public-mailing-list/archives/1406/ , and people seem to generally agree.

But does that mean that for the screenshot snippet that Paul wrote above, instead of 

let width = window.innerWidth;
let height = window.innerHeight;

the code should read

let width = (document.documentElement.getBoundingClientRect().width * window.devicePixelRatio)|0;
let height = (document.documentElement.getBoundingClientRect().height * window.devicePixelRatio)|0;

(the |0 to explicitly say 'truncate') and it would be guaranteed to match the device size?

Comment 9

[Markus Stange [:mstange]]

(In reply to Jukka Jylänki from comment #8)
> Markus: what if I write a page with a canvas element <canvas
> style="width:320px; height:533px;"></canvas>? What should I feed to
> canvas.width and canvas.height to make the rendering pixel-perfect?

You can't know because the pixel-snapped size of the canvas depends on the fractional part of its position. (At least that's what happens for regular painting; maybe canvas is special-cased, I haven't checked.) But anyway, if there's already a discussion going on, let's not repeat it here.

> But does that mean that for the screenshot snippet that Paul wrote above,
> instead of 
> 
> let width = window.innerWidth;
> let height = window.innerHeight;
> 
> the code should read
> 
> let width = (document.documentElement.getBoundingClientRect().width *
> window.devicePixelRatio)|0;
> let height = (document.documentElement.getBoundingClientRect().height *
> window.devicePixelRatio)|0;
> 
> (the |0 to explicitly say 'truncate') and it would be guaranteed to match
> the device size?

Hmm, I think the problem in this case is that document.documentElement.getBoundingClientRect() gives you the whole scrollable size of the page and is not clipped to the scroll port. I don't know if there's a non-rounding API that gives you the scroll port size from the inside of the page.

Comment 10

[Julian Descottes [:jdescottes]]

WebIDE triage. Filter on TRIAGE-JD201705

For the record, taking screenshots with WebIDE seems completely broken on Nightly 

> Operation failed: taking screenshot: TypeError: type.frontClass is not a constructor

Comment 11

[Julian Descottes [:jdescottes]]

WebIDE was completely removed in Bug 1539462 (Firefox 71). Closing all the bugs remaining in the component.