1278507 - chrome.tabs.capturevisibletab doesn't return dot by dot image

Status: RESOLVED DUPLICATE of bug 1636508

(WebExtensions :: Frontend, defect, P5)

Description

[pasta0915]

User Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_10_5) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/50.0.2661.102 Safari/537.36

Steps to reproduce:

You write code below on WebExtension

chrome.tabs.captureVisibleTab(null, {format: 'png'}, function (data) {
  const img = new Image
  img.onload = function () {console.log(img.naturalWidth, img.naturalHeight)}
  img.src = data
})

then exec extension on HiDPI Display ( such as Retina Display )


Actual results:

My environment is Retina Display (window.devicePixcelRation is 2, I call it `scale` on below)

onChrome, img.naturalWidth/Height equal window.innerWidth/Height * scale
onFirefox, img.naturalWidth/Height equal window.innerWidth/Height

So capture image on Firefox is blured.


Expected results:

I think Firefox should return image same as Chrome.

Comment 1

[Hector Zhao [:hectorz]]

Not sure, but this may be not a WebExtensions API issue, but something to be fixed in like `canvas.drawWindow`?

One of the common feedback to our screenshot extension before the WebExtensions API conversion is the low resolution of screenshot on retina display.

Comment 2

[Hector Zhao [:hectorz]]

(In reply to Hector Zhao [:hectorz] from comment #1)
> Not sure, but this may be not a WebExtensions API issue, but something to be
> fixed in like `canvas.drawWindow`?

Hello Nicolas, would you be able to help deciding whether this is a canvas.drawWindow issue? Thanks!

Comment 3

[Nicolas Silva [:nical]]

(In reply to Hector Zhao [:hectorz] from comment #2)
> Hello Nicolas, would you be able to help deciding whether this is a
> canvas.drawWindow issue? Thanks!

If I understand correctly the WebExtension api to capture the content of the window internally uses a canvas that is scaled down on hidpi displays, and that naturally causes the page to look blurry because the capture happens at half resolution. If so, then I think it would make sense that the WebExtension makes sure the canvas it uses internally is not scaled. This falls outside the realm of gfx so I don't think I can help much outside of that. I googled around and it looks like webkit has a non-standard property backingStorePixelRatio to compensate the hidpi scaling factor, while other vendors like us don't. What seems to be possible is to create a canvas that is bigger (effectively compensating the scale factor that is applied to the canvas storage size) and use CSS to display it at the proper size on the page.

I think that canvas.drawWindow does what it is supposed to do. However the canvas itself should be created with a higher resolution when the css pixel scaling factor is not equal to 1, which I think is something the WebExtension implementation should manage. (but again that's a bit far away from what  work on, so I might not be the best person to answer this question).

Comment 4

[Hector Zhao [:hectorz]]

(In reply to Nicolas Silva [:nical] from comment #3)

Thanks for your response!

Comment 5

[vishaalkal]

I am part of the team working on the OneNote Web Clipper (https://addons.mozilla.org/en-US/firefox/addon/onenote-clipper/?src=ss) and we are also running into this resolution issue. 

Here is a link explaining the backing store that Nicolas mentioned: https://www.html5rocks.com/en/tutorials/canvas/hidpi/

Here is a related work-around for Chrome: http://outof.me/chrome-extension-retina-capturevisibletab-translate3d-2-x-res/

Does scaling a canvas prior to drawing into it increase the resolution? 

I can confirm that the image returned by captureVisibleTab on Chrome and Firefox has the same dataUrl. However, using context.drawImage(...) with this as the source image returns the different results. If needed, I can send code that produces a clean repro.

Comment 6

[Ian Bicking (:ianbicking)]

drawWindow can be used for a higher-resolution image; here's an example: https://github.com/mozilla-services/pageshot/blob/b2c3b5a0e86ab1b748f3f78145e1fdcddf09b247/webextension/selector/shooter.js#L17-L37

(Still would be nice if Firefox worked the same as Chrome here with respect to resolutions.)

Comment 7

[Serg]

(In reply to Ian Bicking (:ianb) from comment #6)
> drawWindow can be used for a higher-resolution image; here's an example:
> https://github.com/mozilla-services/pageshot/blob/
> b2c3b5a0e86ab1b748f3f78145e1fdcddf09b247/webextension/selector/shooter.
> js#L17-L37
> 
> (Still would be nice if Firefox worked the same as Chrome here with respect
> to resolutions.)

As I understand this solution is limited to Add-on SDK and will not be available in WebExtensions. Is it correct?
I see that the status is "Unconfirmed", what do we need to move this bug to Assigned state?

Comment 8

[Andy McKay]

Some one would need to volunteer to work on the bug, its a P5 and not on the radar of anyone working at Mozilla: https://wiki.mozilla.org/WebExtensions/RoadMapFirefox57. Because its flagged with [design-decision-needed] it should also go through that process: https://wiki.mozilla.org/Add-ons/Contribute/Triage.

Comment 9

[Caitlin Neiman [:caitmuenster]]

Hi all, this has been added to the May 23 WebExtenion APIs triage agenda. If you are able to attend, please let me know -- we would love to have you join us. 

Wiki: https://wiki.mozilla.org/Add-ons/Contribute/Triage#Details_.26_How_to_Join

Agenda: https://docs.google.com/document/d/1-j08Zo4sbwAuRZndNNtdIRlDM8TblddDA-PAeRgYMWU/edit#

Comment 10

[Andy McKay]

Not much to say here, other than it went through the meeting and no-one had any complaints.

Comment 11

[Tomislav Jovanovic :zombie]

This was fixed in bug 1636508, see https://bugzilla.mozilla.org/show_bug.cgi?id=1636508#c23