749824 - Cannot "view image" or "save image as" on webgl canvas, result is blank

Status: NEW

(Core :: Graphics: CanvasWebGL, defect, P3)

Description

[Alon Zakai (:azakai)]

Attachment: unsaveable webgl canvas

On the attachment, rightclick->view image or save image as gives blank images instead of the required content. Happens in both nightly and release.

Being able to save the output has been quite useful for me in the past (when generating webgl testcases for emscripten opengl bindings), so this is saddening.

Comment 1

[(no longer active)]

I can reproduce on Mac.

Comment 2

[JannikV]

Just noticed the same or a similar issue.

Rightclick on a webgl canvas. "View Image" or "Save Image As..." and it produces a grey bitmap instead of the proper content.

It worked well in Firefox 21 but this bug rises in Firefox 22 and isn't fixed in Aurora yet.

Testet with Fedora (Linux).

Comment 3

[Milan Sreckovic [:milan] (needinfo for best results)]

Guillaume, can you take a look?

Comment 4

[Guillaume Abadie]

I build the lastest mozilla central, and it seams to works fine on Linux or Mac. But the thing strange is when I'm saving, the image's aspect ratio is not the same as the canvas' one... Is it normal?

Would you have precise steps to reproduce the bug?

Comment 5

[Alon Zakai (:azakai)]

STR in the first comment still show the problem for me. I am on nightly on linux.

Comment 6

[JannikV]

I'm currently on Aurora "24.0a2 (2013-08-05)".

* Open the attached "unsaveable webgl canvas"
* right click and choose "View Image" or "Save Image as..."

View image produces the grey bitmap, Save Image results in a fully transparent file (but the aspect ratio is fine). I will attach a screenshot and the saved image...

Comment 7

[JannikV]

Attachment: view-image.png

result of "View Image"

Comment 8

[JannikV]

Attachment: save-image.png

result of "Save Image As..."

Comment 9

[JannikV]

Just viewed my own attachments in the browser and it seems the grey and the transparent images I mentioned above are the same. Sorry for confusion..

Comment 10

[Guillaume Abadie]

Never mind, I did one thing wrong...

So I can save all WebGL canvas I want on the web, but not yours. And the thing is I can't find any window.requestAnimationFrame in your code.

Comment 11

[JannikV]

[ Ok, just switched over to 25.0a2 (2013-08-07) ]

Strange.. Alon Zakais example is neither viewable nor saveable for me.

But I tested the following two examples in a tutorial.
This ( http://learningwebgl.com/lessons/lesson02/index.html ) static canvas isn't viewable or saveable.
This ( http://learningwebgl.com/lessons/lesson03/index.html ) animated canvas is both and works as expected!

Comment 12

[Guillaume Abadie]

(In reply to JannikV from comment #11)
> [ Ok, just switched over to 25.0a2 (2013-08-07) ]
> 
> Strange.. Alon Zakais example is neither viewable nor saveable for me.
> 
> But I tested the following two examples in a tutorial.
> This ( http://learningwebgl.com/lessons/lesson02/index.html ) static canvas
> isn't viewable or saveable.
It works for me, but it might not work because WebGL call are not done in window.requestAnimationFrame. 
> This ( http://learningwebgl.com/lessons/lesson03/index.html ) animated
> canvas is both and works as expected!
For me too, and uses window.requestAnimationFrame.

Comment 13

[Guillaume Abadie]

I searched and found the problem of this bug:

When we do a right click -> "Save image as", the bug is we are reading the OpenGL context's back buffer instead of the currently shown buffer. Actually, we are using a triple buffer for WebGL. Therefore, when you draw only one frame like done in http://learningwebgl.com/lessons/lesson02/index.html, The two others are still blank.

But officially, from the point of view of the WebGL specification, this is not a bug, but a undefined behavior (Thanks jgilbert!).

https://www.khronos.org/registry/webgl/specs/1.0/ chapter 2.2:
"This default behavior can be changed by setting the preserveDrawingBuffer attribute of the WebGLContextAttributes object. If this flag is true, the contents of the drawing buffer shall be preserved until the author either clears or overwrites them. If this flag is false, attempting to perform operations using this context as a source image after the rendering function has returned can lead to undefined behavior. This includes readPixels or toDataURL calls, or using this context as the source image of another context's texImage2D or drawImage call."

A solution would be to read the shown buffer instead, but we would have to change to many abstract classes in our implementation, like mozilla::gfx::SurfaceStream, and directly hit the buffers synchronization mechanism.

Comment 14

[JannikV]

Ok, I understand, thank you for your investigation.

I can confirm, that it works if you set preserveDrawingBuffer.
-----
var glContextAttributes = { preserveDrawingBuffer: true };
var gl = canvas.getContext("experimental-webgl", glContextAttributes);
-----

That means, the success of the view and save operations depends on the website programmers choice of setting the option.
I'm not sure if this is a good way to handle the "undefined behavior".
It's up to you to decide what Firefox should do (initially I was confused because the behavior changed between Firefox 21 and 22..).

Comment 15

[Alon Zakai (:azakai)]

preserveDrawingBuffer is used in the testcase (in comment #0), but it still fails.

Comment 16

[Benoit Jacob [:bjacob] (mostly away)]

I am not sure that the above interpretation of the spec is correct, because just above the above-quoted excerpt from Section 2.2, there is this:

> WebGL presents its drawing buffer to the HTML page compositor immediately before a compositing operation, *** BUT ONLY IF *** clear, drawArrays or drawElements have been called since the last compositing operation, while the drawing buffer is the currently bound framebuffer.

(emphasis mine)

IIRC that is a recent update to this part of the spec, intended to remove exactly this kind of undefined-behavior cases.

Given this, do you still think that this is undefined behavior and that we are bug-less here? Either way please keep the bug assigned to someone until resolved.

Comment 17

[Kelsey Gilbert [:jgilbert]]

(In reply to Benoit Jacob [:bjacob] from comment #16)
> I am not sure that the above interpretation of the spec is correct, because
> just above the above-quoted excerpt from Section 2.2, there is this:
> 
> > WebGL presents its drawing buffer to the HTML page compositor immediately before a compositing operation, *** BUT ONLY IF *** clear, drawArrays or drawElements have been called since the last compositing operation, while the drawing buffer is the currently bound framebuffer.
> 
> (emphasis mine)
> 
> IIRC that is a recent update to this part of the spec, intended to remove
> exactly this kind of undefined-behavior cases.
> 
> Given this, do you still think that this is undefined behavior and that we
> are bug-less here? Either way please keep the bug assigned to someone until
> resolved.

This doesn't change anything, but does formalize when we should be presenting to the compositor. The same undef-behav remains.

Comment 18

[Guillaume Abadie]

I tried for last 2 days to find what is wrong. But sadly, I don't have the answer.

 1) Any clear done from the WebGL API
 2) The copy done in SurfaceStream_TripleBuffer_Copy works perfectly (understandable, jgilbert is the author... ;) )
 3) Any interesting clear done after calling SurfaceStream_TripleBuffer_Copy::SwapProducer's call
 3) Can't track all Clear, to many are done by the composer.
 4) Try a break point in raw_fClear if we are clearing a framebuffer that have a texture bound on the COLOR_ATTACHMENT0 having same size.

I'm running out of ideas to find the problem.

Comment 19

[Guillaume Abadie]

Attachment: debug_code.diff

Here is the code I did for tracking operation done on buffers.

Comment 20

[Guillaume Abadie]

Attachment: lldb_log.log

Here is the log of the execution of the tracking code.

Sorry to not be able to go further.

Comment 21

[Emily Bowman]

Maybe this is graphics card+driver dependent, as so much is?

This bug affects me with Intel HD 4000 & HD 4600, latest drivers (10.18.10.3652, from 6/16/14).

Comment 22

[Kelsey Gilbert [:jgilbert]]

(In reply to Joshua Bowman from comment #21)
> Maybe this is graphics card+driver dependent, as so much is?
> 
> This bug affects me with Intel HD 4000 & HD 4600, latest drivers
> (10.18.10.3652, from 6/16/14).

It shouldn't be, for once. (though it's always possible there's a secondary driver bug affecting someone here)

preserveDrawingBuffer:false canvases don't really have a guarantee that they'll be retrievable in this way. We should be able to do it with some work, but I can't guarantee it.

preserveDrawingBuffer:true canvases should always work for this.

Comment 23

[Milan Sreckovic [:milan] (needinfo for best results)]

Walter, this is a bit of a puzzler, but maybe a fresh pair of eyes can help.

Comment 24

[Kelsey Gilbert [:jgilbert]]

Attachment: testcase

Here's a simpler testcase. You can see from the source that `preserveDrawingBuffer` is `true`. You should be able to 'view image' this canvas and see a snapshot of the canvas.

This works on Linux, but does not appear to work on Windows. I haven't tested Mac.

Comment 25

[Kelsey Gilbert [:jgilbert]]

This should be easy to add a mochitest for, as well.

Comment 26

[Walter Litwinczyk [:walter]]

As a note Jeff's test case works in FF30/Nightly for OSX + Linux and works in FF30 on Windows, but not in Nightly.

Comment 27

[Walter Litwinczyk [:walter]]

Assuming I didn't mis-perform the bisect, it says that this is the first commit with the gray image on windows:

https://hg.mozilla.org/mozilla-central/rev/68dfeae0d1be

Comment 28

[Milan Sreckovic [:milan] (needinfo for best results)]

Sounds like at least the simple test case was regressed by bug 1000640, though the original bug predates that fix.

Comment 29

[Walter Litwinczyk [:walter]]

I'm running into difficulties when trying to run older versions of FF. At FF-23 I can't get it to initialize WebGL on Windows 8, so I'll see if I can get a different version of Windows to test with. But it's at least still present in FF-24. (Although above they mention FF21 and FF22, which is what I was aiming to get down to).

Comment 30

[Kelsey Gilbert [:jgilbert]]

This might be related to bug 1045955.

Comment 31

[Ciprian Georgiu, Desktop QA]

I found the same issues by playing Webgl games. Here are my information:

[Affected versions]:
- latest Nightly 51.0a1 (2016-09-07)
- latest Aurora 50.0a2 (2016-09-07)
- Firefox Beta 49.0b10
- Firefox Release 48.0.2

[Affected platforms]:
- Win 10 Pro, x64
- Ubuntu 16.04 LTS, x64
- Mac OS Sierra 10.12 Beta 

[Steps to reproduce]:
1. Launch Firefox 
2. Go to this WebGL game: http://blackberry.github.io/WebGL-Samples/tunneltilt/
3. Play the game
4. Right click on the game's window for displaying the context menu
5. Select “view image”

[Expected result]:
- You are redirected to a new page where the image is displayed.

[Actual result]:
- A new page displaying a blank image is shown

[Additional notes]:
- The URL while "view image" is selected looks like this: blob:null/de3c740f-29ff-4a8d-825b-75ede0f91295
- Screenshot attached http://imgur.com/a/2rBi5

Comment 32

[Milan Sreckovic [:milan] (needinfo for best results)]

Peter, this could be a interesting bug for somebody, see comment 31 for STR.

Comment 33

[Kelsey Gilbert [:jgilbert]]

Comment 13 is important to understand. This is not a bug according to the spec.

For View Image and Save Image, we should copy the frontbuffer, instead of the backbuffer as we do now. IIRC View/Save Image use the same mechanism as toDataURL, which *must* in many cases copy from the backbuffer.

What we want to do is copy from the backbuffer, unless the backbuffer has not been dirtied by draw calls yet. If the backbuffer is clean of draw calls, we should copy from the frontbuffer.