818810 - WebGL spec w.r.t. texImage2D y-orientation for HTML element uploads is underdefined, and inconsistent as implemented

Status: RESOLVED INVALID

(Core :: Graphics: CanvasWebGL, defect)

Description

[Kelsey Gilbert [:jgilbert]]

This is interesting, but might be irreparable at this point.

The following two lines yield different results:
[1] gl.copyTexImage2D(gl.TEXTURE_2D, 0, gl.RGBA, 0, 0, gl.drawingBufferWidth, gl.drawingBufferHeight, 0);
[2] gl.texImage2D(gl.TEXTURE_2D, 0, gl.RGBA, gl.RGBA, gl.UNSIGNED_BYTE, gl.canvas);

[1] matches this: [3]
var data = new Uint8Array(gl.drawingBufferWidth * gl.drawingBufferHeight * 4);
gl.readPixels(0, 0, gl.drawingBufferWidth, gl.drawingBufferHeight, gl.RGBA, gl.UNSIGNED_BYTE, data);
gl.texImage2D(gl.TEXTURE_2D, 0, gl.RGBA, gl.drawingBufferWidth, gl.drawingBufferHeight, 0, gl.RGBA, gl.UNSIGNED_BYTE, data);

UNPACK_FLIP_Y_WEBGL=true makes [2] the same as [1], but now [3] is upside-down.


Currently, what we do is convert and upload HTML elements top-row-first into texImage2D, which makes them upside-down according to OpenGL's 0,0=bottom-left coordinates. 
Orientation doesn't appear to be specified in the spec for texImage2D on HTML elements, but this is what everyone has implemented.

To fix this (that is, make HTML element texture uploads match OpenGL's coord system), we can flip UNPACK_FLIP_Y_WEBGL from (the default) false to true. However, this means that subsequent uploads of TypedArray data are 0,0=top-left under OpenGL coordinates.

The only way I can think of being able to handle both these cases sensibly (assuming we can't break content) is to add another UNPACK_ boolean that specifies only for HTML element uploads. At least with this new pixelStore option, we'd have the option to be able to configure ourselves into a consistent system.

Comment 1

[Kelsey Gilbert [:jgilbert]]

Attachment: testcase showing inconsistent behavior of texImage

Comment 2

[Kelsey Gilbert [:jgilbert]]

I should note that what the script does. Starting from the state shown by "Source:", each case uses a different method to copy the framebuffer into a texture. Then, we draw a blue triangle and add the texture copy of the framebuffer for each method. (copyTexImage, texImage(gl.canvas), texImage(readPixels)) We draw each case with UNPACK_FLIP_Y_WEBGL as false and true, to show the differences.

Note how our texture copy of the backbuffer in texImage2D(canvas) is upside-down compared to what we would expect from the other two. We can fix this case by using y-flip, but we see that this breaks texImage2D(readPixels), since we're now flipping texImage2D(TypedArray) uploads.

Comment 3

[Kelsey Gilbert [:jgilbert]]

If it I wasn't clear enough:
texImage appears inconsistent when used with HTML elements, though every browser appears to implement this the same way.
That is, behavior is consistent between browsers, but the orientation of texImage+HTML element uploads appears internally inconsistent with coordinate system used by the rest of WebGL.

Comment 4

[Kelsey Gilbert [:jgilbert]]

Attachment: testcase showing readPixels orientation

Comment 5

[Kelsey Gilbert [:jgilbert]]

This was resolved in the WebGL spec:
UNPACK_FLIP_Y_WEBGL of type boolean
    If set, then during any subsequent calls to texImage2D or texSubImage2D, the source data is flipped along the vertical axis, so that conceptually the last row is the first one transferred.

The first row for HTML elements is the top row, whereas for WebGL constructs, it's the bottom row.