WebGL spec w.r.t. texImage2D y-orientation for HTML element uploads is underdefined, and inconsistent as implemented

RESOLVED INVALID

Status

()

defect
--
minor
RESOLVED INVALID
7 years ago
6 years ago

People

(Reporter: jgilbert, Unassigned)

Tracking

Firefox Tracking Flags

(Not tracked)

Details

Attachments

(2 attachments)

This is interesting, but might be irreparable at this point.

The following two lines yield different results:
[1] gl.copyTexImage2D(gl.TEXTURE_2D, 0, gl.RGBA, 0, 0, gl.drawingBufferWidth, gl.drawingBufferHeight, 0);
[2] gl.texImage2D(gl.TEXTURE_2D, 0, gl.RGBA, gl.RGBA, gl.UNSIGNED_BYTE, gl.canvas);

[1] matches this: [3]
var data = new Uint8Array(gl.drawingBufferWidth * gl.drawingBufferHeight * 4);
gl.readPixels(0, 0, gl.drawingBufferWidth, gl.drawingBufferHeight, gl.RGBA, gl.UNSIGNED_BYTE, data);
gl.texImage2D(gl.TEXTURE_2D, 0, gl.RGBA, gl.drawingBufferWidth, gl.drawingBufferHeight, 0, gl.RGBA, gl.UNSIGNED_BYTE, data);

UNPACK_FLIP_Y_WEBGL=true makes [2] the same as [1], but now [3] is upside-down.


Currently, what we do is convert and upload HTML elements top-row-first into texImage2D, which makes them upside-down according to OpenGL's 0,0=bottom-left coordinates. 
Orientation doesn't appear to be specified in the spec for texImage2D on HTML elements, but this is what everyone has implemented.

To fix this (that is, make HTML element texture uploads match OpenGL's coord system), we can flip UNPACK_FLIP_Y_WEBGL from (the default) false to true. However, this means that subsequent uploads of TypedArray data are 0,0=top-left under OpenGL coordinates.

The only way I can think of being able to handle both these cases sensibly (assuming we can't break content) is to add another UNPACK_ boolean that specifies only for HTML element uploads. At least with this new pixelStore option, we'd have the option to be able to configure ourselves into a consistent system.
Severity: enhancement → minor
Attachment #689110 - Attachment description: testcase showing inconsisent behavior of texImage → testcase showing inconsistent behavior of texImage
I should note that what the script does. Starting from the state shown by "Source:", each case uses a different method to copy the framebuffer into a texture. Then, we draw a blue triangle and add the texture copy of the framebuffer for each method. (copyTexImage, texImage(gl.canvas), texImage(readPixels)) We draw each case with UNPACK_FLIP_Y_WEBGL as false and true, to show the differences.

Note how our texture copy of the backbuffer in texImage2D(canvas) is upside-down compared to what we would expect from the other two. We can fix this case by using y-flip, but we see that this breaks texImage2D(readPixels), since we're now flipping texImage2D(TypedArray) uploads.
If it I wasn't clear enough:
texImage appears inconsistent when used with HTML elements, though every browser appears to implement this the same way.
That is, behavior is consistent between browsers, but the orientation of texImage+HTML element uploads appears internally inconsistent with coordinate system used by the rest of WebGL.
This was resolved in the WebGL spec:
UNPACK_FLIP_Y_WEBGL of type boolean
    If set, then during any subsequent calls to texImage2D or texSubImage2D, the source data is flipped along the vertical axis, so that conceptually the last row is the first one transferred.

The first row for HTML elements is the top row, whereas for WebGL constructs, it's the bottom row.
Status: NEW → RESOLVED
Closed: 6 years ago
Resolution: --- → INVALID
You need to log in before you can comment on or make changes to this bug.