1306380 - WebGL2 TransformFeedback: shader link error on Windows, not on OSX

Status: NEW

(Core :: Graphics: CanvasWebGL, defect, P3)

Description

[Andre Weissflog]

User Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_12_1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/55.0.2873.4 Safari/537.36

Steps to reproduce:

Apologies for just copy-pasting the description from the following Chromium ticket (https://bugs.chromium.org/p/chromium/issues/detail?id=651493), but the bug is completely identical between Chrome and FF (I guess it's an ANGLE issue):

--- here it goes:

The following WebGL2 transform-feedback demo works as expected on OSX in Canary (and FF Nightly), but produces a shader linker error on Windows (also both in Canary and FF) (don't forget to enable WebGL2 in Canary, if every thing works as expected the gray point cube should turn into a colorful whirling point cloud when moving the mouse over it):

http://floooh.github.io/oryol-sticky-tests/VertexCapture.html

Firefox provides a useful error message in the JS console:

"Transform feedback varying webgl_**** does not exist in the vertex shader"

The natively compiled demo also works as expected on desktop GL implementations (although I haven't tested yet on GLES3 platforms).

Sources for vertex and fragment shader, and the transform-feedback varyings are pasted below. My guess is that ANGLE detects that the captured vertex shader outputs are not used in the fragment shader code, and "optimizes them away" in the fragment shader input and all the way to the vertex shader outputs, so that the transform-feedback-varying is not found in the vertex shader output when the shader is linked.

Here's the vertex shader source (output of my shader source code generator, thus a bit messy-looking). Note that the vertex shader has 2 outputs, 'pos' and 'vel':

#version 300 es
#define _position gl_Position
#define mul(m,v) (m*v)
layout (std140) uniform captureParams {
  vec3 centerOfGravity;
  float _pad_centerOfGravity;
};
in vec3 position;
in vec3 normal;
out vec3 pos;
out vec3 vel;
void main() {
  vec3 dir = centerOfGravity - position;
  float dist = (length(dir) + 0.0001);
  float i = min(1.0 / (dist * dist), 20.0);
  vel = 0.995 * (normal + normalize(dir) * i);
  pos = position + vel * 0.0001;
  _position = vec4(0.0);
}

And here's the fragment shader, since I'm only interested in the transform feedback result, it's basically just a discard, but it still defines two inputs (pos and vel). I did this primarily because my shader-code-generator enforces that the vertex shader output has to match the fragment shader input (this is also necessary on other 3D-APIs):

#version 300 es
precision mediump float;
#define _color _FragColor
in vec3 pos;
in vec3 vel;
out vec4 _FragColor;
void main() {
  discard;
  _color = vec4(0.0);
}

Before glLinkProgram(), glTransformFeedbackVaryings() is called with the two strings "pos" and "vel".

Next thing I'll try is to use the "pos" and "vel" inputs in the fragment shader, hoping that ANGLE won't drop the vertex shader outputs since they are used in the fragment shader, but even if this works I think that Windows and OSX should behave the same (either both should work, or both should fail).

Oh, and another interesting thing:

On an older Firefox Nightly, the demo actually worked at first, only after I updated to the very latest Nightly it broke. I can't say how old the previous version was, but I think it wasn't older than a few weeks, so I guess the change in ANGLE must be quite recent (or at least Firefox must have updated to a newer ANGLE quite recently).

Comment 1

[Andre Weissflog]

Changing the fragment shader so that the pos and vel variables are used fixes the demo in Firefox Nightly on Windows, but I think this is a hack that shouldn't be necessary. See here for a 'worked-around' version: https://floooh.github.io/oryol-webgl2/asmjs/VertexCapture.html

The fragment shader now looks like this:

#version 300 es
precision mediump float;
#define _color _FragColor
in vec3 pos;
in vec3 vel;
out vec4 _FragColor;
void main() {
  discard;
  _color = vec4(pos+vel, 1.0);
}

At least this confirms my suspicion that ANGLE seems to 'optimize away' vertex shader outputs if they are not used used in the fragment shader, but should be captured by transform-feedback.

Comment 2

[Kelsey Gilbert [:jgilbert]]

I'm not seeing an error on Windows, but it doesn't seem to work on ANGLE. (either in Firefox or Chrome Canary)
Works on native Windows OpenGL, though.