589268 - shader behave strange when webgl.shader_validator is true

Status: RESOLVED WORKSFORME

(Core :: Graphics: CanvasWebGL, defect)

Description

[Sebastian]

User-Agent:       Mozilla/5.0 (Windows NT 6.1; rv:2.0b5pre) Gecko/20100820 Minefield/4.0b5pre
Build Identifier: Mozilla/5.0 (Windows NT 6.1; rv:2.0b5pre) Gecko/20100820 Minefield/4.0b5pre

in a pixelshader that "just" does per-pixel specular lighting, some vectors aren't summed up correctly, when webgl.shader_validator is enabled. Disabling produces normal (IMHO) behaviour.
Shader Code:
--------8<----------------
precision highp float;

// Standard Varyings
varying vec4 vPosition;
varying vec3 vNormal;
varying vec2 vTexCoord; 
uniform vec3 lPosition;
uniform vec3 ePosition;
uniform float spotSize;
uniform float diffuse;

void main()
{
   vec3 lV = normalize(lPosition-(vPosition.xyz));
   vec3 eP = normalize(-vPosition.xyz);
   vec3 tmp1 = reflect(-lV,vNormal); 
   vec3 R = normalize(tmp1);
   vec3 D = vec3(diffuse,diffuse,diffuse)*max(dot(vNormal,lV),0.0);
   vec3 S = vec3(1,1,1)*pow(max(dot(R,eP),0.0),spotSize);
   vec3 A = vec3(0.1, 0.1, 0.1);
   gl_FragColor=vec4(A+D+S,1.0);
}
------8<--------
when webgl.shader_validator is enabled, values < 1.0 for the "A"-Term produce black Pixels. (e.g. setting vec3 A = vec3(0.99, 0.99, 0.99)), all pixels remain black, if I set A to vec3 (1.0, 1.0, 1.0), all pixels get white (as expected).
Disabling webgl.shader_validator (=false) and reloading the page, everything is as expected (pixels in dark gray). FYI: the vertex shader just multiplies the vertex position with the MV-Matrix (mat4).
Concerning the specs, the behaviour that shader_validator=false produces should be the correct one (again, IMO)

Reproducible: Always

Steps to Reproduce:
I draw a sphere, do have a standard vertex shader (gl_Position = projMatrix*mvMatrix*PositionAttribute etc) and using this (standard) Pixel Shader shows the result.
Actual Results:  
Pixels that should get grey ( by having a ambient term of vec3(0.1,0.1,0.1)) get black, even when the elements in the vec3 are set to 0.99 (not expected). Setting them to 1.0 gives a white pixel (expected). 

Expected Results:  
pixels in dark grey. 

as said, disabling webgl.shader_validator produces expected results

Comment 1

[Benoit Jacob [:bjacob] (mostly away)]

I'm pretty sure that this has been fixed by bug 592101.

Can you retry now?

Comment 2

[Sebastian]

still wrong like I described.

Comment 3

[Benoit Jacob [:bjacob] (mostly away)]

Actually, I now realize that this is probably the same bug as bug 594489.

Can you try with US American locale (i.e. a locale that use '.' as the decimal point instead of ',') ?

That would be a ANGLE bug, upstreamed, with a patch.

Comment 4

[Sebastian]

I'm already using the latest Minefield with the locale set to en-US :-S

Comment 5

[Benoit Jacob [:bjacob] (mostly away)]

There might however be multiple notions of 'locale' here. I'm talking about the one that the C standard library on your platform picks up. See:

http://publib.boulder.ibm.com/infocenter/tivihelp/v2r1/index.jsp?topic=/com.ibm.itame.doc_6.0/am60_audit29.htm

This page suggests that this is the same as the locale set in the Windows GUI ("the language setting in the Regional Settings of the Control Panel"). What locale are you using there?

To avoid changing your global Windows settings, can you try:

   LANG=C firefox

or

   LC_ALL=C firefox

in the command line.

Comment 6

[wdumon]

I ran into this too: shaders behaving wrong when the shader validator is enabled. In my case, a 'varying' wasn't varying at all, causing wrong (constant) lighting on a complex textured surface.

When the shader validator is disabled, things work as expected (and similar as chrome).

Observed in Mozila FF 4.0b7

Fragment shader:

#ifdef GL_ES
precision highp float;
#endif

varying vec2 vTextureCoord;
varying vec3 vLightWeighting;

uniform sampler2D uSampler;

void main(void) {
  vec4 texColor = texture2D(uSampler, vec2(vTextureCoord.s, TextureCoord.t));
  gl_FragColor = vec4(texColor.xyz * vLightWeighting, 1);
  // To debug: renderling vLightWeighting shows that it is constant
  //gl_FragColor = vec4(vLightWeighting, 1);
}


Vertex shader:

attribute vec3 aVertexPosition;
attribute vec3 aVertexNormal;
attribute vec2 aTextureCoord;

uniform mat4 uMVMatrix;
uniform mat4 uCMatrix;
uniform mat4 uPMatrix;
uniform mat4 uNMatrix;

varying vec2 vTextureCoord;
varying vec3 vLightWeighting;

void main(void) {
  gl_Position = uPMatrix * uCMatrix * uMVMatrix * vec4(aVertexPosition, 1.0);
  vTextureCoord = aTextureCoord;

  vec3 transformedNormal = normalize((uNMatrix * vec4(normalize(aVertexNormal), 0)).xyz);
  vec3 lightingDirection = normalize(vec3(1, 1, 1));
  float directionalLightWeighting = max(dot(transformedNormal, lightingDirection), 0.0);
  vec3 uAmbientLightColor = vec3(0.2, 0.2, 0.2);
  vec3 uDirectionalColor = vec3(0.8, 0.8, 0.8);
  vLightWeighting = uAmbientLightColor + uDirectionalColor  * directionalLightWeighting;
}

Comment 7

[Vladimir Vukicevic [:vlad] [:vladv] (needinfo me, slow to respond)]

Is there a testcase available for this?

Comment 8

[Sebastian]

sorry for not replying for a while. had to do a lot of different work. with the actual version (4.0b9pre 2010-12-25), everything works as expected with validator turned on. no more differences visible! I think, bug can be closed. Thank you for the work on FF4 and WebGL!