590729 - Update WebGL tests to upstream r12308, and Mochitest improvements

Status: RESOLVED FIXED

(Core :: Graphics: CanvasWebGL, defect)

Description

[Benoit Jacob [:bjacob] (mostly away)]

Attachment: Update WebGL tests to upstream r12308, and Mochitest improvements

This patch does 3 things:

 * it syncs our copy of the WebGL tests with upstream (khronos.org) svn revision 12308.

 * for the errors about having to remove a page from failing_tests.txt, it only considers OSMesa results. This avoids inconsistent errors when testing with native GL.

 * it adds useful, parsable console output about test results:
   -- successful pages give 'WebGL test page successful: ' + filename
   -- failing pages give 'WebGL test error: test page failure: ' + filename
   -- pages timing out give 'WebGL test error: test page timeout: ' + filename
   -- pages that must me removed from failing_tests.txt give 'WebGL test error: page must be removed from failing_tests.txt: ' + filename


So for example, you could run the mochitest like this if you want to record this info:

EXTRA_TEST_ARGS='--setpref=webgl.osmesalib=\"libOSMesa.so.6\"' TEST_PATH=content/canvas/test/webgl/test_webgl_conformance_test_suite.html make mochitest-plain 2>&1 | tee log

And you can then parse this info like this:

 - If you want a list of successful WebGL test pages:

grep 'WebGL test page successful' log

 - If you want a list of WebGL test page having errors:

grep 'WebGL test error' log

 - If you want to update failing_tests.txt:

grep 'WebGL test error' log | cut -d ':' -f 3 | cut -d ' ' -f 2 > failing_tests.txt


This blocks 589814 because it makes Armen's life easier (the logging, and also the number of timeouts reduced to 1 and that's actually a plain JS error so will give same result everywhere).

Comment 1

[Vladimir Vukicevic [:vlad] [:vladv] (needinfo me, slow to respond)]

Comment on attachment 469222 [details] [diff] [review]
Update WebGL tests to upstream r12308, and Mochitest improvements

Looks great, land away!

Comment 2

[Benoit Jacob [:bjacob] (mostly away)]

Landed: http://hg.mozilla.org/mozilla-central/rev/39755990a9ae

Notice that the last timeout remaining is the plain JS error reported there:
http://www.khronos.org/bugzilla/show_bug.cgi?id=350