The default bug view has changed. See this FAQ.

WinXP debug intermittent but very frequent: test_webgl_conformance_test_suite.html | Test expected to fail, but passed: conformance/canvas-test.html

RESOLVED FIXED in mozilla6

Status

()

Core
Canvas: WebGL
RESOLVED FIXED
6 years ago
4 years ago

People

(Reporter: philor, Assigned: Joe Drew (not getting mail))

Tracking

({intermittent-failure})

Trunk
mozilla6
x86
Windows XP
intermittent-failure
Points:
---
Dependency tree / graph

Firefox Tracking Flags

(status2.0 ?)

Details

(Whiteboard: [test disabled])

Attachments

(1 attachment)

(Reporter)

Description

6 years ago
Ever since they were turned on this morning, every WinXP debug mochitest-1 has been orange like

http://tinderbox.mozilla.org/showlog.cgi?log=Firefox/1299774166.1299776765.2061.gz
Rev3 WINNT 5.1 mozilla-central debug test mochitests-1/5 on 2011/03/10 08:22:46
s: talos-r3-xp-015

36154 ERROR TEST-UNEXPECTED-FAIL | /tests/content/canvas/test/webgl/test_webgl_conformance_test_suite.html | Test expected to fail, but passed: conformance/canvas-test.html

So, I pushed http://hg.mozilla.org/try/rev/58bba9e5bb04 to try, to only expect it to fail in WinXP opt, and for my trouble got

http://tinderbox.mozilla.org/showlog.cgi?log=MozillaTry/1299822320.1299824852.17505.gz
Rev3 WINNT 5.1 tryserver debug test mochitests-1/5 on 2011/03/10 21:45:20
s: talos-r3-xp-045

36059 ERROR TEST-UNEXPECTED-FAIL | /tests/content/canvas/test/webgl/test_webgl_conformance_test_suite.html | Test failed, "pixels are 0,255,255,159 expected 0,0,0,0" (URL: conformance/canvas-test.html)

No idea whether that's just awful coincidence, and it's really random whether it fails or not, or a sign that there's a bug in the harness' ability to tell whether something that was supposed to fail did fail.
(Reporter)

Comment 1

6 years ago
Random. Ran my patch three more times, got one more failure with "Test failed, "pixels are 253,251,253,156 expected 0,0,0,0" (URL: conformance/canvas-test.html)" and two successes, so we either need to give the harness understanding of random-if, or just skip running it completely.
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
(Reporter)

Comment 4

6 years ago
But just being frequently orange isn't enough to let this hold an entire test suite hostage, I'm unhiding the WinXP Md1.
Blocks: 438871
Summary: WinXP debug permaorange: test_webgl_conformance_test_suite.html | Test expected to fail, but passed: conformance/canvas-test.html → WinXP debug intermittent but very frequent: test_webgl_conformance_test_suite.html | Test expected to fail, but passed: conformance/canvas-test.html
Whiteboard: [perma-orange] → [orange]
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
This is totally unacceptable.  Benoit, can you please investigate?  I'm afraid that I'm going to have to submit a patch to disable WebGL on XP if this test keeps failing this way, since I bet money that this is a code problem.
Assignee: nobody → bjacob
Well, this <http://mxr.mozilla.org/mozilla-central/source/content/canvas/test/webgl/test_webgl_conformance_test_suite.html?force=1#360> tells me that we're kind of working around an intermittent failure by pretending that the test always fails, and that's apparently the wrong assumption.  This looks like a very bad practice to me. :(
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)

Updated

6 years ago
Assignee: bjacob → ehsan
Depends on: 646229
Whiteboard: [orange] → [orange][will be fixed by bug 646229]
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
(Assignee)

Comment 73

6 years ago
Unfortunately this won't be fixed by updating ANGLE; it just seemed like it would.
Assignee: ehsan → joe
Whiteboard: [orange][will be fixed by bug 646229] → [orange]
(Assignee)

Comment 74

6 years ago
Created attachment 524935 [details] [diff] [review]
ignore canvas-test on XP

Since the canvas test can't be relied upon on Windows XP, let's just ignore the results of that test.

On try: http://tbpl.mozilla.org/?tree=MozillaTry&rev=cda3bc3bbaa6
Attachment #524935 - Flags: review?(bjacob)
(Assignee)

Comment 75

6 years ago
Comment on attachment 524935 [details] [diff] [review]
ignore canvas-test on XP

>diff --git a/content/canvas/test/webgl/test_webgl_conformance_test_suite.html b/content/canvas/test/webgl/test_webgl_conformance_test_suite.html
>--- a/content/canvas/test/webgl/test_webgl_conformance_test_suite.html
>+++ b/content/canvas/test/webgl/test_webgl_conformance_test_suite.html
>@@ -50,16 +50,17 @@ function start() {
> 
>   var reportType = WebGLTestHarnessModule.TestHarness.reportType;
> 
>   var Page = function(reporter, url) {
>     this.reporter = reporter;
>     this.url = url;
>     this.totalTests = 0;
>     this.totalSuccessful = 0;
>+    this.totalFailed = 0;
>     this.totalTimeouts = 0;
> 
>     var li = reporter.localDoc.createElement('li');
>     var div = reporter.localDoc.createElement('div');
>     var a = reporter.localDoc.createElement('a');
>     a.href = url;
>     var node = reporter.localDoc.createTextNode(url);
>     a.appendChild(node);

Please ignore this hunk.
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment on attachment 524935 [details] [diff] [review]
ignore canvas-test on XP

OK, having a list of ignored tests is a good idea anyway.
Attachment #524935 - Flags: review?(bjacob) → review+
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
http://hg.mozilla.org/projects/cedar/rev/7a0acca5854b

This makes me sad.  :(
Whiteboard: [orange] → [orange][fixed-in-cedar][test disabled]
I'd appreciate if you guys can debug this when you're back in the office.  I really want to see what's the cause of the failure here.  Thanks!
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Test disabled in http://hg.mozilla.org/mozilla-central/rev/7a0acca5854b
Whiteboard: [orange][fixed-in-cedar][test disabled] → [orange][test disabled]
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
(In reply to comment #95)
> Test disabled in http://hg.mozilla.org/mozilla-central/rev/7a0acca5854b

FWIW, comment 96, 97 and 98 happened on revisions before this one.
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
(Assignee)

Updated

6 years ago
No longer depends on: 646229
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
I think we should disable this particular test on 2.0
Comment hidden (Treeherder Robot)
The intermittent orange has been fixed by disabling the test.  Please file a new bug for investigating why it happens if needed.
Status: NEW → RESOLVED
Last Resolved: 6 years ago
Resolution: --- → FIXED
status2.0: --- → ?
Target Milestone: --- → mozilla6
Keywords: intermittent-failure
Whiteboard: [orange][test disabled] → [test disabled]
You need to log in before you can comment on or make changes to this bug.