Closed Bug 726400 Opened 14 years ago Closed 11 years ago

Intermittent REFTEST TEST-UNEXPECTED-FAIL | svg/smil/anim-path-d-01.svg | image comparison (==)

Categories

(Core :: SVG, defect)

Product:
Core
Component:
SVG
Platform:
x86
Windows XP
Type:
defect
Priority:
Not set
Severity:
normal

Tracking

()

Status:
RESOLVED FIXED
Milestone:
mozilla30
Tracking Flags:
Tracking Status
firefox28 --- fixed
firefox29 --- fixed
firefox30 --- fixed
firefox-esr24 --- fixed
b2g-v1.2 --- fixed
b2g-v1.3 --- fixed
b2g-v1.3T --- fixed
b2g-v1.4 --- fixed

People

(Reporter: philor, Assigned: longsonr)

References

Details

(Keywords: intermittent-failure)

Attachments

(2 files)

reftest log
45.25 KB, text/plain
Details
Mark test as fuzzy
1.10 KB, patch
dholbert
: review+
jwatt
: checkin+
Details | Diff | Splinter Review
Attached file reftest log —
https://tbpl.mozilla.org/php/getParsedLog.php?id=9261006&tree=Mozilla-Inbound Rev3 WINNT 5.1 mozilla-inbound debug test reftest on 2012-02-10 20:50:48 PST for push dfe18a0ece02 https://tbpl.mozilla.org/php/getParsedLog.php?id=9272509&tree=Mozilla-Inbound Rev3 WINNT 5.1 mozilla-inbound debug test reftest on 2012-02-11 21:15:12 PST for push 6ac07d3e3a61 REFTEST TEST-START | file:///c:/talos-slave/test/build/reftest/tests/layout/reftests/svg/smil/anim-path-d-01-ref.svg | 6034 / 7358 (82%) ++DOMWINDOW == 50 (08FF5030) [serial = 17405] [outer = 051D3E88] REFTEST fuzzy match REFTEST TEST-UNEXPECTED-FAIL | file:///c:/talos-slave/test/build/reftest/tests/layout/reftests/svg/smil/anim-path-d-01.svg | image comparison (==) ... REFTEST number of differing pixels: 1 max difference: 1 REFTEST INFO | Saved log: START file:///c:/talos-slave/test/build/reftest/tests/layout/reftests/svg/smil/anim-path-d-01.svg REFTEST INFO | Saved log: [CONTENT] OnDocumentLoad triggering AfterOnLoadScripts REFTEST INFO | Saved log: Initializing canvas snapshot REFTEST INFO | Saved log: DoDrawWindow 0,0,800,1000 REFTEST INFO | Saved log: [CONTENT] RecordResult fired REFTEST INFO | Saved log: RecordResult fired REFTEST INFO | Saved log: START file:///c:/talos-slave/test/build/reftest/tests/layout/reftests/svg/smil/anim-path-d-01-ref.svg REFTEST INFO | Saved log: [CONTENT] OnDocumentLoad triggering AfterOnLoadScripts REFTEST INFO | Saved log: Initializing canvas snapshot REFTEST INFO | Saved log: DoDrawWindow 0,0,800,1000 REFTEST INFO | Saved log: [CONTENT] RecordResult fired REFTEST INFO | Saved log: RecordResult fired REFTEST INFO | Loading a blank page
The failure just has a single-pixel difference, at the point of an angle. It's rgb(174,174,255) in the testcase but rgb(175,175,255) in the reference case. The pushlog for 2 days before the first report in comment 0 (likely to contain the regressor, since this has been failing about once per 24 hours): http://hg.mozilla.org/integration/mozilla-inbound/pushloghtml?fromchange=ea77fdb9db83&tochange=71f5bf4df2f6 The only obviously-SVG-related stuff in there is this push by jwatt: http://hg.mozilla.org/integration/mozilla-inbound/pushloghtml?changeset=073b91c24a1e Its checkin messages mention covered regions, which sounds plausible as being able to cause miscalculations on a stroked pixel at the very point of an angle. jwatt, if you have cycles to look into this, it'd probably be good -- 1 failure per 24 hours is fairly frequent. Alternately, we could always label the test with the keyword mattwoodrow added recently ("fuzz" or something?) that allows us to say something like "N pixels are allowed be off by M%". (I'm not advocating that we do that, merely pointing out that it's an option.)
(In reply to Daniel Holbert [:dholbert] from comment #6) > The only obviously-SVG-related stuff in there is this push by jwatt: > http://hg.mozilla.org/integration/mozilla-inbound/ > pushloghtml?changeset=073b91c24a1e For the record, I pushed four SVG changes together - that's only the last of the four. Let's see if the patch in bug 726928 fixes this (not too hopeful about that, but maybe...). > Alternately, we could always label the test with the keyword mattwoodrow > added recently ("fuzz" or something?) that allows us to say something like > "N pixels are allowed be off by M%". (I'm not advocating that we do that, > merely pointing out that it's an option.) I think fuzzy only allows pixel components to differ by 1 (out of 255), not a percentage. That would seem sufficient given the rgb differences in comment 6 though.
Attached patch Mark test as fuzzy — — Splinter Review
Looking at this, I can't see why the covered regions stuff would cause this to fail, so here's a patch to mark the test as fuzzy if we need to. I'd also note that once again this is a WinXP failure that's happening only on mozilla-inbound, not mozilla-central. Weird.
Assignee: nobody → jwatt
Attachment #598059 - Flags: review?(dholbert)
(In reply to Jonathan Watt [:jwatt] from comment #8) > Looking at this, I can't see why the covered regions stuff would cause this > to fail, so here's a patch to mark the test as fuzzy if we need to. I'm OK with this given that (a) the failure is unrelated to what this test is actually testing (b) it's only a single pixel & isn't visually perceptible (c) it's only on one (old-ish) platform (and the fuzz annotation targets that platform) If we start hitting this on other platforms, I'd be less onboard with a fuzzy-everywhere annotation. > I'd also note that once again this is [...] happening only > on mozilla-inbound, not mozilla-central. Weird. Keep in mind that mozilla-inbound gets many, many more test-cycles than m-c does. Just picking a number at random, let's say m-i has 6x as many pushes as m-c -- then we'd expect to hit this on m-c once every 6 days (instead of every ~1 day on m-i). It's only just now been 6 days since the comment 0 timeframe was merged over to m-c, so I'd expect we'll start hitting this on m-c pretty soon. (aside from the fact that we're about to mark it as fuzzy). :)
Comment on attachment 598059 [details] [diff] [review] Mark test as fuzzy >-== anim-path-d-01.svg anim-path-d-01-ref.svg >+fuzzy-if(/^Windows\x20NT\x205\.1/.test(http.oscpu)) == anim-path-d-01.svg anim-path-d-01-ref.svg Add at the end of this line: # Bug 726400 r=me with that
Attachment #598059 - Flags: review?(dholbert) → review+
(In reply to Daniel Holbert [:dholbert] from comment #10) > >-== anim-path-d-01.svg anim-path-d-01-ref.svg > >+fuzzy-if(/^Windows\x20NT\x205\.1/.test(http.oscpu)) == anim-path-d-01.svg anim-path-d-01-ref.svg > > Add at the end of this line: > # Bug 726400 > > r=me with that I took the liberty of making this change and pushing the patch, to stop the orange. https://hg.mozilla.org/integration/mozilla-inbound/rev/5f235a1502dd
Target Milestone: --- → mozilla13
Status: NEW → ASSIGNED
Backed out for: { REFTEST TEST-UNEXPECTED-FAIL | | EXCEPTION: Error 1 in manifest file file:///home/cltbld/talos-slave/test/build/reftest/tests/layout/reftests/svg/smil/reftest.list line 125 } https://tbpl.mozilla.org/?tree=Mozilla-Inbound&rev=5f235a1502dd https://hg.mozilla.org/integration/mozilla-inbound/rev/b030104de654
Target Milestone: mozilla13 → ---
Depends on: 729784
No longer depends on: 729784
Thanks, Ed -- sorry about that. Looks like fuzzy-if takes extra parameters now, as of bug 718329, for difference-threshold and number of differing pixels. Re-pushed w/ values for those arguments (difference = 2, num pixels = 1, per first chunk in comment 6): https://hg.mozilla.org/integration/mozilla-inbound/rev/874051cddf05 (I verified locally that the reftest harness likes the updated patch.)
Whiteboard: [orange] → [orange][test marked as fuzzy]
Whiteboard: [orange][test marked as fuzzy] → [test marked as fuzzy]
Attachment #598059 - Flags: checkin+
Fixed by check-in of bug 971285
Status: ASSIGNED → RESOLVED
Closed: 11 years ago
Resolution: --- → FIXED
Whiteboard: [test marked as fuzzy]
Assignee: jwatt → longsonr
status-firefox28: --- → affected
status-firefox29: --- → affected
status-firefox30: --- → fixed
status-firefox-esr24: --- → affected
Depends on: 971285
Target Milestone: --- → mozilla30
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Creator:
Created:
Updated:
Size: