Intermittent REFTEST TEST-UNEXPECTED-FAIL | svg/smil/anim-path-d-01.svg | image comparison (==)

RESOLVED FIXED in Firefox 28, Firefox OS v1.2

Status

()

Core
SVG
RESOLVED FIXED
6 years ago
4 years ago

People

(Reporter: philor, Assigned: longsonr)

Tracking

({intermittent-failure})

Trunk
mozilla30
x86
Windows XP
intermittent-failure
Points:
---
Dependency tree / graph

Firefox Tracking Flags

(firefox28 fixed, firefox29 fixed, firefox30 fixed, firefox-esr24 fixed, b2g-v1.2 fixed, b2g-v1.3 fixed, b2g-v1.3T fixed, b2g-v1.4 fixed)

Details

Attachments

(2 attachments)

(Reporter)

Description

6 years ago
Created attachment 596425 [details]
reftest log

https://tbpl.mozilla.org/php/getParsedLog.php?id=9261006&tree=Mozilla-Inbound
Rev3 WINNT 5.1 mozilla-inbound debug test reftest on 2012-02-10 20:50:48 PST for push dfe18a0ece02

https://tbpl.mozilla.org/php/getParsedLog.php?id=9272509&tree=Mozilla-Inbound
Rev3 WINNT 5.1 mozilla-inbound debug test reftest on 2012-02-11 21:15:12 PST for push 6ac07d3e3a61

REFTEST TEST-START | file:///c:/talos-slave/test/build/reftest/tests/layout/reftests/svg/smil/anim-path-d-01-ref.svg | 6034 / 7358 (82%)
++DOMWINDOW == 50 (08FF5030) [serial = 17405] [outer = 051D3E88]
REFTEST fuzzy match
REFTEST TEST-UNEXPECTED-FAIL | file:///c:/talos-slave/test/build/reftest/tests/layout/reftests/svg/smil/anim-path-d-01.svg | image comparison (==)
...
REFTEST number of differing pixels: 1 max difference: 1
REFTEST INFO | Saved log: START file:///c:/talos-slave/test/build/reftest/tests/layout/reftests/svg/smil/anim-path-d-01.svg
REFTEST INFO | Saved log: [CONTENT] OnDocumentLoad triggering AfterOnLoadScripts
REFTEST INFO | Saved log: Initializing canvas snapshot
REFTEST INFO | Saved log: DoDrawWindow 0,0,800,1000
REFTEST INFO | Saved log: [CONTENT] RecordResult fired
REFTEST INFO | Saved log: RecordResult fired
REFTEST INFO | Saved log: START file:///c:/talos-slave/test/build/reftest/tests/layout/reftests/svg/smil/anim-path-d-01-ref.svg
REFTEST INFO | Saved log: [CONTENT] OnDocumentLoad triggering AfterOnLoadScripts
REFTEST INFO | Saved log: Initializing canvas snapshot
REFTEST INFO | Saved log: DoDrawWindow 0,0,800,1000
REFTEST INFO | Saved log: [CONTENT] RecordResult fired
REFTEST INFO | Saved log: RecordResult fired
REFTEST INFO | Loading a blank page
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
The failure just has a single-pixel difference, at the point of an angle.  It's rgb(174,174,255) in the testcase but rgb(175,175,255) in the reference case.

The pushlog for 2 days before the first report in comment 0 (likely to contain the regressor, since this has been failing about once per 24 hours):
http://hg.mozilla.org/integration/mozilla-inbound/pushloghtml?fromchange=ea77fdb9db83&tochange=71f5bf4df2f6

The only obviously-SVG-related stuff in there is this push by jwatt:
 http://hg.mozilla.org/integration/mozilla-inbound/pushloghtml?changeset=073b91c24a1e

Its checkin messages mention covered regions, which sounds plausible as being able to cause miscalculations on a stroked pixel at the very point of an angle.

jwatt, if you have cycles to look into this, it'd probably be good -- 1 failure per 24 hours is fairly frequent.

Alternately, we could always label the test with the keyword mattwoodrow added recently ("fuzz" or something?) that allows us to say something like "N pixels are allowed be off by M%". (I'm not advocating that we do that, merely pointing out that it's an option.)
(In reply to Daniel Holbert [:dholbert] from comment #6)
> The only obviously-SVG-related stuff in there is this push by jwatt:
>  http://hg.mozilla.org/integration/mozilla-inbound/
> pushloghtml?changeset=073b91c24a1e

For the record, I pushed four SVG changes together - that's only the last of the four.

Let's see if the patch in bug 726928 fixes this (not too hopeful about that, but maybe...).

> Alternately, we could always label the test with the keyword mattwoodrow
> added recently ("fuzz" or something?) that allows us to say something like
> "N pixels are allowed be off by M%". (I'm not advocating that we do that,
> merely pointing out that it's an option.)

I think fuzzy only allows pixel components to differ by 1 (out of 255), not a percentage. That would seem sufficient given the rgb differences in comment 6 though.
Created attachment 598059 [details] [diff] [review]
Mark test as fuzzy

Looking at this, I can't see why the covered regions stuff would cause this to fail, so here's a patch to mark the test as fuzzy if we need to.

I'd also note that once again this is a WinXP failure that's happening only on mozilla-inbound, not mozilla-central. Weird.
Assignee: nobody → jwatt
Attachment #598059 - Flags: review?(dholbert)
(In reply to Jonathan Watt [:jwatt] from comment #8)
> Looking at this, I can't see why the covered regions stuff would cause this
> to fail, so here's a patch to mark the test as fuzzy if we need to.

I'm OK with this given that
  (a) the failure is unrelated to what this test is actually testing
  (b) it's only a single pixel & isn't visually perceptible
  (c) it's only on one (old-ish) platform (and the fuzz annotation targets that platform)

If we start hitting this on other platforms, I'd be less onboard with a fuzzy-everywhere annotation.

> I'd also note that once again this is [...] happening only
> on mozilla-inbound, not mozilla-central. Weird.

Keep in mind that mozilla-inbound gets many, many more test-cycles than m-c does.

Just picking a number at random, let's say m-i has 6x as many pushes as m-c -- then we'd expect to hit this on m-c once every 6 days (instead of every ~1 day on m-i).  It's only just now been 6 days since the comment 0 timeframe was merged over to m-c, so I'd expect we'll start hitting this on m-c pretty soon. (aside from the fact that we're about to mark it as fuzzy). :)
Comment on attachment 598059 [details] [diff] [review]
Mark test as fuzzy

>-== anim-path-d-01.svg anim-path-d-01-ref.svg
>+fuzzy-if(/^Windows\x20NT\x205\.1/.test(http.oscpu)) == anim-path-d-01.svg anim-path-d-01-ref.svg

Add at the end of this line:
  # Bug 726400

r=me with that
Attachment #598059 - Flags: review?(dholbert) → review+
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
Comment hidden (Treeherder Robot)
(In reply to Daniel Holbert [:dholbert] from comment #10)
> >-== anim-path-d-01.svg anim-path-d-01-ref.svg
> >+fuzzy-if(/^Windows\x20NT\x205\.1/.test(http.oscpu)) == anim-path-d-01.svg anim-path-d-01-ref.svg
> 
> Add at the end of this line:
>   # Bug 726400
> 
> r=me with that

I took the liberty of making this change and pushing the patch, to stop the orange.
https://hg.mozilla.org/integration/mozilla-inbound/rev/5f235a1502dd
Target Milestone: --- → mozilla13
Status: NEW → ASSIGNED
Backed out for:
{
REFTEST TEST-UNEXPECTED-FAIL | | EXCEPTION: Error 1 in manifest file file:///home/cltbld/talos-slave/test/build/reftest/tests/layout/reftests/svg/smil/reftest.list line 125
}

https://tbpl.mozilla.org/?tree=Mozilla-Inbound&rev=5f235a1502dd


https://hg.mozilla.org/integration/mozilla-inbound/rev/b030104de654
Target Milestone: mozilla13 → ---
Depends on: 729784
No longer depends on: 729784
Thanks, Ed -- sorry about that.  Looks like fuzzy-if takes extra parameters now, as of bug 718329, for difference-threshold and number of differing pixels.

Re-pushed w/ values for those arguments (difference = 2, num pixels = 1, per first chunk in comment 6):
  https://hg.mozilla.org/integration/mozilla-inbound/rev/874051cddf05

(I verified locally that the reftest harness likes the updated patch.)
Comment hidden (Treeherder Robot)

Updated

5 years ago
Whiteboard: [orange] → [orange][test marked as fuzzy]
Keywords: intermittent-failure
Whiteboard: [orange][test marked as fuzzy] → [test marked as fuzzy]
(Assignee)

Comment 22

4 years ago
Fixed by check-in of bug 971285
Status: ASSIGNED → RESOLVED
Last Resolved: 4 years ago
Resolution: --- → FIXED
(Assignee)

Updated

4 years ago
Whiteboard: [test marked as fuzzy]
Assignee: jwatt → longsonr
status-firefox28: --- → affected
status-firefox29: --- → affected
status-firefox30: --- → fixed
status-firefox-esr24: --- → affected
Depends on: 971285
Target Milestone: --- → mozilla30
https://hg.mozilla.org/releases/mozilla-aurora/rev/ec87f9ef48ec
https://hg.mozilla.org/releases/mozilla-beta/rev/53cc37f62b49
https://hg.mozilla.org/releases/mozilla-b2g26_v1_2/rev/9aff0bf49198
https://hg.mozilla.org/releases/mozilla-esr24/rev/2b3e6c8ea260
status-b2g-v1.2: --- → fixed
status-b2g-v1.4: --- → fixed
status-firefox28: affected → fixed
status-firefox29: affected → fixed
status-firefox-esr24: affected → fixed
status-b2g-v1.3T: --- → fixed
You need to log in before you can comment on or make changes to this bug.