Annotate svg-as-an-image reftests that fail on d2d as "fails-if", not "random-if"

RESOLVED FIXED

Status

()

RESOLVED FIXED
8 years ago
8 years ago

People

(Reporter: dholbert, Assigned: dholbert)

Tracking

Trunk
Points:
---

Firefox Tracking Flags

(Not tracked)

Details

Attachments

(1 attachment)

(Assignee)

Description

8 years ago
Currently, the following tests are marked random-if(d2d):
> 63 random-if(d2d) == img-widthAndHeight-meet-1.html  img-widthAndHeight-meet-1-ref.html
> 64 random-if(d2d) == img-widthAndHeight-meet-2.html  img-widthAndHeight-meet-2-ref.html
> 65 random-if(d2d) == img-widthAndHeight-slice-1.html img-widthAndHeight-slice-1-ref.html
> 66 random-if(d2d) == img-widthAndHeight-slice-2.html img-widthAndHeight-slice-2-ref.html
> 67 
> 68 random-if(d2d) == img-height-meet-1.html   img-height-meet-1-ref.html
> 69 random-if(d2d) == img-height-meet-2.html   img-height-meet-2-ref.html
> 70 random-if(d2d) == img-height-slice-1.html  img-height-slice-1-ref.html
> 71 random-if(d2d) == img-height-slice-2.html  img-height-slice-2-ref.html
> 72 
> 73 random-if(d2d) == img-width-meet-1.html   img-width-meet-1-ref.html
> 74 random-if(d2d) == img-width-meet-2.html   img-width-meet-2-ref.html
> 75 random-if(d2d) == img-width-slice-1.html  img-width-slice-1-ref.html
> 76 random-if(d2d) == img-width-slice-2.html  img-width-slice-2-ref.html
http://mxr.mozilla.org/mozilla-central/source/layout/reftests/svg/as-image/reftest.list?force=1

I'm pretty sure they're all perma-fail (with a few px difference) on D2D-enabled windows, though -- not random.

Filing this bug on
  (a) verifying that the above is true on TryServer
  (b) applying s/random-if/fails-if/ (assuming that (a) succeeds)
(Assignee)

Comment 1

8 years ago
I just gave a quick whack at (a), layered on top of my patches for bug 614649 -- we'll see how it turns out.

Tentative-patch: http://hg.mozilla.org/try/rev/866ed3a1ea1d
TryServer tbpl: http://tbpl.mozilla.org/?tree=MozillaTry&rev=575929d8cb0f
(Assignee)

Comment 2

8 years ago
Turns out some of the tests seem to pass.  Pushed to try again with the fails-if/random-if(d2d) annotation completely removed for those tests, and it came up green:  http://tbpl.mozilla.org/?tree=MozillaTry&43f9c9e39e66

I just requested additional reftest runs for that build, to get a little more statistical significance. If they're green, too, then I think we can assume these tests are stable.
(Assignee)

Comment 3

8 years ago
oops, I left out the "rev=" in the URL from comment 2.  Meant to say:
  http://tbpl.mozilla.org/?tree=MozillaTry&rev=43f9c9e39e66
(Assignee)

Comment 4

8 years ago
Ok, cool - so I got 4 reftest runs each, on Win debug and Win opt, and they were all green (modulo a known randomorange in plugins code) - so I think it's safe to say that these tests all reliably pass or fail as marked in that TryServer push:  http://hg.mozilla.org/try/rev/315189c83a1c
(Assignee)

Comment 5

8 years ago
Created attachment 511275 [details] [diff] [review]
fix: annotate tests as passing/failing based on TryServer performance
Assignee: nobody → dholbert
Status: NEW → ASSIGNED
Attachment #511275 - Flags: review?(roc)
(Assignee)

Comment 6

8 years ago
I don't think there's a bug tracking these D2D failures themselves, so I filed bug 633072 on that.

I added a comment with that bug number to all the "fails-if(d2d)" entries in as-image/reftest.list, and landed: http://hg.mozilla.org/mozilla-central/rev/f00a192644fd
Status: ASSIGNED → RESOLVED
Last Resolved: 8 years ago
Resolution: --- → FIXED
You need to log in before you can comment on or make changes to this bug.