Closed Bug 1193447 Opened 4 years ago Closed 2 years ago

[R] Tracking Reftests for win10 failing

Categories

(Testing :: Reftest, defect)

Unspecified
Windows 10
defect
Not set

Tracking

(Not tracked)

RESOLVED FIXED

People

(Reporter: Callek, Assigned: bas.schouten)

References

Details

So, Reftests for win 10 suffer from a few pandemic issues, I'm unsure how many are the same issue nor how many of those issues are code and how many are machine-configuration.

This bug is to help identify the sub-bugs and to green reftests up for win10

We're looking at:

Debug:
https://treeherder.mozilla.org/logviewer.html#?job_id=10282100&repo=try
(analyzer): http://hg.mozilla.org/mozilla-central/raw-file/tip/layout/tools/reftest/reftest-analyzer.xhtml#logurl=http://ftp.mozilla.org/pub/mozilla.org/firefox/try-builds/Callek@gmail.com-8cf3952a5c3a/try-win64-debug/try_win10_64-debug_test-reftest-bm109-tests1-windows-build2.txt.gz&only_show_unexpected=1

And Opt:
https://treeherder.mozilla.org/logviewer.html#?job_id=10283144&repo=try
(analyzer): http://hg.mozilla.org/mozilla-central/raw-file/tip/layout/tools/reftest/reftest-analyzer.xhtml#logurl=http://ftp.mozilla.org/pub/mozilla.org/firefox/try-builds/Callek@gmail.com-8cf3952a5c3a/try-win64/try_win10_64_test-reftest-bm109-tests1-windows-build1.txt.gz&only_show_unexpected=1

The issues *seem* to me to be of 3 main types, with a few genuine fails spread in:

* Text (kerning/aliasing it seems at my untrained eye)
* Rounded Corners (probably aliasing)
* Opacity (some tests seem to have darker areas in the test vs the reference image)

Milan :vlad nominated you to look into this.
I know we don't really assign meta bugs (and this one feels like it), but Bas is the best person to judge (or get feedback) on what the real problems are (the icon one?) and what should just be fuzzed.
Assignee: nobody → bas
Flags: needinfo?(bas)
OS: Unspecified → Windows 10
I had a quick look, there's an issue here where textmetrics are different between test and reference, those are probably best investigated by jfkthame or jdaggett.

Then there's a bunch of very subtle anti-aliasing differences, those are most likely just improvements to Direct2D showing up and should be best dealt by fuzzing them.

Then there's one or two ones with an actually interesting blending difference which seems like it might be improved from a code perspective. The interesting one being file:///C:/slave/test/build/tests/reftest/tests/layout/reftests/svg/smil/motion/animateMotion-flattening-1.svg, this is likely best investigated by someone who knows SVG animations.

Then there's the icon one, I'll see if I can reproduce that one.
Flags: needinfo?(bas)
Depends on: 1195499
https://dxr.mozilla.org/mozilla-central/source/image/test/reftest/icon/win/reftest.list#1

Can eliminate that first one (icon) I'll tackle it in Bug 1195499 along with the fuzzing.

...I just pushed a new job (based on current inbound tip) to try with all the w8 fails/fuz/skips that also fail on w10 now addressed incase you're curious.

https://treeherder.mozilla.org/#/jobs?repo=try&revision=1599243602f5
Depends on: 1196206
Bas, can you elaborate on your c#2 based on c#3 here, and point out if there are any other already-filed bugs tracking the remaining issues?

(if not I'll file, so we can get reftests green on w10, even if thats by fails-if, fuzz, etc)
Flags: needinfo?(bas)
(In reply to Justin Wood (:Callek) [back on Mar 21] from comment #4)
> Bas, can you elaborate on your c#2 based on c#3 here, and point out if there
> are any other already-filed bugs tracking the remaining issues?
> 
> (if not I'll file, so we can get reftests green on w10, even if thats by
> fails-if, fuzz, etc)

Is this still an issue?
Flags: needinfo?(bas)
This bug was from 2015 and has been superseeded by the migration we've already done for talos tests to w10 and the taskcluster work that's ongoing.
Status: NEW → RESOLVED
Closed: 2 years ago
Resolution: --- → FIXED
You need to log in before you can comment on or make changes to this bug.