1361946 - reftest fails consistently on taskcluster windows 10

Status: RESOLVED DUPLICATE of bug 1435844

(Release Engineering :: General, defect)

Description

[Rob Thijssen [:grenade (EET/UTC+0300)]]

reftest fails on windows 10 with:

---
display-block-baselines-1-ref.html | image comparison, max difference: 24, number of differing pixels: 68

for chunk 5 on both opt and debug

see: https://treeherder.mozilla.org/#/jobs?repo=try&revision=6c7c0c515f27da8b910435363cb8e9ad54286ce4&filter-searchStr=reftest-5&exclusion_profile=false&group_state=expanded&selectedJob=96407451

---
1089388-2-ref.html | image comparison, max difference: 255, number of differing pixels: 5

for chunk 8 on opt

see: https://treeherder.mozilla.org/#/jobs?repo=try&revision=6c7c0c515f27da8b910435363cb8e9ad54286ce4&filter-searchStr=reftest-8&exclusion_profile=false&group_state=expanded

this is tracked as an intermittent on buildbot tests under bug 1316946

Comment 1

[Joel Maher ( :jmaher ) (UTC -8)]

on a recent push:
https://treeherder.mozilla.org/#/jobs?repo=try&revision=3293c486132d7ba24da29279727bcc3a35c3ea2d&exclusion_profile=false&filter-tier=1&filter-tier=2&filter-tier=3&filter-searchStr=win x64 reftest&selectedJob=100058686

I see ran r2/r3 in e10s mode for opt- both r2/ru2 and r3/ru3 fail for the same test on both opt/debug.  Running on a win10-ix box these run just fine- this is done by hand, on build 1703, and hardware; ideally we could run these by hand on a win10 loaner vm and see what happens.

Comment 2

[Justin Wood (:Callek)]

Not an issue on mozilla-central, so not related to last wednesdays migration

Comment 3

[Chris AtLee [:catlee]]

Is this still an issue?

Comment 4

[Rob Thijssen [:grenade (EET/UTC+0300)]]

seems to only be an issue on Stylo-disabled tests:
https://treeherder.mozilla.org/#/jobs?repo=try&revision=f385bc93a5b5c52457f69222b709a963de851110&group_state=expanded&selectedJob=136766053

Comment 5

[Joel Maher ( :jmaher ) (UTC -8)]

we still get many blue jobs and the one perma fail is actually passing because the acceptable range of 'fuzzy' pixels is accurate for stylo, but not for non stylo- I assume it was documented as stylo when we were switching to win10 and we left the old annotation in place.

Comment 6

[Rob Thijssen [:grenade (EET/UTC+0300)]]

these appear to be passing now

Comment 7

[Rob Thijssen [:grenade (EET/UTC+0300)]]

i misread the test results where successes were on buildbot

Comment 8

[Joel Maher ( :jmaher ) (UTC -8)]

we are fixing this in bug 1435844 as we will be running reftests on windows 10