Open Bug 1574067 Opened 3 months ago Updated 21 days ago

Categories

(Core :: CSS Parsing and Computation, defect, P5)

defect

Tracking

()

People

(Reporter: intermittent-bug-filer, Unassigned, NeedInfo)

Details

(Keywords: intermittent-failure, leave-open, regression, Whiteboard: [stockwell unknown])

Attachments

(4 obsolete files)

Filed by: cbrindusan [at] mozilla.com
Parsed log: https://treeherder.mozilla.org/logviewer.html#?job_id=261729387&repo=mozilla-inbound
Full log: https://queue.taskcluster.net/v1/task/LzqdEidaSMSmKGklFK9nXg/runs/0/artifacts/public/logs/live_backing.log
Reftest URL: https://hg.mozilla.org/mozilla-central/raw-file/tip/layout/tools/reftest/reftest-analyzer.xhtml#logurl=https://queue.taskcluster.net/v1/task/LzqdEidaSMSmKGklFK9nXg/runs/0/artifacts/public/logs/live_backing.log&only_show_unexpected=1


[task 2019-08-15T05:09:06.164Z] 05:09:06 INFO - TEST-START | /css/css-ui/appearance-textfield-001.html
[task 2019-08-15T05:09:06.164Z] 05:09:06 INFO - PID 2736 | 1565845746160 Marionette INFO Testing http://web-platform.test:8000/css/css-ui/appearance-textfield-001.html == http://web-platform.test:8000/css/css-ui/appearance-textfield-001-ref.html
[task 2019-08-15T05:09:06.347Z] 05:09:06 INFO - PID 2736 | 1565845746344 Marionette INFO No differences allowed
[task 2019-08-15T05:09:06.405Z] 05:09:06 INFO - TEST-UNEXPECTED-FAIL | /css/css-ui/appearance-textfield-001.html | Testing http://web-platform.test:8000/css/css-ui/appearance-textfield-001.html == http://web-platform.test:8000/css/css-ui/appearance-textfield-001-ref.html

James, since https://hg.mozilla.org/mozilla-central/rev/dd98ad5fd916#l2.10 landed I believe the majority of failures classified here (which are on win 7 shippable opt) changed from unexpected fail to unexpected pass.

Could a possible solution be adding to the manifest file the following line?
if (os == "win") and not debug and (processor == "x86"): [FAIL, PASS]

The .ini seems already pretty extensive and a bit confusing.

Flags: needinfo?(james)

I'm pretty sure the right solution here is to annotate this as fuzzy on windows, maybe something like

fuzzy:
  if os="win": maxDifference=0-10;totalPixels=0-24
Flags: needinfo?(james)

Depends on D43009

Attachment #9087904 - Attachment is obsolete: true
Attachment #9087904 - Attachment is obsolete: false
Attachment #9087904 - Attachment is obsolete: true

Depends on D43009

Attachment #9087906 - Attachment is obsolete: true
Attachment #9087290 - Attachment description: Bug 1574067 - Update expectations cause of frequent windows !debug failures. r=jgraham → Bug 1574067 - Updated expectation to fuzzy on windows r=jgraham

James, did a try push with the changes you said but the tests are still failing: https://treeherder.mozilla.org/#/jobs?repo=try&group_state=expanded&revision=5e1ac06233ae54f560528a166b4e20fd6bee812a&selectedJob=263352469

Found 351, 81, 180, 143, 51 pixels different, maximum difference per channel 12

Will go ahead and revert the patch to the initial version if (os == "win") and not debug: [PASS, FAIL] and land it.

Attachment #9087962 - Attachment is obsolete: true
Pushed by csabou@mozilla.com:
https://hg.mozilla.org/integration/mozilla-inbound/rev/6dd86b824d3f
Update test expectations on windows !debug cause of frequent failures. r=jgraham
Attachment #9087290 - Attachment is obsolete: true
Keywords: leave-open
Whiteboard: [stockwell disable-recommended]
Whiteboard: [stockwell disable-recommended]
Whiteboard: [stockwell disable-recommended]
Whiteboard: [stockwell disable-recommended]
Whiteboard: [stockwell disable-recommended]
Flags: needinfo?(svoisen)
Whiteboard: [stockwell needswork:owner]
You need to log in before you can comment on or make changes to this bug.