Closed Bug 1591611 Opened 2 years ago Closed 2 years ago

ubuntu1804 - forms/input/number/number-selected.html == forms/input/number/number-selected-ref.html | image comparison, max difference: 181, number of differing pixels: 722

Categories

(Core :: Layout: Form Controls, defect, P5)

defect

Tracking

()

RESOLVED FIXED

People

(Reporter: intermittent-bug-filer, Assigned: egao)

References

Details

(Keywords: leave-open)

Attachments

(1 file)

Filed by: egao [at] mozilla.com
Parsed log: https://treeherder.mozilla.org/logviewer.html#?job_id=271937747&repo=try
Full log: https://queue.taskcluster.net/v1/task/ZzSzlxrYRf6IPj2LVkHEXA/runs/0/artifacts/public/logs/live_backing.log
Reftest URL: https://hg.mozilla.org/mozilla-central/raw-file/tip/layout/tools/reftest/reftest-analyzer.xhtml#logurl=https://queue.taskcluster.net/v1/task/ZzSzlxrYRf6IPj2LVkHEXA/runs/0/artifacts/public/logs/live_backing.log&only_show_unexpected=1


platform: debian 10
suite: reftest-2

context:
[task 2019-10-18T17:27:38.486Z] 17:27:38 INFO - REFTEST TEST-START | file:///builds/worker/workspace/build/tests/reftest/tests/layout/reftests/forms/input/number/number-selected.html == file:///builds/worker/workspace/build/tests/reftest/tests/layout/reftests/forms/input/number/number-selected-ref.html
[task 2019-10-18T17:27:38.487Z] 17:27:38 INFO - REFTEST TEST-LOAD | file:///builds/worker/workspace/build/tests/reftest/tests/layout/reftests/forms/input/number/number-selected.html | 18 / 24 (75%)
[task 2019-10-18T17:27:38.587Z] 17:27:38 INFO - REFTEST TEST-LOAD | file:///builds/worker/workspace/build/tests/reftest/tests/layout/reftests/forms/input/number/number-selected-ref.html | 18 / 24 (75%)
[task 2019-10-18T17:27:38.689Z] 17:27:38 INFO - REFTEST INFO | REFTEST fuzzy test (0, 0) <= (202, 769) <= (1, 1)
[task 2019-10-18T17:27:38.789Z] 17:27:38 INFO - REFTEST TEST-UNEXPECTED-FAIL | file:///builds/worker/workspace/build/tests/reftest/tests/layout/reftests/forms/input/number/number-selected.html == file:///builds/worker/workspace/build/tests/reftest/tests/layout/reftests/forms/input/number/number-selected-ref.html | image comparison, max difference: 202, number of differing pixels: 769

(changing bug summary to reflect that debian10 is no longer being pursued, focus being shifted to ubuntu1804)

treeherder push: https://treeherder.mozilla.org/#/jobs?repo=try&group_state=expanded&revision=614122027e862bb723e83f495eaf6fd5bb267f55&selectedJob=279110151
log: https://firefoxci.taskcluster-artifacts.net/NgYisKYwRIuUNDn-Gzuiew/0/public/logs/live_backing.log
reftest analyzer: https://hg.mozilla.org/mozilla-central/raw-file/tip/layout/tools/reftest/reftest-analyzer.xhtml#logurl=https://firefox-ci-tc.services.mozilla.com/api/queue/v1/task/NgYisKYwRIuUNDn-Gzuiew/runs/0/artifacts/public/logs/live_backing.log&only_show_unexpected=1

[task 2019-12-02T17:45:30.229Z] 17:45:30     INFO - REFTEST TEST-START | file:///builds/worker/workspace/build/tests/reftest/tests/layout/reftests/forms/input/number/number-selected.html == file:///builds/worker/workspace/build/tests/reftest/tests/layout/reftests/forms/input/number/number-selected-ref.html
[task 2019-12-02T17:45:30.229Z] 17:45:30     INFO - REFTEST TEST-LOAD | file:///builds/worker/workspace/build/tests/reftest/tests/layout/reftests/forms/input/number/number-selected.html | 18 / 24 (75%)
[task 2019-12-02T17:45:30.375Z] 17:45:30     INFO - REFTEST TEST-LOAD | file:///builds/worker/workspace/build/tests/reftest/tests/layout/reftests/forms/input/number/number-selected-ref.html | 18 / 24 (75%)
[task 2019-12-02T17:45:30.521Z] 17:45:30     INFO - REFTEST INFO | REFTEST fuzzy test (0, 0) <= (181, 722) <= (1, 1)
[task 2019-12-02T17:45:30.622Z] 17:45:30     INFO - REFTEST TEST-UNEXPECTED-FAIL | file:///builds/worker/workspace/build/tests/reftest/tests/layout/reftests/forms/input/number/number-selected.html == file:///builds/worker/workspace/build/tests/reftest/tests/layout/reftests/forms/input/number/number-selected-ref.html | image comparison, max difference: 181, number of differing pixels: 722

:jwatt - this is a new failure that is consistently observed on ubuntu1804 test image. Would you be able to take a look, or is fuzzing the exected outcome for ubuntu1804 an acceptable way forward? To me, it looks like the new GNOME3-based desktop in ubuntu1804 has different input field sizing as the Unity-based ubuntu1604, which is something I've seen in other tests.

To run tests against the ubuntu1804 image, please use --ubuntu-bionic argument after ./mach try fuzzy.

Flags: needinfo?(jwatt)
Summary: Debian 10 - forms/input/number/number-selected.html == forms/input/number/number-selected-ref.html | image comparison, max difference: 202, number of differing pixels: 769 → ubuntu1804 - forms/input/number/number-selected.html == forms/input/number/number-selected-ref.html | image comparison, max difference: 202, number of differing pixels: 769
Summary: ubuntu1804 - forms/input/number/number-selected.html == forms/input/number/number-selected-ref.html | image comparison, max difference: 202, number of differing pixels: 769 → ubuntu1804 - forms/input/number/number-selected.html == forms/input/number/number-selected-ref.html | image comparison, max difference: 181, number of differing pixels: 722
Assignee: nobody → egao
Status: NEW → ASSIGNED

Intention after the ubuntu1804 migration completes is to tighten the pixel deviation range such that future deviations can be caught.

Pushed by egao@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/f9c8c2982d0f
add permissive fuzzy-if for number-selected.html for ubuntu1804 r=jmaher

Fixed in bug 1608582.

Status: ASSIGNED → RESOLVED
Closed: 2 years ago
Resolution: --- → FIXED
Flags: needinfo?(jwatt)
You need to log in before you can comment on or make changes to this bug.