1591611 - ubuntu1804 - forms/input/number/number-selected.html == forms/input/number/number-selected-ref.html | image comparison, max difference: 181, number of differing pixels: 722

Status: RESOLVED FIXED

(Core :: Layout: Form Controls, defect, P5)

Description

[Treeherder Bug Filer]

Filed by: egao [at] mozilla.com
Parsed log: https://treeherder.mozilla.org/logviewer.html#?job_id=271937747&repo=try
Full log: https://queue.taskcluster.net/v1/task/ZzSzlxrYRf6IPj2LVkHEXA/runs/0/artifacts/public/logs/live_backing.log
Reftest URL: https://hg.mozilla.org/mozilla-central/raw-file/tip/layout/tools/reftest/reftest-analyzer.xhtml#logurl=https://queue.taskcluster.net/v1/task/ZzSzlxrYRf6IPj2LVkHEXA/runs/0/artifacts/public/logs/live_backing.log&only_show_unexpected=1

---

platform: debian 10
suite: reftest-2

context:
[task 2019-10-18T17:27:38.486Z] 17:27:38 INFO - REFTEST TEST-START | file:///builds/worker/workspace/build/tests/reftest/tests/layout/reftests/forms/input/number/number-selected.html == file:///builds/worker/workspace/build/tests/reftest/tests/layout/reftests/forms/input/number/number-selected-ref.html
[task 2019-10-18T17:27:38.487Z] 17:27:38 INFO - REFTEST TEST-LOAD | file:///builds/worker/workspace/build/tests/reftest/tests/layout/reftests/forms/input/number/number-selected.html | 18 / 24 (75%)
[task 2019-10-18T17:27:38.587Z] 17:27:38 INFO - REFTEST TEST-LOAD | file:///builds/worker/workspace/build/tests/reftest/tests/layout/reftests/forms/input/number/number-selected-ref.html | 18 / 24 (75%)
[task 2019-10-18T17:27:38.689Z] 17:27:38 INFO - REFTEST INFO | REFTEST fuzzy test (0, 0) <= (202, 769) <= (1, 1)
[task 2019-10-18T17:27:38.789Z] 17:27:38 INFO - REFTEST TEST-UNEXPECTED-FAIL | file:///builds/worker/workspace/build/tests/reftest/tests/layout/reftests/forms/input/number/number-selected.html == file:///builds/worker/workspace/build/tests/reftest/tests/layout/reftests/forms/input/number/number-selected-ref.html | image comparison, max difference: 202, number of differing pixels: 769

Comment 1

[Edwin Takahashi (:egao | infrequent contributor)]

(changing bug summary to reflect that debian10 is no longer being pursued, focus being shifted to ubuntu1804)

treeherder push: https://treeherder.mozilla.org/#/jobs?repo=try&group_state=expanded&revision=614122027e862bb723e83f495eaf6fd5bb267f55&selectedJob=279110151
log: https://firefoxci.taskcluster-artifacts.net/NgYisKYwRIuUNDn-Gzuiew/0/public/logs/live_backing.log
reftest analyzer: https://hg.mozilla.org/mozilla-central/raw-file/tip/layout/tools/reftest/reftest-analyzer.xhtml#logurl=https://firefox-ci-tc.services.mozilla.com/api/queue/v1/task/NgYisKYwRIuUNDn-Gzuiew/runs/0/artifacts/public/logs/live_backing.log&only_show_unexpected=1

[task 2019-12-02T17:45:30.229Z] 17:45:30     INFO - REFTEST TEST-START | file:///builds/worker/workspace/build/tests/reftest/tests/layout/reftests/forms/input/number/number-selected.html == file:///builds/worker/workspace/build/tests/reftest/tests/layout/reftests/forms/input/number/number-selected-ref.html
[task 2019-12-02T17:45:30.229Z] 17:45:30     INFO - REFTEST TEST-LOAD | file:///builds/worker/workspace/build/tests/reftest/tests/layout/reftests/forms/input/number/number-selected.html | 18 / 24 (75%)
[task 2019-12-02T17:45:30.375Z] 17:45:30     INFO - REFTEST TEST-LOAD | file:///builds/worker/workspace/build/tests/reftest/tests/layout/reftests/forms/input/number/number-selected-ref.html | 18 / 24 (75%)
[task 2019-12-02T17:45:30.521Z] 17:45:30     INFO - REFTEST INFO | REFTEST fuzzy test (0, 0) <= (181, 722) <= (1, 1)
[task 2019-12-02T17:45:30.622Z] 17:45:30     INFO - REFTEST TEST-UNEXPECTED-FAIL | file:///builds/worker/workspace/build/tests/reftest/tests/layout/reftests/forms/input/number/number-selected.html == file:///builds/worker/workspace/build/tests/reftest/tests/layout/reftests/forms/input/number/number-selected-ref.html | image comparison, max difference: 181, number of differing pixels: 722

:jwatt - this is a new failure that is consistently observed on ubuntu1804 test image. Would you be able to take a look, or is fuzzing the exected outcome for ubuntu1804 an acceptable way forward? To me, it looks like the new GNOME3-based desktop in ubuntu1804 has different input field sizing as the Unity-based ubuntu1604, which is something I've seen in other tests.

To run tests against the ubuntu1804 image, please use --ubuntu-bionic argument after ./mach try fuzzy.

Comment 2

[Edwin Takahashi (:egao | infrequent contributor)]

Attachment: Bug 1591611 - add permissive fuzzy-if for number-selected.html for ubuntu1804

Comment 3

[Edwin Takahashi (:egao | infrequent contributor)]

Intention after the ubuntu1804 migration completes is to tighten the pixel deviation range such that future deviations can be caught.

Comment 4

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/f9c8c2982d0f
add permissive fuzzy-if for number-selected.html for ubuntu1804 r=jmaher

Comment 5

[Cristina Coroiu [:ccoroiu]]

https://hg.mozilla.org/mozilla-central/rev/f9c8c2982d0f

Comment 6

[Edwin Takahashi (:egao | infrequent contributor)]

Fixed in bug 1608582.