1599622 - ubuntu1804 - forms/input/radio/radio-clamp-02.html == forms/input/radio/radio-clamp-02-ref.html | image comparison, max difference: 24, number of differing pixels: 16

Status: RESOLVED FIXED

(Core :: Layout: Form Controls, defect, P5)

Description

[Treeherder Bug Filer]

Filed by: egao [at] mozilla.com
Parsed log: https://treeherder.mozilla.org/logviewer.html#?job_id=277707835&repo=try
Full log: https://firefox-ci-tc.services.mozilla.com/api/queue/v1/task/U8PqUNlgQYOJsYm0gCU91A/runs/0/artifacts/public/logs/live_backing.log
Reftest URL: https://hg.mozilla.org/mozilla-central/raw-file/tip/layout/tools/reftest/reftest-analyzer.xhtml#logurl=https://firefox-ci-tc.services.mozilla.com/api/queue/v1/task/U8PqUNlgQYOJsYm0gCU91A/runs/0/artifacts/public/logs/live_backing.log&only_show_unexpected=1

---

suite: reftest
chunk: 1

context:
[task 2019-11-22T19:45:03.723Z] 19:45:03 INFO - REFTEST TEST-START | file:///builds/worker/workspace/build/tests/reftest/tests/layout/reftests/forms/input/radio/radio-clamp-02.html == file:///builds/worker/workspace/build/tests/reftest/tests/layout/reftests/forms/input/radio/radio-clamp-02-ref.html
[task 2019-11-22T19:45:03.723Z] 19:45:03 INFO - REFTEST TEST-LOAD | file:///builds/worker/workspace/build/tests/reftest/tests/layout/reftests/forms/input/radio/radio-clamp-02.html | 8 / 10 (80%)
[task 2019-11-22T19:45:03.795Z] 19:45:03 INFO - REFTEST TEST-LOAD | file:///builds/worker/workspace/build/tests/reftest/tests/layout/reftests/forms/input/radio/radio-clamp-02-ref.html | 8 / 10 (80%)
[task 2019-11-22T19:45:03.965Z] 19:45:03 INFO - REFTEST TEST-UNEXPECTED-FAIL | file:///builds/worker/workspace/build/tests/reftest/tests/layout/reftests/forms/input/radio/radio-clamp-02.html == file:///builds/worker/workspace/build/tests/reftest/tests/layout/reftests/forms/input/radio/radio-clamp-02-ref.html | image comparison, max difference: 24, number of differing pixels: 16

Comment 1

[Edwin Takahashi (:egao | infrequent contributor)]

:svoisen - I am not sure who I can contact directly for this bug, as the person responsible is no longer with mozilla. It would be appreciated if you could pass the ni to someone that may be able to take a look. Thanks!

Comment 2

[Daniel Holbert [:dholbert]]

Mats (the reviewer on the original bug) is still active & would be the best authority here, but I'm happy to chime in too (and I suspect he'll agree with my thoughts below).

The failure here is trivial, and probably just wants a fuzzy annotation. Can you try adding fuzzy-if(gtkWidget,0-24,0-16) to this file's line in reftest.list? (right after its skip-if annotation) Sometimes multiple annotations for a single test can combine/override in non-intuitive ways, but I think these ones will combine the way you'd expect...

(The failure isn't entirely unexpected -- the test is already annotated as skipped on Windows and Mac, which seems to have been due to a pixel-rounding issue, per bug 1405986 comment 11. So there's probably a subtle pixel-rounding issue that's exposed in some widget themes, including the theme on Ubuntu 18.04.)

Comment 3

[Edwin Takahashi (:egao | infrequent contributor)]

I could do the fuzzy-if once I make the switchover to running these tests on ubuntu1804 for sure. My needinfo series were to ensure the teams that worked on the features/tests were made aware of the possible need to do so and make sure I have the agreement before doing anything. So thanks for getting back quickly :)

There are other tests (in other bugs that I've set ni on) that may require much larger fuzz; those are the ones I'm particularly concerned about.

Comment 4

[Daniel Holbert [:dholbert]]

OK.

For this particular case, fuzzy-if seems totally fine.

Comment 5

[Edwin Takahashi (:egao | infrequent contributor)]

I will put up a fuzz against this bug, thanks :dholbert!

Comment 6

[Edwin Takahashi (:egao | infrequent contributor)]

Attachment: Bug 1599622 - add permissive annotation for gtkWidget to radio-clamp-02.html test for ubuntu1804

Comment 7

[Edwin Takahashi (:egao | infrequent contributor)]

Try push with the fuzzy patch running ubuntu1804:
https://treeherder.mozilla.org/#/jobs?repo=try&group_state=expanded&revision=dd5d4433258d87f95740bbba4025c8b467d92cfc

[task 2019-11-27T21:42:28.432Z] 21:42:28     INFO - REFTEST TEST-START | file:///builds/worker/workspace/build/tests/reftest/tests/layout/reftests/forms/input/radio/radio-clamp-01.html == file:///builds/worker/workspace/build/tests/reftest/tests/layout/reftests/forms/input/radio/radio-clamp-01-ref.html
[task 2019-11-27T21:42:28.432Z] 21:42:28     INFO - REFTEST TEST-SKIP | file:///builds/worker/workspace/build/tests/reftest/tests/layout/reftests/forms/input/radio/radio-clamp-01.html == file:///builds/worker/workspace/build/tests/reftest/tests/layout/reftests/forms/input/radio/radio-clamp-01-ref.html | 
[task 2019-11-27T21:42:28.432Z] 21:42:28     INFO - REFTEST TEST-END | file:///builds/worker/workspace/build/tests/reftest/tests/layout/reftests/forms/input/radio/radio-clamp-01.html == file:///builds/worker/workspace/build/tests/reftest/tests/layout/reftests/forms/input/radio/radio-clamp-01-ref.html
[task 2019-11-27T21:42:28.433Z] 21:42:28     INFO - REFTEST TEST-START | file:///builds/worker/workspace/build/tests/reftest/tests/layout/reftests/forms/input/radio/radio-clamp-02.html == file:///builds/worker/workspace/build/tests/reftest/tests/layout/reftests/forms/input/radio/radio-clamp-02-ref.html
[task 2019-11-27T21:42:28.433Z] 21:42:28     INFO - REFTEST TEST-LOAD | file:///builds/worker/workspace/build/tests/reftest/tests/layout/reftests/forms/input/radio/radio-clamp-02.html | 8 / 10 (80%)
[task 2019-11-27T21:42:28.435Z] 21:42:28     INFO - [process-switch]: cannot find tabbrowser for loading tab - ignoring
[task 2019-11-27T21:42:28.499Z] 21:42:28     INFO - REFTEST TEST-LOAD | file:///builds/worker/workspace/build/tests/reftest/tests/layout/reftests/forms/input/radio/radio-clamp-02-ref.html | 8 / 10 (80%)
[task 2019-11-27T21:42:28.500Z] 21:42:28     INFO - [process-switch]: cannot find tabbrowser for loading tab - ignoring
[task 2019-11-27T21:42:28.557Z] 21:42:28     INFO - REFTEST INFO | REFTEST fuzzy test (24, 16) <= (24, 16) <= (24, 16)
[task 2019-11-27T21:42:28.557Z] 21:42:28     INFO - REFTEST TEST-PASS | file:///builds/worker/workspace/build/tests/reftest/tests/layout/reftests/forms/input/radio/radio-clamp-02.html == file:///builds/worker/workspace/build/tests/reftest/tests/layout/reftests/forms/input/radio/radio-clamp-02-ref.html | image comparison, max difference: 24, number of differing pixels: 16
[task 2019-11-27T21:42:28.559Z] 21:42:28     INFO - REFTEST TEST-END | file:///builds/worker/workspace/build/tests/reftest/tests/layout/reftests/forms/input/radio/radio-clamp-02.html == file:///builds/worker/workspace/build/tests/reftest/tests/layout/reftests/forms/input/radio/radio-clamp-02-ref.html

Comment 8

[Intermittent Failures Robot]

1 failures in 4867 pushes (0.0 failures/push) were associated with this bug in the last 7 days.

Repository breakdown:

• try: 1

Platform breakdown:

• linux64: 1

For more details, see:
https://treeherder.mozilla.org/intermittent-failures.html#/bugdetails?bug=1599622&startday=2019-12-09&endday=2019-12-15&tree=all

Comment 9

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/2aa9ace75336
add permissive annotation for gtkWidget to radio-clamp-02.html test for ubuntu1804 r=jmaher

Comment 10

[Cristina Coroiu [:ccoroiu]]

https://hg.mozilla.org/mozilla-central/rev/2aa9ace75336