1599640 - ubuntu1804 - mathml/subscript-italic-correction.html == mathml/subscript-italic-correction-ref.html | image comparison, max difference: 255, number of differing pixels: 776226

Status: RESOLVED FIXED

(Core :: MathML, defect, P5)

Description

[Treeherder Bug Filer]

Filed by: egao [at] mozilla.com
Parsed log: https://treeherder.mozilla.org/logviewer.html#?job_id=277707825&repo=try
Full log: https://firefox-ci-tc.services.mozilla.com/api/queue/v1/task/aL4x841UQhuAORh-FXi9sw/runs/0/artifacts/public/logs/live_backing.log
Reftest URL: https://hg.mozilla.org/mozilla-central/raw-file/tip/layout/tools/reftest/reftest-analyzer.xhtml#logurl=https://firefox-ci-tc.services.mozilla.com/api/queue/v1/task/aL4x841UQhuAORh-FXi9sw/runs/0/artifacts/public/logs/live_backing.log&only_show_unexpected=1

---

suite: reftest
chunk: 3

context:
[task 2019-11-22T19:58:25.529Z] 19:58:25 INFO - REFTEST TEST-START | file:///builds/worker/workspace/build/tests/reftest/tests/layout/reftests/mathml/subscript-italic-correction.html == file:///builds/worker/workspace/build/tests/reftest/tests/layout/reftests/mathml/subscript-italic-correction-ref.html
[task 2019-11-22T19:58:25.529Z] 19:58:25 INFO - REFTEST TEST-LOAD | file:///builds/worker/workspace/build/tests/reftest/tests/layout/reftests/mathml/subscript-italic-correction.html | 283 / 384 (73%)
[task 2019-11-22T19:58:25.612Z] 19:58:25 WARNING - REFTEST WARNING | [CONTENT] MozInvalidateEvent didn't invalidate
[task 2019-11-22T19:58:25.639Z] 19:58:25 INFO - REFTEST TEST-LOAD | file:///builds/worker/workspace/build/tests/reftest/tests/layout/reftests/mathml/subscript-italic-correction-ref.html | 283 / 384 (73%)
[task 2019-11-22T19:58:25.820Z] 19:58:25 INFO - REFTEST TEST-UNEXPECTED-FAIL | file:///builds/worker/workspace/build/tests/reftest/tests/layout/reftests/mathml/subscript-italic-correction.html == file:///builds/worker/workspace/build/tests/reftest/tests/layout/reftests/mathml/subscript-italic-correction-ref.html | image comparison, max difference: 255, number of differing pixels: 776226

Comment 1

[Daniel Holbert [:dholbert]]

This one looks like a real failure -- the testcase snapshot has a red background.

The test is set up such that it initially has a red background, and then does some verification in JS, and if everything checks out, then it uses JS to change the background to green.

So: the red background indicates that one of the test's JS comparisons failed. This behavior (vs. our older 16.04 testrunners) might be due to a difference in system fonts, I'm guessing. (One additional data point: the test passes on my local Ubuntu 18.10 machine. shrug.)

Anyway; probably not too worrisome, particularly given that the test already has known failures on Windows, per its reftest.list annotation. If we can use an Ubuntu 18.04-specific reftest variable, then that'd be great and we can annotate this as failing on that platform. But if we don't have a failure like that, then we can just annotate this as random-if(gtkWidget) I guess, given that we need to allow it to pass on some of our linux testrunners and fail on other linux testrunners.

Comment 2

[Edwin Takahashi (:egao | infrequent contributor)]

As noted in the other bug 1599638, I will put up a patch to mark this as fails-if(gtkWidget) once the full switchover to ubuntu1804 happens, just in case the random-if(gtkWidget) annotation masks any actual failures.

ni myself to have a reminder.

Comment 3

[Edwin Takahashi (:egao | infrequent contributor)]

Attachment: Bug 1599640 - add permissive fuzzy-if for subscript-italic-correction.html

Comment 4

[Edwin Takahashi (:egao | infrequent contributor)]

Temporarily marking the test with fuzzy-if so that ubuntu1604 and ubuntu1804 both work; marking this bug with leave-open so I will revisit later and mark this test with a fail-if accordingly.

Comment 5

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/918f0ea5f8bc
add permissive fuzzy-if for subscript-italic-correction.html r=jmaher

Comment 6

[Cristina Coroiu [:ccoroiu]]

https://hg.mozilla.org/mozilla-central/rev/918f0ea5f8bc

Comment 7

[Frédéric Wang (:fredw)]

I believe was correct, this is a real failure not a fuzzy failure. Maybe random failure, as that depends on the actual font used.

In bug 1882817 I'm converting it to a testharness test rather than a reftest. Ideally, to make the test more reliable we should use a custom font with a slanted glyph (but without italic correction provided in a MATH table)...

(In reply to Edwin Takahashi (:egao | infrequent contributor) from comment #4)

Temporarily marking the test with fuzzy-if so that ubuntu1604 and ubuntu1804 both work; marking this bug with leave-open so I will revisit later and mark this test with a fail-if accordingly.

It seems this was not marked as leave-open and the fuzzy was preserved. Anyway, will handle that in bug 1882817.