Closed Bug 1600777 Opened 2 years ago Closed 2 years ago

ubuntu1804 - indic-shaping/gujarati-3b.html == indic-shaping/gujarati-3-ref.html | image comparison, max difference: 255, number of differing pixels: 46

Categories

(Core :: Layout: Text and Fonts, defect, P5)

defect

Tracking

()

RESOLVED FIXED

People

(Reporter: intermittent-bug-filer, Assigned: egao)

References

Details

Attachments

(3 files)

Filed by: egao [at] mozilla.com
Parsed log: https://treeherder.mozilla.org/logviewer.html#?job_id=279110151&repo=try
Full log: https://firefox-ci-tc.services.mozilla.com/api/queue/v1/task/NgYisKYwRIuUNDn-Gzuiew/runs/0/artifacts/public/logs/live_backing.log
Reftest URL: https://hg.mozilla.org/mozilla-central/raw-file/tip/layout/tools/reftest/reftest-analyzer.xhtml#logurl=https://firefox-ci-tc.services.mozilla.com/api/queue/v1/task/NgYisKYwRIuUNDn-Gzuiew/runs/0/artifacts/public/logs/live_backing.log&only_show_unexpected=1


suite: reftest
chunk: 2

context:
[task 2019-12-02T17:42:08.979Z] 17:42:08 INFO - REFTEST TEST-START | file:///builds/worker/workspace/build/tests/reftest/tests/layout/reftests/indic-shaping/gujarati-3b.html == file:///builds/worker/workspace/build/tests/reftest/tests/layout/reftests/indic-shaping/gujarati-3-ref.html
[task 2019-12-02T17:42:08.980Z] 17:42:08 INFO - REFTEST TEST-LOAD | file:///builds/worker/workspace/build/tests/reftest/tests/layout/reftests/indic-shaping/gujarati-3b.html | 10 / 23 (43%)
[task 2019-12-02T17:42:09.144Z] 17:42:09 INFO - REFTEST TEST-UNEXPECTED-FAIL | file:///builds/worker/workspace/build/tests/reftest/tests/layout/reftests/indic-shaping/gujarati-3b.html == file:///builds/worker/workspace/build/tests/reftest/tests/layout/reftests/indic-shaping/gujarati-3-ref.html | image comparison, max difference: 255, number of differing pixels: 46

:jfkthame - this is a new, consistent failure observed under ubuntu1804 test image.

I am not certain what caused this failure to crop up recently as the try push from last week does not have this issue.

Would you have someone that may be able to investigate this failure, or would it be acceptable to simply annotate this as fuzzy-if for ubuntu1804?

Flags: needinfo?(jfkthame)

(In reply to Edwin Takahashi (:egao, :etakahashi) from comment #1)

:jfkthame - this is a new, consistent failure observed under ubuntu1804 test image.

I am not certain what caused this failure to crop up recently as the try push from last week does not have this issue.

It looks like this failure did occur on that try push as well; it's in the R5 chunk there.

The failure implies that Gujarati script shaping is failing in an unexpected way, so it's not something we can just ignore. If you remove the color: white; property from both the testcase and the reference, we'd be able to see what's happening (it will definitely fail in that case, because the text is different, but the expectation is that the width should be the same; making the text visible may help clarify how that's going wrong).

Flags: needinfo?(jfkthame)

(In reply to Jonathan Kew (:jfkthame) from comment #2)

(In reply to Edwin Takahashi (:egao, :etakahashi) from comment #1)

:jfkthame - this is a new, consistent failure observed under ubuntu1804 test image.

I am not certain what caused this failure to crop up recently as the try push from last week does not have this issue.

It looks like this failure did occur on that try push as well; it's in the R5 chunk there.

My mistake. Guess it's something that's been failing for longer than I thought.

What I could do is in this case then, is to push to try with the specified property removed and run it on ubuntu1804.

The try push is here:
https://treeherder.mozilla.org/#/jobs?repo=try&revision=91773f9a66545963b9350a40ed136ab7841aed03

I will edit the comment with the results once the test runs complete.

Edit:

reftest analyzer: https://hg.mozilla.org/mozilla-central/raw-file/tip/layout/tools/reftest/reftest-analyzer.xhtml#logurl=https://firefox-ci-tc.services.mozilla.com/api/queue/v1/task/LSdtQDjzQ-enUeaHBC7f6g/runs/0/artifacts/public/logs/live_backing.log&only_show_unexpected=1

It looks like the TEST image has a stroke going diagonally right that the REFERENCE image lacks, and the right half of the TEST image is slightly offset to the right compared to the REFERENCE image (possibly related, not sure).

Image attached for record for when the logs are no longer available.

(In reply to Edwin Takahashi (:egao, :etakahashi) from comment #3)

It looks like the TEST image has a stroke going diagonally right that the REFERENCE image lacks,

That's expected, it is a deliberate difference between the test and reference text (and that's why the text is given color:white in the reftest, so that this won't cause a mismatch).

and the right half of the TEST image is slightly offset to the right compared to the REFERENCE image (possibly related, not sure).

This is the problem; the text is expected to have the same width in both cases.

I'm inclined to suspect this is a minor rounding issue at some level (it looks like a one-pixel discrepancy), but I'm not sure exactly where it's coming from.

(In reply to Jonathan Kew (:jfkthame) from comment #5)

I'm inclined to suspect this is a minor rounding issue at some level (it looks like a one-pixel discrepancy), but I'm not sure exactly where it's coming from.

Do you think it may have to do with the aliasing and/or font scaling options?

I have the side-by-side comparison of gnome tweak tool showing on ubuntu1604 and ubuntu1804 in VM, screenshot attached. Not sure if this difference could be partially responsible, or just different labelling of the options.

Those options look equivalent, they've just redesigned the UI of the tweak tool a bit.

I notice that the reftest screenshot shows no antialiasing at all, though, despite the system setting calling for RGBA. Not sure whether that's something we're deliberately doing to try and reduce fuzz in reftests, or why it's disabled here? If antialiasing were in use, I believe we'd also get subpixel glyph positioning, which might make any discrepancy much smaller.

Lee, do you know if we normally expect to use antialiasing in reftests on Linux, or what's going on here?

Flags: needinfo?(lsalzman)

Subpixel anti-aliasing is explicitly disabled on Linux infra.

Flags: needinfo?(lsalzman)

OK, I thought I recalled something like that. So then we're looking at pixel-snapped non-antialiased text, and we're seeing a one-pixel spacing discrepancy, though I can't see any reason for it: the test and reference text should have exactly the same glyph advances AFAICT.

Would it make sense for me to explicitly turn off anti-aliasing on the ubuntu1804 machine?

EDIT: pushed a try run with antialiasing turned off via gsettings, let's see what happens:
https://treeherder.mozilla.org/#/jobs?repo=try&group_state=expanded&selectedJob=280332678&revision=23bf012a81267d2c5cd9f7208003787f4bf693c8

EDIT 2: turning off aliasing didn't seem to do the trick.

Would it be acceptable if I temporarily marked this test with annotations that would enable both ubuntu1604 and ubuntu1804 to pass for the time being? Once the migration to ubuntu1804 is complete, I can tighten up the annotation to the exact diff/pixelcount values.

Assignee: nobody → egao
Attachment #9118104 - Attachment description: Bug 1600777 - annotate gujarati-3b.html as being fuzzy-if on ubuntu1804 → Bug 1600777 - annotate gujarati-3b.html with permissive fuzzy-if annotation on ubuntu1804
Status: NEW → ASSIGNED
QA Whiteboard: leave-open,
Keywords: leave-open
Pushed by egao@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/3c2625208e78
annotate gujarati-3b.html with permissive fuzzy-if annotation on ubuntu1804 r=jmaher

This failure has been effectively silenced on ubuntu1804, but the issue still remains and should be fixed.

Assignee: egao → nobody
Status: ASSIGNED → NEW
QA Whiteboard: leave-open,
Keywords: leave-open
Status: NEW → RESOLVED
Closed: 2 years ago
Resolution: --- → FIXED
You need to log in before you can comment on or make changes to this bug.