1593059 - gnome3 - investigate font difference

Status: NEW

(Firefox Build System :: Task Configuration, task)

Description

[Edwin Takahashi (:egao | infrequent contributor)]

Investigate if the differences in various Reftest and Mochitest failures are due to font differences between Debian 10 vs Ubuntu 16.04.

Comment 1

[Edwin Takahashi (:egao | infrequent contributor)]

Try push:
https://treeherder.mozilla.org/#/jobs?repo=try&group_state=expanded&revision=8defe58b055a6cb334b247785d878e9a6e2c6fc3

From what I have investigated, it seems that Ubuntu 16.04 uses the default font of Ubuntu 11.
In Debian 10 under GNOME, it seems to use Sans 11.

Fortunately it is possible to install the Ubuntu font on debian 10, but as the try push above shows it doesn't seem to resolve the pixel differences in reftests.

Comment 2

[Edwin Takahashi (:egao | infrequent contributor)]

This is my previous push:
https://treeherder.mozilla.org/#/jobs?repo=try&group_state=expanded&revision=8defe58b055a6cb334b247785d878e9a6e2c6fc3

Changes in the revision:

• install Ubuntu font
• set the following fonts:

gsettings set org.gnome.desktop.interface document-font-name 'Sans 11'
gsettings set org.gnome.desktop.interface font-name 'Ubuntu 11'
gsettings set org.gnome.desktop.interface monospace-font-name 'Ubuntu Mono 13'

Reftest continues to have identical failures as the reference result.

Comment 3

[Tracy Walker [:tracy]]

This has severely impacted our Iris tests on Linux for Nightly validation. It has been a painstaking journey to make our Ubuntu 16.04 based tests function correctly in Debian 10. Going forward, if the Iris team is asked to expand the number of tests significantly, it would be ideal if this font rendering issue could be addressed.

Comment 4

[Edwin Takahashi (:egao | infrequent contributor)]

:tracy - would you happen to know something about fonts and rendering on linux systems? I'm not familiar with Iris, but I wonder if we've both trying to solve the same issues.

Comment 5

[Tracy Walker [:tracy]]

:egeo. No, I don't know the under-the-hood happenings here. We've worked around the problem with Iris. We are able to adjust by pattern manipulation and pattern match sensitivity to make the tests work, albeit precariously on the verge of failure.

Comment 6

[Edwin Takahashi (:egao | infrequent contributor)]

:tracy - I think I have been able to narrow down the cause of the font rendering difference.

I'm going to use Reftest as an example:
https://hg.mozilla.org/mozilla-central/raw-file/tip/layout/tools/reftest/reftest-analyzer.xhtml#logurl=https://firefox-ci-tc.services.mozilla.com/api/queue/v1/task/FsiI11RDQfuF4FjPAK0slQ/runs/0/artifacts/public/logs/live_backing.log&only_show_unexpected=1

Using the cyclone emoji reference at emojipedia for example:

• reftest reference image uses Google emoji
• reftest actual image from Debian uses Twitter emoji

Now, the same thing is observed with Ubuntu 18.04:
https://hg.mozilla.org/mozilla-central/raw-file/tip/layout/tools/reftest/reftest-analyzer.xhtml#logurl=https://firefox-ci-tc.services.mozilla.com/api/queue/v1/task/GpEdx_sbSkO_OZgvhZTljA/runs/0/artifacts/public/logs/live_backing.log&only_show_unexpected=1the

Ubuntu 18.04 has a subset of failures observed on Debian 10, so it seems some of the issues are common. What is common between these two distributions is GNOME shell - something I suspect is at least partially responsible for the issues with fonts.

Now, with that said, I have now decided to abandon Debian 10 image, instead using Ubuntu 18.04 (and 20.04 in the future). I understand that :KWierso stood up some tests to run based off Debian 10; would this change in direction affect your tests? Would you prefer that I leave the Debian10-Iris image as is until your team is ready to move to Ubuntu? Note that I do not intend to remove the Debian 10 image from the tree for the time being.

Comment 7

[Tracy Walker [:tracy]]

We have discovered that the failures we have been seeing (also affecting tests outside of task cluster) is a change in Firefox font rendering (I believe some Webrender changes in Fx71.)

That said, if Ubuntu were to be available for task cluster, we could move to that (our tests are based on 16.04). I haven't tried running our tests in Ubuntu 18.04. Anyway, for now, what we're running is working in the Debian 10 setup.