Closed Bug 1843749 Opened 1 year ago Closed 1 year ago

Text rendering issues in Firefox 115 on Android

Categories

(Core :: Graphics: Text, defect)

Product:
Core
Component:
Graphics: Text
Platform:
Unspecified
Android
Type:
defect

Tracking

()

Status:
VERIFIED FIXED
Milestone:
117 Branch
Tracking Flags:
Tracking Status
firefox-esr102 --- unaffected
firefox-esr115 --- unaffected
firefox115 --- wontfix
firefox116 + verified
firefox117 + verified

People

(Reporter: cpeterson, Assigned: jnicol)

References

(Regression)

Details

(Keywords: regression)

Attachments

(2 files)

screenshot.png
180.89 KB, image/png
Details
Bug 1843749 - Block qcom_tiled_rendering on Adreno 308 GPUs. r?#gfx-reviewers
48 bytes, text/x-phabricator-request
diannaS
: approval-mozilla-release+
Details | Review
Attached image screenshot.png —

Users are reporting different scenarios where text is not displaying properly in Firefox 115 on Android. Given the number of reports now, this seems like a regression in 115. See the attached screenshot for examples of:

  • white bars covering text
  • text size smaller or larger, and can’t be adjusted
  • text displaying in a different font than intended

A user reports that a long press to highlight and select text will reveal missing text (blank space rendering).

SUMO forum threads with device info and more screenshots:

https://support.mozilla.org/en-US/questions/1417857
https://support.mozilla.org/en-US/questions/1418040
https://support.mozilla.org/en-US/questions/1417829
https://support.mozilla.org/en-US/questions/1418293

Affected devices mentioned:

Samsung Galaxy A42 5G
Samsung Galaxy Tab A, SM-T380 Android Tablet running Android 9
Samsung Galaxy Tab A8, model SM-T387V
Alcatel 1B 2020
Motorola G6 Play running Android 9: https://www.gsmarena.com/motorola_moto_g6_play-9002.php

The SUMO team's Trello support ticket for this issue:

https://trello.com/c/IQpJQrib/83-text-rendering-issues-in-firefox-for-android

Ashley Hale hypothesizes this is a GLES driver problem on Samsung Exynos-based hardware. She recommends users try installing Fenix Nightly and, if they can still reproduce the bug, use about:support to flip the gfx.webrender.software pref from false to true. That pref will avoid using the GLES hardware. It's not a bug fix, but it will help isolate where the problem is.

The user in https://support.mozilla.org/en-US/questions/1417857 reports that gfx.webrender.software = true fixes the problem (on their Samsung Galaxy Tab A (2017) SM-T380), so this looks like a GLES driver or hardware issue:

I ... installed the Nightly build of Firefox for Android ... and toggled gfx.webrender.software to true, which did not initially fix anything, but after restarting the app, pages started to render correctly with this change. I toggled it back to false and restarted the app again and the problem returned immediately.

(Restarting the app is necessary to pick up the new WebRender pref at app startup.)

Jamie, do you have a similarish device that could potentially reproduce?

Flags: needinfo?(jnicol)

QA, do you have any of these devices? If so, can you reproduce this bug? If you can reproduce the bug on one of these devices, you don't need to test others.

If you can reproduce the bug, does setting the gfx.webrender.software pref = true (in Nightly's about:config and force restarting the app) fix the bug for you (like the user in comment 2)?

Flags: qe-verify+

Hello, I am the user mentioned in comment #2 (https://bugzilla.mozilla.org/show_bug.cgi?id=1843749#c2) of this thread, experiencing this issue on a Samsung Galaxy Tab A SM-T380 Android Tablet running Android 9. If there's anything I can do to help narrow down this issue, please let me know.

Hi! With the devices available for us, we couldn't reproduce the issue presented above.
We started with the build this issue was originally reported (115.0.1) and checked the latest RC (115.2.1) as well.

The closest devices to the ones from the list were:

  • Samsung Galaxy Tab A6 (Android 5.1).
  • Motorola G6 (XT1925-5) (Android 8).
  • Samsung Galaxy Tab S3 (Android 9).

Also tried reproducing on:

  • Oppo Find X5 (Android 13).
  • Samsung Galaxy S22 Ultra (Android 13).
  • Pixel 7 (Android 14).
  • Pixel 7 Pro (Android 14).

Mostly we tried reproducing scenarios presented here trying to use the exact same pages as well, even raised the zoom level to 200% on a tablet (Samsung Galaxy Tab S3 with Android 9) to have the same arrangement as appears on the screenshots. Also different long text selections. All worked as expected.

Flags: qe-verify+

The bug has been present for multiple releases but got worse with 115 according to the first report. So maybe this isn't a regression? But it seems like something changed (in Firefox or the devices' OS?) around the 115 release because we received multiple reports of this bug within just a few days, but not before 115.

status-firefox116: --- → affected
status-firefox117: --- → affected

what I can see from looking at gsmarena and about:support where provided, these are the affected phones and chipsets:

What I see in common is Adreno 308 GPU, the SoC is one of Qualcomm MSM8917 Snapdragon 425, Qualcomm QM215 Snapdragon 215, Qualcomm MSM8920 Snapdragon 427.

Here are the driver versions:

  • OpenGL ES 3.0 V@415.0 (GIT@3c77002, I1807e7e5a1, 1609249121) (Date:12/29/20)
  • OpenGL ES 3.0 V@331.0 (GIT@054a40f, I67e1628f4e) (Date:05/18/19)
  • OpenGL ES 3.0 V@415.0 (GIT@7eb20a8, I89e02c0951, 1589413919) (Date:05/13/20)

(In reply to Chris Peterson [:cpeterson] from comment #7)

The bug has been present for multiple releases but got worse with 115 according to the first report. So maybe this isn't a regression? But it seems like something changed (in Firefox or the devices' OS?) around the 115 release because we received multiple reports of this bug within just a few days, but not before 115.

Yes, the bug would only occasionally show up on seemingly random (but text-heavy) webpages, such as Wikipedia articles, but it happened so infrequently, and affected the pages so little (usually highlighting some text on a page would render whatever text was invisible/cropped), that it never really bothered me much to file any report unfortunately. I should have reported this sooner as that would have likely made this problem easier to pinpoint.

For what it's worth, the last Samsung security update to my Galaxy Tab A (2017) SM-T380 was released on December 1, 2020, but I am not sure if this is considered an OS update or not.

The problem became glaring with Firefox 115.0.1. I've checked news.ycombinator.com every morning for a couple years now, and once I did that on this particular version, the page was just missing huge portions of text, which never happened as egregiously before. So far I've been using the Nightly build with gfx.webrender.software on just so I can browse sites fairly normally.

See Also: → 1831136

yungtarp99, if you have experience using Android adb and command line tools and you can reproduce the bug reliably, you can try using Mozilla's mozregression Python script to install and test GeckoView Nightly builds to pinpoint the first broken build.

mozregression --app gve --good 114

"gve" stands for GeckoView Example, a test browser that uses the same Gecko engine as Firefox and will probably have the same text rendering bug. mozregression will download GeckoView builds and ask you which are "good" and "bad" until it locates the first bad build.

https://mozilla.github.io/mozregression/install.html

https://mozilla.github.io/mozregression/documentation/usage.html

I'm not familiar with adb or command line use, unfortunately.

As an alternative, is there perhaps a way to downgrade Firefox/Nightly builds or an official archive of old Android APK builds of Firefox that I could download and install on the affected device itself to essentially replicate this process? I don't really care about losing my profiles or configuration.

You can manually download and install Nightly Android builds from https://archive.mozilla.org/pub/fenix/nightly/2023/, your device can use the ones labelled arm64-v8a. Downgrading requires uninstalling the existing version first, so it may be faster to test versions going forwards in small steps rather than jumping backwards and forwards in a typical bisect.

(In reply to Kestrel from comment #12)

You can manually download and install Nightly Android builds from https://archive.mozilla.org/pub/fenix/nightly/2023/, your device can use the ones labelled arm64-v8a. Downgrading requires uninstalling the existing version first, so it may be faster to test versions going forwards in small steps rather than jumping backwards and forwards in a typical bisect.

I downloaded the 103.0.0 Nightly Android arm64-v8a APK and attempted to install it (with Installing unknown programs being set to 'allowed' within My Files), it launches, asks me if I want to install and says that the program requires no special permissions, I accept, the progress bar appears and gets almost to the end, then it says the App was not installed and the APK is deleted off the tablet. I also tried this with the most recent 117.0.0 arm64-v8a APK in the archive, same result.

Any idea what could be blocking the installation? I don't have any antivirus on this phone, and I haven't had any problems installing other APKs before. I apologize, it feels like I'm just adding more problems into the equation here, but I am completely willing to go through however many builds to pinpoint exactly where this text rendering problem first appears if I can just figure out how to actually install these builds.

I want to clarify that I both tried installing over an existing installation of Firefox Nightly for Android, and again after completely uninstalling that installation, so a fresh install. Also, is there any way to edit comments on this site after they've been posted that I am overlooking?

Could you try the armeabi-v7a version instead of the arm64-v8a? It's possible your device is running a 32 bit OS on a 64 bit CPU, meaning the 64 bit firefox won't install.

Bug 1828248 jumps out from a quick scan of the changelog. I'm pretty sure I have an Adreno 308 device somewhere so will dig that out and see if I can reproduce.

I can reproduce on a Moto G6 Play, and mozregression confirms bug 1828248 is indeed the culprit.

We noticed during landing that bug that there was a driver bug on adreno driver version V@0490 on a Pixel 4, so we blocked the optimization on that driver version. Presumably this is the same or a similar bug, so we should additionally block on the affected driver versions.

Thank you for summarizing the affected devices, Ashley. It looks like we need to block V@331 and V@415. Chris, is the list of SUMO threads linked in comment 0 exhaustive? Or have we compiled all the about:supports that have been provided anywhere? I want to double check that we haven't missed an affected driver version.

Flags: needinfo?(jnicol) → needinfo?(cpeterson)
Keywords: regressionwindow-wanted
Regressed by: 1828248

Set release status flags based on info from the regressing bug 1828248

status-firefox-esr102: --- → unaffected
status-firefox-esr115: --- → affected

Interestingly while the previous bug with QCOM_tiled_rendering we discovered affected a variety of GPU models on driver version V@0490, that doesn't appear to be the case here. My Moto G7 Play (as opposed to the G6 mentioned in comment 16) is an Adreno 506, with driver version V@415. This driver version matches some of the reports we've received on Adreno 308. Therefore this bug appears to be specific to Adreno 308 rather than only dependent on the driver version.

Since it's nice to have this optimization, particularly on low powered GPUs, I think we should only block the affected versions on the 308.

Assignee: nobody → jnicol

We started using this extension as an optimization on Adreno GPUs in
bug 1828248. At the time, we discovered that there was a driver bug in
version 0490 of the Adreno driver resulting in rendering errors on a
variety of Adreno GPUs. We therefore blocked the usage of the
extension on that driver version.

We have now discovered another bug causing even more apparent
rendering issues. This appears to only affect Adreno 308 GPUs running
driver versions 331 or 415. This patch therefore additionally blocks
the extension on such devices.

Pushed by jnicol@mozilla.com: https://hg.mozilla.org/integration/autoland/rev/c33ea5e7fd57 Block qcom_tiled_rendering on Adreno 308 GPUs. r=gfx-reviewers,lsalzman

Comment on attachment 9345393 [details]
Bug 1843749 - Block qcom_tiled_rendering on Adreno 308 GPUs. r?#gfx-reviewers

Beta/Release Uplift Approval Request

  • User impact if declined: Faulty rendering on Adreno 308 devices
  • Is this code covered by automated tests?: No
  • Has the fix been verified in Nightly?: No
  • Needs manual test from QE?: No
  • If yes, steps to reproduce:
  • List of other uplifts needed: None
  • Risk to taking this patch: Low
  • Why is the change risky/not risky? (and alternatives if risky): Disables an optimization on a certain GPU. Most devices run without the optimization enabled so we are reverting to the better tested path.
  • String changes made/needed:
  • Is Android affected?: Yes
Attachment #9345393 - Flags: approval-mozilla-release?

Comment on attachment 9345393 [details]
Bug 1843749 - Block qcom_tiled_rendering on Adreno 308 GPUs. r?#gfx-reviewers

Approved for 116.0rc1

Attachment #9345393 - Flags: approval-mozilla-release? → approval-mozilla-release+
Pushed by nerli@mozilla.com: https://hg.mozilla.org/mozilla-central/rev/12931a93e28c Block qcom_tiled_rendering on Adreno 308 GPUs. r=gfx-reviewers,lsalzman

https://hg.mozilla.org/mozilla-central/rev/12931a93e28c

Status: NEW → RESOLVED
Closed: 1 year ago
status-firefox117: affected → fixed
Resolution: --- → FIXED
Target Milestone: --- → 117 Branch

We still couldn't reproduce the issue on 115, after a second try.
We did verify as fixed on the latest RC 116.0 and latest Nightly build (117.0a1 from 2023-07-25) as well.

Devices used:

  • Xiaomi mi4i (Android 5.0.2).
  • Samsung Galaxy Tab A6 (Android 5.1).
  • LG G7 fit (Android 8.1.0).
  • Samsung Galaxy Tab S3 (Android 9).
  • Motorola Moto G30 (Android 12).
  • Oppo Find X5 (Android 13).
  • Samsung Galaxy S22 Ultra (Android 13).
  • Pixel 7 (Android 14).

Marking the ticket as Verified for both 116 and 117.

Status: RESOLVED → VERIFIED

(In reply to Jamie Nicol [:jnicol] from comment #16)

Thank you for summarizing the affected devices, Ashley. It looks like we need to block V@331 and V@415. Chris, is the list of SUMO threads linked in comment 0 exhaustive? Or have we compiled all the about:supports that have been provided anywhere? I want to double check that we haven't missed an affected driver version.

No other devices have been reported as affected by the bug.

Flags: needinfo?(cpeterson)

Excellent, thanks. Hopefully we've got them all, but let's just keep our eyes peeled then in case we receive any further reports, it will be trivial to add more devices/driver versions to the fix.

One last thing is that some of the reports mention an incorrect font being used. That wouldn't be caused by this issue. Do we think that might just be users misdiagnosing what they see? Or perhaps it's a separate bug. Perhaps we should follow up with those users after 116 has been rolled out.

Depends on: 1847319
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Creator:
Created:
Updated:
Size: