Closed Bug 1539212 Opened 5 years ago Closed 5 years ago

implement readability backplate for high contrast mode

Categories

(Core :: Layout, enhancement, P2)

Product:
Core
Component:
Layout
Type:
enhancement

Tracking

()

Status:
RESOLVED FIXED
Milestone:
mozilla71
Tracking Flags:
Tracking Status
relnote-firefox --- 73+
firefox70 - wontfix
firefox71 --- fixed

People

(Reporter: asa, Assigned: morgan)

References

(Depends on 1 open bug, Blocks 1 open bug, Regressed 1 open bug,
URL
)

Details

(Keywords: access)

Attachments

(9 files, 2 obsolete files)

testcase 1 (exercising a few scenarios with overlapping content & backgrounds)
1.13 KB, text/html
Details
screenshot of Edge rendering testcase 1 in High Contrast mode
36.77 KB, image/png
Details
testcase 2 (comparing relpos inline vs. inline-block)
544 bytes, text/html
Details
screenshot of testcase 2
32.15 KB, image/png
Details
backplate-bg-image-002.html
353 bytes, text/html
Details
nightly test.png
498.30 KB, image/png
Details
Screenshot (1).png
659.69 KB, image/png
Details
Bug 1539212: Add readability backplate for all elements containing text when HCM or a11y theme is active.
47 bytes, text/x-phabricator-request
Details | Review
Telemetry Questions
2.91 KB, text/plain
chutten
: data-review+
Details

Microsoft describes it here https://github.com/MicrosoftEdge/MSEdgeExplainers/blob/main/Accessibility/HighContrast/explainer.md

The goal of high contrast is to ensure a certain level of contrast between foreground and background colors. A problem arises with images. If text lies atop an image, altering the color of the text in high contrast will not guarantee its readability. One option would be to override images to allow text readability. This solution, however, is not an ideal one, as it can alter the context of a webpage for users under high contrast.

Instead, a preferred solution is to draw a so-called "readability backplate" behind all text to ensure contrast for text lying above images. As illustrated in the screenshots below, adding a backplate behind text in high contrast can drastically increase its readability. This solution is currently used in Microsoft Edge to ensure the readability of text in high contrast.

This backplate does not replace the background of an element, but rather is drawn on an intermediary layer:

As the diagram demonstrates, an element's text content is rendered using the WindowText system color and a backplate with a Window system color fill is drawn behind the text. These are then layered on top of the element's background (with background-color being filtered out). In the case of links, the text would instead use the appropriate high contrast link color.

URL: https://github.com/MicrosoftEdge/MSEd...
Priority: -- → P3
See Also: → 1506364

I looked into how this could be implemented today. I don't think it would be 100% straightforward, like adding a background to text frames. The problem is that if there is a paragraph with a line height of 1em or smaller, the running background will obscure part of the preceding line.

Blocks: 1266172
Depends on: 1575689

Here's a testcase to exercise a few interesting scenarios, to load in Edge and understand its behavior a bit better.

Here's a screenshot of Edge's take on testcase 1, in High Contrast mode. A few observations:

  • It is possible for a text-backplate to stomp on top of the text from another element, as shown by "ABSPOS TEXT" at top left (its backplate visibly stomps on the "relpos inline" text). This makes sense; it's just a good demonstration that they don't do "all text goes in front of all backplates, globally" or anything like that (which I was worried might be a possibility).
  • Looking at the lime background-image: Edge does not seem to render background images for display:inline elements (first box), though it does render them for display:inline-block elements (second box). (The background-image property still computes to the image URL, but it simply doesn't render.)
  • The background image on the relatively-positioned inline-block (second box) does cover up the text that follows it, which makes sense since it's positioned which means it draws in front. (This demonstrates that they don't e.g. draw all backplates in front of all backgrounds, or that sort of thing.)
  • WEIRD: the final box, with a negative z-index, seems to go "behind" the canvas entirely! It doesn't show up at all.

Also worth noting: when I turn off High-Contrast mode, Edge agrees with Firefox entirely.

As shown in this screenshot of testcase 2, with overlapping text:

  • A display:inline element seems to have its backplate drawn "cooperatively" with other text on the same line (since the text is overlapping but visible).
    ...vs:
  • A display:inline-block element's backplate seems to be drawn separately from (and potentially on top of) other text & inline-level content in the same line (since the text is overlapping and the inline-block's backplate visibly stomps on the text that it overlaps).
Attached file Bug 1539212 [wip] (obsolete) —

Depends on D43264

Attached image nightly test.png —

Weird "edge" case (haha...) on the test case I just uploaded. Have to post screenshot from my windows machine but here's the behaviour of that test in nightly with my current working patch:

Attached image Screenshot (1).png —

same test case on edge; note div height is different

[the above are mostly for me to note this is something I need to fix; my windows machine has about a billion updates it wants to do, so the screenshot is my reference lol]

Bug 1539212: Add readability backplate support for block elements containing text.

Bug 1539212 [wip]

Bug 1539212: Margin fixes.

Bug 1539212: Add readability support for all elements containing text when HCM or A11y Theme is active.

Attachment #9087767 - Attachment is obsolete: true
Attachment #9088212 - Attachment is obsolete: true

Hi Morgan, do you have someone to review this? I'm interested in getting this fix into 70 (for bug 1575220).

Assignee: nobody → mreschenberg
status-firefox70: --- → affected
tracking-firefox70: --- → +
Flags: needinfo?(mreschenberg)

Changing the priority to p2 as the bug is tracked by a release manager for the current nightly.
See What Do You Triage for more information

Priority: P3 → P2

(In reply to Liz Henry (:lizzard) from comment #14)

Hi Morgan, do you have someone to review this? I'm interested in getting this fix into 70 (for bug 1575220).

Hi! I have @dholbert and @eeejay set to review the layout and accessibility portions of this bug (respectively). There's a bit more work to be done re: creating a user-facing preference to enable or disable this feature, but hopefully the conversations (and work) around that portion will wrap up today and this weekend.

Flags: needinfo?(mreschenberg)

(In reply to Liz Henry (:lizzard) from comment #14)

Hi Morgan, do you have someone to review this? I'm interested in getting this fix into 70 (for bug 1575220).

Hi Liz, this is an ambitious feature that will need time in nightly, and will likely be preffed off for a cycle or two. I think they need to find another solution in bug 1575220 that does not use background images.

Hey Liz. As per comment 17, targeting this for 70 release is probably not feasible. Can we remove the tracking flag here? Thanks!

Flags: needinfo?(lhenry)

Yes, no problem. Thanks for the explanation!

status-firefox70: affected → wontfix
status-firefox71: --- → fix-optional
tracking-firefox70: + → -
Flags: needinfo?(lhenry)
Blocks: 1581583
Pushed by mreschenberg@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/f5b8fba989b4
Add readability backplate for all elements containing text when HCM or a11y theme is active. r=dholbert,mattwoodrow
Flags: needinfo?(mreschenberg)
Pushed by mreschenberg@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/928b755ef57b
Add readability backplate for all elements containing text when HCM or a11y theme is active. r=dholbert,mattwoodrow

Backed out changeset 928b755ef57b (Bug 1539212) for backplate-bg-image-006.html failures

Push with failures: https://treeherder.mozilla.org/#/jobs?repo=autoland&searchStr=android%2C7.0%2Cx86-64%2Copt%2Creftests%2Ctest-android-em-7.0-x86_64%2Fopt-geckoview-reftest-e10s-2%2Cr%28r2%29&fromchange=6fdb8149f1d594bb652682602970da228f6d28e9&tochange=d6f9a934bea2b6345d46a5fa4896bd0a75a54a34&selectedJob=267557697

Backout link: https://hg.mozilla.org/integration/autoland/rev/d6f9a934bea2b6345d46a5fa4896bd0a75a54a34

Failure log: https://treeherder.mozilla.org/logviewer.html#/jobs?job_id=267557697&repo=autoland&lineNumber=9851

[task 2019-09-20T00:29:09.796Z] 00:29:09 INFO - REFTEST TEST-START | http://10.0.2.2:8854/tests/layout/reftests/high-contrast/backplate-bg-image-006.html == http://10.0.2.2:8854/tests/layout/reftests/high-contrast/backplate-bg-image-006-ref.html
[task 2019-09-20T00:29:09.796Z] 00:29:09 INFO - REFTEST INFO | SET PREFERENCE pref(browser.display.document_color_use,2)
[task 2019-09-20T00:29:09.810Z] 00:29:09 INFO - REFTEST TEST-LOAD | http://10.0.2.2:8854/tests/layout/reftests/high-contrast/backplate-bg-image-006.html | 1560 / 7713 (20%)
[task 2019-09-20T00:29:09.812Z] 00:29:09 INFO - REFTEST INFO | RESTORE PREFERENCE pref(browser.display.document_color_use,0)
[task 2019-09-20T00:29:09.812Z] 00:29:09 INFO - REFTEST TEST-LOAD | http://10.0.2.2:8854/tests/layout/reftests/high-contrast/backplate-bg-image-006-ref.html | 1560 / 7713 (20%)
[task 2019-09-20T00:29:09.812Z] 00:29:09 INFO - REFTEST TEST-UNEXPECTED-PASS | http://10.0.2.2:8854/tests/layout/reftests/high-contrast/backplate-bg-image-006.html == http://10.0.2.2:8854/tests/layout/reftests/high-contrast/backplate-bg-image-006-ref.html | image comparison, max difference: 0, number of differing pixels: 0
[task 2019-09-20T00:29:09.818Z] 00:29:09 INFO - REFTEST IMAGE: data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAyAAAAPoCAYAAAAmy5qxAAAebklEQVR4nO3d...
[task 2019-09-20T00:29:09.818Z] 00:29:09 INFO - REFTEST INFO | Saved log: START http://10.0.2.2:8854/tests/layout/reftests/high-contrast/backplate-bg-image-006.html
[task 2019-09-20T00:29:09.818Z] 00:29:09 INFO - REFTEST INFO | Saved log: [CONTENT] OnDocumentLoad triggering AfterOnLoadScripts
[task 2019-09-20T00:29:09.818Z] 00:29:09 INFO - REFTEST INFO | Saved log: Initializing canvas snapshot
[task 2019-09-20T00:29:09.818Z] 00:29:09 INFO - REFTEST INFO | Saved log: DoDrawWindow 0,0,800,1000
[task 2019-09-20T00:29:09.819Z] 00:29:09 INFO - REFTEST INFO | Saved log: [CONTENT] RecordResult fired
[task 2019-09-20T00:29:09.819Z] 00:29:09 INFO - REFTEST INFO | Saved log: RecordResult fired
[task 2019-09-20T00:29:09.819Z] 00:29:09 INFO - REFTEST INFO | Saved log: START http://10.0.2.2:8854/tests/layout/reftests/high-contrast/backplate-bg-image-006-ref.html
[task 2019-09-20T00:29:09.819Z] 00:29:09 INFO - REFTEST INFO | Saved log: [CONTENT] OnDocumentLoad triggering AfterOnLoadScripts
[task 2019-09-20T00:29:09.819Z] 00:29:09 INFO - REFTEST INFO | Saved log: Initializing canvas snapshot
[task 2019-09-20T00:29:09.819Z] 00:29:09 INFO - REFTEST INFO | Saved log: DoDrawWindow 0,0,800,1000
[task 2019-09-20T00:29:09.819Z] 00:29:09 INFO - REFTEST INFO | Saved log: [CONTENT] RecordResult fired
[task 2019-09-20T00:29:09.819Z] 00:29:09 INFO - REFTEST INFO | Saved log: RecordResult fired
[task 2019-09-20T00:29:09.819Z] 00:29:09 INFO - REFTEST TEST-END | http://10.0.2.2:8854/tests/layout/reftests/high-contrast/backplate-bg-image-006.html == http://10.0.2.2:8854/tests/layout/reftests/high-contrast/backplate-bg-image-006-ref.html

Flags: needinfo?(mreschenberg)
Flags: needinfo?(mreschenberg)

More in-depth info about the reftest.list fuzzy annotations:

(1) Both Windows and Mac suffer from Ahem anti-aliasing, so I had to add fuzzy-if annotations for both platforms to the following tests:

(2) Native and non-native select elements have issues with visual overflow, and so do MacOS native buttons. On MacOS native button and select elements, extra visual overflow is added in the vertical and horizontal directions. For native select elements on Windows/Linux, and non-native firefox select elements, extra overflow is added in the horizontal direction only. The size does not scale with the number of <option>'s in any of the above cases; it looks like it has to do with some extra, widget-specific visual padding. We also have some differing pixels because I was unable to mock the select backplate in the reftest case. The following tests' fuzzy-if annotations are because of the visual overflow difference and select mocking:

  • Mac: 006, 008, 009 [native button, native select, non-native select]
  • Windows: 008, 009 [native select, non-native select]
  • Android: 008, 009 ["]
  • Linux: 008, 009 ["]
Attached file Telemetry Questions —
Attachment #9098291 - Flags: data-review?(chutten)
Comment on attachment 9098291 [details]
Telemetry Questions

DATA COLLECTION REVIEW RESPONSE:

    Is there or will there be documentation that describes the schema for the ultimate data set available publicly, complete and accurate?

Yes. This collection is Telemetry so is documented in its definitions file [Scalars.yaml](https://hg.mozilla.org/mozilla-central/file/tip/toolkit/components/telemetry/Scalars.yaml) and the [Probe Dictionary](https://telemetry.mozilla.org/probe-dictionary/).

    Is there a control mechanism that allows the user to turn the data collection on and off?

Yes. This collection is Telemetry so can be controlled through Firefox's Preferences.

    If the request is for permanent data collection, is there someone who will monitor the data over time?

No. This collection will expire in Firefox 73.

    Using the category system of data types on the Mozilla wiki, what collection type of data do the requested measurements fall under?

Category 2, Interaction. (Well, maybe it's Cat 1 Technical because it's a pref, but it's a user-set pref so... 2)

    Is the data collection request for default-on or default-off?

Default on for all channels.

    Does the instrumentation include the addition of any new identifiers?

No.

    Is the data collection covered by the existing Firefox privacy notice?

Yes.

    Does there need to be a check-in in the future to determine whether to renew the data?

Yes. :morgan is responsible for renewing or removing the collection before it expires in Firefox 73.

---
Result: datareview+
Attachment #9098291 - Flags: data-review?(chutten) → data-review+
Pushed by mreschenberg@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/6baef669e147
Add readability backplate for all elements containing text when HCM or a11y theme is active. r=dholbert,mattwoodrow,Jamie

https://hg.mozilla.org/mozilla-central/rev/6baef669e147

Status: NEW → RESOLVED
Closed: 5 years ago
status-firefox71: fix-optional → fixed
Resolution: --- → FIXED
Target Milestone: --- → mozilla71

In addition to fixing that error, the reftest.list should probably explicitly set the preference so that the tests will pass even when EARLY_BETA_OR_EARLIER isn't true.

Regressions: 1586577

(In reply to David Baron :dbaron: (away October 5-21) from comment #31)

In addition to fixing that error, the reftest.list should probably explicitly set the preference so that the tests will pass even when EARLY_BETA_OR_EARLIER isn't true.

Is explicitly setting the pref different from what the first few lines of the file do right now?

default-preferences test-pref(browser.display.document_color_use,2) test-pref(browser.display.permit_backplate,true)

https://searchfox.org/mozilla-central/rev/5cb522c7baba24e55874809e0e206b001494c1e9/layout/reftests/high-contrast/reftest.list#2

Flags: needinfo?(mreschenberg) → needinfo?(dbaron)

Looks like we really need value: @IS_EARLY_BETA_OR_EARLIER@ in the .yaml file (we're currently missing the "IS_").

And RE the prefs in reftest.list: dbaron is out of office, so he probably won't see your request right away -- but I believe the existing default-preferences line is sufficient & equivalent to setting the pref for each test. dbaron probably just didn't see it at first glance.

We'll find out for sure once we land the other change to unbreak late-beta simulation builds, anyway. :) If the default-preferences annotation is sufficient (as I think it should be), then the tests will pass.

Flags: needinfo?(dbaron)
Flags: needinfo?(mreschenberg)

Landed follow-up patch on bug 1586577 :)

Flags: needinfo?(mreschenberg)
Regressions: 1586482
See Also: → 1587127
Keywords: access
Regressions: 1591431
Regressions: 1593449
Depends on: 1593267
Regressions: 1595122
Regressions: 1604832
No longer regressions: 1604832
Blocks: 1604832
Depends on: 1608889

Release Note Request (optional, but appreciated)

[Why is this notable]:
Firefox high-contrast mode will now render background images. Additionally, all visible text will be backplated to ensure it meets accessibility standards for high-contrast and readability.

[Suggested wording]:
High-contrast mode has been updated to allow background images. To maintain readability and ensure sufficient contrast, visible text in high-contrast mode will be backplated by the theme's background color.

[Links (documentation, blog post, etc)]:
This bug! https://bugzilla.mozilla.org/show_bug.cgi?id=1539212
As with global zoom, this is a possible blog post topic for later, once the accessibility blog is up and running

[I did this in the wrong order, sorry for the double comment]

Release Note Request (optional, but appreciated)

[Why is this notable]:
Firefox high-contrast mode will now render background images. Additionally, all visible text will be backplated to ensure it meets accessibility standards for high-contrast and readability.

[Suggested wording]:
High-contrast mode has been updated to allow background images. To maintain readability and ensure sufficient contrast, visible text in high-contrast mode will be backplated by the theme's background color.

[Links (documentation, blog post, etc)]:
This bug! https://bugzilla.mozilla.org/show_bug.cgi?id=1539212
As with global zoom, this is a possible blog post topic for later, once the accessibility blog is up and running

relnote-firefox: --- → ?

This is being preffed on in 73 (shipping default on instead of default off, which was shipped 71, 72), so requesting a relnote for 73 :)

Added to the Fx73 relnotes.

relnote-firefox: ? → 73+
See Also: → 1634222
See Also: → 1645086
Regressions: 1707653
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Creator:
Created:
Updated:
Size: