Closed Bug 1230366 Opened 9 years ago Closed 8 years ago

Skia Text is thin on OS X

Categories

(Core :: Graphics: Text, defect)

40 Branch
x86
macOS
defect
Not set
normal

Tracking

()

VERIFIED FIXED
mozilla48
Tracking Status
firefox48 --- verified

People

(Reporter: mchang, Assigned: mchang)

References

Details

(Whiteboard: gfx-noted)

Attachments

(12 files)

The chrome font for the URL bar and tab title bars are thin compared to the other browsers.
I'm not sure "thin" text is your problem here. Looks like integer pixel placement, note the awful placement of glyphs for "reddit". Can you confirm that subpixel placement is allowed/enabled for the Skia backend?

Can you include steps to reproduce this, build flags/contents/whatever?
(In reply to John Daggett (:jtd) from comment #2)
> I'm not sure "thin" text is your problem here. Looks like integer pixel
> placement, note the awful placement of glyphs for "reddit". Can you confirm
> that subpixel placement is allowed/enabled for the Skia backend?
> 
> Can you include steps to reproduce this, build flags/contents/whatever?

You can go to about:config, and change the preference "gfx.content.azure.backends" to "skia". I know that on an updated skia version, Skia rounds, so it may not support subpixel placement? I'll dig. Thanks for the tip!
Thanks for the tip to look for subpixel placement! That seemed to have fixed another bug :). This is a picture with explicit subpixel text allowed. It looks a bit better, but still a bit thin. The actual content on reddit.com looks fine, it's just the tab / title bar. Any other ideas on what to look for?
Flags: needinfo?(jdaggett)
Depends on: 1230357
Hmmm, hard to judge from such a small sample. Can you try this waterfall testcase and attach the screenshots from CG vs. Skia?
Flags: needinfo?(jdaggett) → needinfo?(mchang)
Flags: needinfo?(mchang)
Attached image waterfall cg rendering
Actually with these pictures, I can barely tell the difference. CG still seems a bit "thicker", but only while toggling between them in preview. Do you see anything specific here? Thanks for your help!
Flags: needinfo?(jdaggett)
If you zoom in on the images and toggle between them, it's clear that they have very different rasterization of the glyphs, and in general the Skia version seems less consistently "solid". Seems like it's not doing as good a job of antialiasing, or something like that.

I notice the following in SkFontHost_mac.cpp:

http://hg.mozilla.org/mozilla-central/file/e02b17a2b5b8/gfx/skia/skia/src/ports/SkFontHost_mac.cpp#l783

The comment says "FIXME: lcd smoothed un-hinted rasterization unsupported", and my understanding is that "lcd smoothed un-hinted" would be what's normally used on OS X, so perhaps this is the problem?
(In reply to Jonathan Kew (:jfkthame) from comment #8)
> If you zoom in on the images and toggle between them, it's clear that they
> have very different rasterization of the glyphs, and in general the Skia
> version seems less consistently "solid". Seems like it's not doing as good a
> job of antialiasing, or something like that.
> 
> I notice the following in SkFontHost_mac.cpp:
> 
> http://hg.mozilla.org/mozilla-central/file/e02b17a2b5b8/gfx/skia/skia/src/
> ports/SkFontHost_mac.cpp#l783
> 
> The comment says "FIXME: lcd smoothed un-hinted rasterization unsupported",
> and my understanding is that "lcd smoothed un-hinted" would be what's
> normally used on OS X, so perhaps this is the problem?

I looked into this. We already explicitly disable hinting in this case - https://dxr.mozilla.org/mozilla-central/source/gfx/2d/DrawTargetSkia.cpp?from=DrawTargetSkia.cpp#609 

And I never saw that code being called. I'll take a look at the AA settings. Thanks for the hint!
I'm not sure this is actually a bug, but by design. We're already enabling AA and after browsing the web a bit, the font we use with Skia is equal to Chrome, which actually makes sense. Since Firefox and Safari use CG directly, they have a different font than Chrome on the same sites. With the Skia backend and AA enabled, everything looks like Chrome in terms of text, instead of Safari.
Chrome is not necessarily a good reference. We should be aiming to conform to the platform's standards for how fonts are rendered, and Chrome doesn't always do that.... e.g. it seems to like to disable subpixel AA on Retina screens, whereas the OS in general still uses it.

Comparing your CG and Skia waterfall screenshots, it seems clear to me that the Skia rendering is inferior; this is particularly obvious in the tab title and URL bar, but affects the content as well. The exception is the (very artificial) "iiiiiIIIIIiiiii..." testcase, where the Skia result seems more even in some cases; but overall for normal text the result is worse.

Whether that's a bug of some kind on our side, or an inherent flaw within Skia, it's still a bug, and unless it gets fixed, switching to skia will amount to a text-rendering regression. :(
Comparison of Mason's screenshots for just the tab/URL bars. Skia on top, CG on bottom.

In both cases, subpixel AA is being applied and the spacing/placement is identical. One thing you might want to look at is if there are any extra styles being applied here (e.g. text-shadow?) such that differences between the Skia/CG handling of those additional effects produces the subtle variation seen in the screenshots.
Flags: needinfo?(jdaggett)
Just an FYI,

I asked 10 people around mozlando on a retina display to see skia vs CG on some random webpages to see if they could tell a difference, and if so, what did they prefer.

6 people couldn't tell a difference.
3 preferred Skia.
1 could barely tell and chose CG.
After talking with Jonathan and a few others, we don't think there are major blockers from text rendering that would prevent us from switching to skia. There are a couple of issues before we can ship though:

1) Light text on dark backgrounds look too thick compared to CG. This may be due to using subpixel AA instead of grayscale AA, even when we've been given the hint from layout.
2) The spacing between characters is just slightly different compared to CG on non-retina displays. We may be able to tweak the hinting to make the spacing act more like CG.
3) Menus from right clicking have a blue background behind the text when the mouse is hovered over it. Apparently we use a secret OS X API with a transparent background to enable subpixel AA in these cases. 

Overall, I'd prefer to have as little, and preferably no, CG hacks / skia changes to ship this.
(In reply to Mason Chang [:mchang] from comment #14)
> 1) Light text on dark backgrounds look too thick compared to CG. This may be
> due to using subpixel AA instead of grayscale AA, even when we've been given
> the hint from layout.

Actually, it looks like we do render grayscale when the -moz-osx-font-smoothing:grayscale property is set; but we (or Skia?) are doing something wrong (using a bad gamma value?) such that the grayscale-smoothed glyphs look much too heavy in light-on-dark cases.

E.g. on arstechnica.com, the headings MAIN MENU, MY STORIES, are white-on-black. With subpixel AA, the difference between CG and Skia is barely visible on a Retina screen; but if you use the Inspector to add -moz-osx-font-smoothing:grayscale, the Skia rendering looks overly fat/blurry, while the CG rendering is thinner and visually crisper.

So both backends are respecting -moz-osx-font-smoothing in some way; but the Skia backend is handling the grayscale rendering poorly.
(In reply to Mason Chang [:mchang] from comment #14)

> 2) The spacing between characters is just slightly different compared to CG
> on non-retina displays. We may be able to tweak the hinting to make the
> spacing act more like CG.

Hmmm, not sure what you mean by this. There's no hinting done under OSX so I don't think there's a handle to tweak here. I think you probably want to figure out the set of glyphs and positions that are passed through to a draw call and then trace it back to where differences occur for the CG/Skia backends.
(In reply to John Daggett (:jtd) from comment #16)
> (In reply to Mason Chang [:mchang] from comment #14)
> 
> > 2) The spacing between characters is just slightly different compared to CG
> > on non-retina displays. We may be able to tweak the hinting to make the
> > spacing act more like CG.
> 
> Hmmm, not sure what you mean by this. There's no hinting done under OSX so I
> don't think there's a handle to tweak here. I think you probably want to
> figure out the set of glyphs and positions that are passed through to a draw
> call and then trace it back to where differences occur for the CG/Skia
> backends.

Sorry, I should've clarified. Skia has some hinting prefs that are options available when drawing glyphs. Since the last comment, changing the hinting prefs didn't seem to do anything. I do know that from layout -> DrawTarget, the glyph positions are the same to both CG / Skia backends. So you're right, we'll probably have to see if Skia changes the glyph positions. On CG, we give the positions to a CG native API, so it's difficult to see what CG is doing to them.
Nice little history from Chrome: http://lists.w3.org/Archives/Public/www-style/2012Oct/0109.html

On why this was added: https://dxr.mozilla.org/mozilla-central/source/gfx/skia/skia/src/ports/SkFontHost_mac.cpp?from=SkFontHost_mac.cpp#833

Which is causing us to enable CG Font Smoothing even on grayscale font smoothing, causing the large bulky text.
Couple of interesting notes:

1) I actually think, since we should act like CG, we should actually be disabling hinting on OS X, like what Jonathan said in comment 8. OS X doesn't actually enable hinting. Then we'd get the proper behavior with -moz-osx-font-smoothing. 
2) To properly mimic what we have with CG, we need to enable both AA + Smooth Font in Skia. This only happens if we actually we have a hint to skia + enable subpixel AA. What's interesting, is that on a retina macbook pro, the value at [1] is set to a normal hint, and thus evaluates to true, even if we set the hinting explicitly to no hinting in DrawTargetSkia. On a late 2013 iMac that is non-retina, the hinting value holds and we'll get proper grayscale AA, but won't enable font smoothing all the time. 
3) Even with font smoothing + AA, the font is still thin compared to Nightly on a non-retina iMac. On a retina macbook pro, the fonts look the same with font smoothing + AA as a stock Nightly.

Still digging.

[1] https://dxr.mozilla.org/mozilla-central/source/gfx/skia/skia/src/ports/SkFontHost_mac.cpp?from=SkFontHost_mac.cpp#1209
With an updated version of skia
Attached image Comparison with CG
I kind of think I found out what was going on. IT seems to happen from this Skia commit:

https://github.com/google/skia/commit/0584631f5254add2f85d94f788d9d6115509211e

Previously, skia would set the text size to be (1) and scale it up with a matrix. For example, if I set text to be 18px, the text scale matrix would be one that scales by 18x. [1]. This transform was used to create the CTFontRef used to make the mask to render the font.

In the new version of skia, the this is no longer happening. It sets the transform matrix to the identity matrix and ensure that we always get the same underlying font [4]. This was changed in Skia here - https://codereview.chromium.org/1344213004

So essentially, Skia would get a font via [2] and now it's fetching fonts like [3]. If I change the lines at [3] to the lines to [2], text looks more like CG again. I guess before we would generally select different underlying fonts depending on the size, whereas now we're selecting a different font than what we used to since we're trying to avoid choosing a different one based on the size?

[1] https://dxr.mozilla.org/mozilla-central/source/gfx/skia/skia/src/ports/SkFontHost_mac.cpp?case=true&from=SkFontHost_mac.cpp#811
[2] https://dxr.mozilla.org/mozilla-central/source/gfx/skia/skia/src/ports/SkFontHost_mac.cpp?case=true&from=SkFontHost_mac.cpp#752
[3] https://github.com/google/skia/blob/master/src/ports/SkFontHost_mac.cpp#L776
[4] https://github.com/google/skia/blob/master/src/ports/SkFontHost_mac.cpp#L700
(In reply to Mason Chang [:mchang] from comment #24)
> I kind of think I found out what was going on. IT seems to happen from this
> Skia commit:
> 
> https://github.com/google/skia/commit/
> 0584631f5254add2f85d94f788d9d6115509211e
> 
> Previously, skia would set the text size to be (1) and scale it up with a
> matrix. For example, if I set text to be 18px, the text scale matrix would
> be one that scales by 18x. [1]. This transform was used to create the
> CTFontRef used to make the mask to render the font.
> 
> In the new version of skia, the this is no longer happening. It sets the
> transform matrix to the identity matrix and ensure that we always get the
> same underlying font [4]. This was changed in Skia here -
> https://codereview.chromium.org/1344213004

First off, I would stick in some printf's and make sure you're dealing with the same font. Are you seeing differences in UI text but not in content? Are you running under 10.11? If that's the case, the UI uses the San Francisco family. Next, dump the glyphs with positions at the Draw call. Are they always the same or different?

That Skia code sure is a catalog of CoreText/CoreGraphics oddities!! Looks like it would make for some interesting bedtime reading.
(In reply to John Daggett (:jtd) from comment #25)
> (In reply to Mason Chang [:mchang] from comment #24)
> > I kind of think I found out what was going on. IT seems to happen from this
> > Skia commit:
> > 
> > https://github.com/google/skia/commit/
> > 0584631f5254add2f85d94f788d9d6115509211e
> > 
> > Previously, skia would set the text size to be (1) and scale it up with a
> > matrix. For example, if I set text to be 18px, the text scale matrix would
> > be one that scales by 18x. [1]. This transform was used to create the
> > CTFontRef used to make the mask to render the font.
> > 
> > In the new version of skia, the this is no longer happening. It sets the
> > transform matrix to the identity matrix and ensure that we always get the
> > same underlying font [4]. This was changed in Skia here -
> > https://codereview.chromium.org/1344213004
> 
> First off, I would stick in some printf's and make sure you're dealing with
> the same font. Are you seeing differences in UI text but not in content? Are
> you running under 10.11? If that's the case, the UI uses the San Francisco
> family. Next, dump the glyphs with positions at the Draw call. Are they
> always the same or different?

I wasn't sure how to tell if I'm getting exactly the same font. The CFFontRefs due to different font sizes. I was still digging to see where I can get the actual underlying font to check if they are the same.

I'm on 10.11. Is there something special to using the San Francisco font family? The findings from comment 24 are all on 10.11, so the font looks different, with both API calls, even if they are using the SF font family.

Also, the differences are both in UI text and in content as well. 

The glyph positions are always the same, as well as the width/heights, mask format, and weight.

> That Skia code sure is a catalog of CoreText/CoreGraphics oddities!! Looks
> like it would make for some interesting bedtime reading.

Perfect for the holidays lol.
Summary: Skia Chrome Text is thin on OS X → Skia Text is thin on OS X
(In reply to Mason Chang [:mchang] from comment #26)

> I wasn't sure how to tell if I'm getting exactly the same font. The
> CFFontRefs due to different font sizes. I was still digging to see where I
> can get the actual underlying font to check if they are the same.
> 
> I'm on 10.11. Is there something special to using the San Francisco font
> family? The findings from comment 24 are all on 10.11, so the font looks
> different, with both API calls, even if they are using the SF font family.

You can dump out font names using this:

NSString* psname = GetRealFamilyName((NSFont*) fontRef);
printf("fontname: %s\n", [psname UTF8String]);

Yeah, the actual font family used will depend upon the size of the font. SF Text is used for smaller sizes, SF Display for larger sizes.

> Also, the differences are both in UI text and in content as well. 
> 
> The glyph positions are always the same, as well as the width/heights, mask
> format, and weight.

So that sounds like it's something to do with the scaling/matrix setup itself and how that somehow interacts with the rasterization of glyphs.
(In reply to John Daggett (:jtd) from comment #27)
> (In reply to Mason Chang [:mchang] from comment #26)
> 
> > I wasn't sure how to tell if I'm getting exactly the same font. The
> > CFFontRefs due to different font sizes. I was still digging to see where I
> > can get the actual underlying font to check if they are the same.
> > 
> > I'm on 10.11. Is there something special to using the San Francisco font
> > family? The findings from comment 24 are all on 10.11, so the font looks
> > different, with both API calls, even if they are using the SF font family.
> 
> You can dump out font names using this:
> 
> NSString* psname = GetRealFamilyName((NSFont*) fontRef);
> printf("fontname: %s\n", [psname UTF8String]);
> 
> Yeah, the actual font family used will depend upon the size of the font. SF
> Text is used for smaller sizes, SF Display for larger sizes.
> 

Thanks! I just have a simple test page that defaults to Times, but explicitly set to 18px size. In this case, I'm getting the same underlying font name of "Times".

> > Also, the differences are both in UI text and in content as well. 
> > 
> > The glyph positions are always the same, as well as the width/heights, mask
> > format, and weight.
> 
> So that sounds like it's something to do with the scaling/matrix setup
> itself and how that somehow interacts with the rasterization of glyphs.

I think you'er right, and it comes down to how CG creates / renders the fonts. Previously, Skia would create fonts with 
CTFontCreateCopyWithAttributes, set a text size of 1, and a scale transform the size of the requested font. Now, Skia creates CG Fonts with CTFontCreateWithGraphicsFont, sets the text size to the requested size, and a null transform. This seems to be the crux of the rendering difference.

I also checked to ensure that we're font smoothing via CGContextSetShouldSmoothFonts(true). If this is disabled and we create fonts with CTFontCreateWithGraphicsFont, we get even thinner fonts, almost like they are fading into the background.
I think we've tracked this down. Our CG backend uses roughly the same API calls as the Skia backend, but the thickness differences are mostly due to gamma correction. Skia renders glyphs one glyph at a time. For each glyph, it calls into CG by rendering the text with white on black background. The resulting pixels have already been gamma corrected by CG.

Skia takes these pixels and inverts their gamma back to linear. [1] It then preblends[2] it with a custom gamma corrected value and stores these gamma corrected values as a mask. This mask is then blitted against other surfaces to render text. Thus whenever Skia renders text, when it masks against other backgrounds, it's already gamma corrected. It doesn't have to blend text linearly with other surfaces. This gamma correction is slightly different than what CG outputs and creates very thin looking text. The effect is most noticeable on non-retina displays.

If I explicitly disable the gamma correction and the preblending, draw white text on a black background, the skia text looks like CG text. However, even if we disabled gamma correction and pre-blending, we would still have slightly different text as Skia always renders with white text on black background, whereas CG would ideally properly linearly blend and then gamma correct.

Jeff or Lee, please feel free to correct anything I might have mis represented. 

I don't think we're going to fix this or block shipping Skia on. Chrome already ships with this code and people generally aren't abandoning Chrome over this. Resolving as WONTFIX.

[1] https://dxr.mozilla.org/mozilla-central/source/gfx/skia/skia/src/ports/SkFontHost_mac.cpp#1281 
[2] https://dxr.mozilla.org/mozilla-central/source/gfx/skia/skia/src/ports/SkFontHost_mac.cpp#1296
Status: NEW → RESOLVED
Closed: 8 years ago
Resolution: --- → WONTFIX
Maybe fix the gamma or keep CG around for specific things. Reopening this until we resolve what we want to od.
Status: RESOLVED → REOPENED
Resolution: WONTFIX → ---
Can you try this build with skia on your mountain lion notebook and see if it looks any different than CG? This just inverts Skia to draw black on white. - https://treeherder.mozilla.org/#/jobs?repo=try&revision=a513bd7cb316
Flags: needinfo?(jmuizelaar)
For those following along at home, we figured out what's going on here. On 10.11 CG is using different glyph masks depending on the luminance of the foreground color. It's not obvious what the difference between these masks is yet, but Skia should probably be modified to also use the different masks depending on the foreground color.
Flags: needinfo?(jmuizelaar)
(In reply to Jeff Muizelaar [:jrmuizel] from comment #34)
> For those following along at home, we figured out what's going on here. On
> 10.11 CG is using different glyph masks depending on the luminance of the
> foreground color. It's not obvious what the difference between these masks
> is yet, but Skia should probably be modified to also use the different masks
> depending on the foreground color.

Other relevant findings from investigations we were all doing (Jeff, Markus, Mason, and I), which I am just posting here so we have them initially documented somewhere:

This old list email here from Ben Wagner has some background on how Apple's font smoothing algorithm works in general: https://lists.w3.org/Archives/Public/www-style/2012Oct/0109.html

Mason found this article highlighting possible motivations for why Apple might be rendering light and dark text differently in the first place: http://www.lighterra.com/articles/macosxtextaabug/

There appears to be 2 differently weighted types of smoothing, one for light text, one for dark text. Markus was doing some testing based on varying the foreground color of the text, with the background color seemingly not being the determining factor, as to which of the smoothing weighting situations is chosen. The dark text seems to be more heavily dilated, whereas the light text is less heavily dilated, the motivations for which would seemingly be elaborated upon in the article Mason found.

We believe the necessary and sufficient condition to determine the switch between them is this:

Given foreground color R,G,B: If min(R,G,B) < 1/3 or R+G+B < 2, treat the text as dark text. Otherwise, treat it as light text.

We observed this switch when trying to animate text color from black to white, there is a single pop in the weighting of the text, strongly indicating it is only flipping between the two weights, based on this condition.

On older version of OS X (at least Jeff verified on 10.9), this different does not seem to be present, so if you render as white-on-black, decode the gamma, then invert it, it comes out to the same value as if you render black-on-white and decode the gamma. On 10.11, this was not the case due to switching described.
One last thing to note, since Skia renders its font masks internally using the white-on-black scenario, it is now activating this new feature, with the less heavily dilated version of the mask being used. Seemingly by switching Skia to render its masks as black-on-white, it makes everything look like how it used to and avoids running into this problem. However, just making this change alone would disable this new feature in situations where we are actually rendering dark text on a light background with the Skia background unless Skia is modified to support two sets of masks.
(In reply to Lee Salzman [:lsalzman] from comment #36)
> One last thing to note, since Skia renders its font masks internally using
> the white-on-black scenario, it is now activating this new feature, with the
> less heavily dilated version of the mask being used. Seemingly by switching
> Skia to render its masks as black-on-white, it makes everything look like
> how it used to and avoids running into this problem. However, just making
> this change alone would disable this new feature in situations where we are
> actually rendering dark text on a light background with the Skia background
> unless Skia is modified to support two sets of masks.

Sorry, I meant in situations where we render light text on a dark background, we would no longer get the benefit of Apple's new handling of smoothing unless we more extensively modify Skia.
Comment on attachment 8727525 [details] [diff] [review]
Render CG font with black on white

Review of attachment 8727525 [details] [diff] [review]:
-----------------------------------------------------------------

Ideally we would want to decide whether we want to implement the dual-mask behavior in Skia - maybe we do, maybe we don't. But for now, just to get us around regressions in the short-term, this seems like a necessary temporary step, since it at least allows the status quo to continue on.
Attachment #8727525 - Flags: review?(lsalzman) → review+
Add this to our skia repo if it lands :)
Flags: needinfo?(lsalzman)
https://hg.mozilla.org/mozilla-central/rev/7e0f3aadcdf6
Status: REOPENED → RESOLVED
Closed: 8 years ago8 years ago
Resolution: --- → FIXED
Target Milestone: --- → mozilla48
(In reply to Mason Chang [:mchang] from comment #41)
> Add this to our skia repo if it lands :)

Got it.
Flags: needinfo?(lsalzman)
Flags: qe-verify+
I verified this bug on 48.0b4 build1 (20160627144420) using
- Mac OS X 10.11.1
- Mac OS X 10.10.5
- Mac OS X 10.9.5
but I observed the differences introduced by the fix only on Mac OS X 10.11.1. Based on comment 35, I think this is expected, so I will set the flags accordingly.
Status: RESOLVED → VERIFIED
Flags: qe-verify+
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Created:
Updated:
Size: