1247714 - Consider using a thicker font (larger font-weight) on OS X for Firefox's UI to compensate for rendering changes caused by skia switch

Status: RESOLVED WONTFIX

(Firefox :: Theme, defect)

Description

[Mason Chang [Inactive] [:mchang]]

With Skia's fonts being slighter thinner than CoreGraphics fonts, we may want to consider changing some of our UI fonts to be thicker / more visible. This is most visible on non-retina displays.

STR:
1) go to about:config
2) change the preference "gfx.content.azure.backends" to skia. This will become the default soon on OS X.
3) Reload Firefox
4) See tab font

Comment 1

[Mason Chang [Inactive] [:mchang]]

Attachment: Tab Font with CoreGraphics

Comment 2

[Mason Chang [Inactive] [:mchang]]

Attachment: Tab Font with Skia

Comment 3

[:Gijs (he/him)]

I'm assuming this is literally about changing the font-weight slightly, is that correct? Is there data about exactly how the new rendering is different, in the form of numbers rather than screenshots? :-)

Also, is the skia switch expected to happen for 47 and ride the trains? Is there a CSS media query that can be used to detect what we're using, so we don't end up with overly heavy fonts if the skia switch gets backed out?

Comment 4

[Mason Chang [Inactive] [:mchang]]

(In reply to :Gijs Kruitbosch from comment #3)
> I'm assuming this is literally about changing the font-weight slightly, is
> that correct? Is there data about exactly how the new rendering is
> different, in the form of numbers rather than screenshots? :-)

Correct. Hmm that's kind of hard, because it really depends on which font is being rendered, what kind of monitor you're on, and the background the font is being rendered on. Generally, the easiest way to see it is to look at a font rendered with Safari versus the same font rendered in Chrome. E.g. on Retina Displays, it's a lot less noticeable. See https://bugzilla.mozilla.org/show_bug.cgi?id=1230366#c30.

> 
> Also, is the skia switch expected to happen for 47 and ride the trains? Is
> there a CSS media query that can be used to detect what we're using, so we
> don't end up with overly heavy fonts if the skia switch gets backed out?

I don't think it'll happen in 47, or at least ride the trains. If we do turn it on in 47, it'll be nightly only until 48. I don't know of any media queries, do you know if the front end has special powers to read preferences? The proper preference is "gfx.content.azure.backends". Do you know how about:support is getting the value for "AzureContentBackend"?

Comment 5

[:Gijs (he/him)]

(In reply to Mason Chang [:mchang] from comment #4)
> (In reply to :Gijs Kruitbosch from comment #3)
> > I'm assuming this is literally about changing the font-weight slightly, is
> > that correct? Is there data about exactly how the new rendering is
> > different, in the form of numbers rather than screenshots? :-)
> 
> Correct. Hmm that's kind of hard, because it really depends on which font is
> being rendered, what kind of monitor you're on, and the background the font
> is being rendered on. Generally, the easiest way to see it is to look at a
> font rendered with Safari versus the same font rendered in Chrome. E.g. on
> Retina Displays, it's a lot less noticeable. See
> https://bugzilla.mozilla.org/show_bug.cgi?id=1230366#c30.

OK.

> I don't know of any media
> queries, do you know if the front end has special powers to read
> preferences? The proper preference is "gfx.content.azure.backends".

We can read prefs from js and set a class or attribute or whatever, I was just hoping there was a CSS-only way to do it.

Stephen, any chance you can look at what corrections you think we need here? It should be possible to flip the pref that Mason mentioned to skia and restart to see what the difference will be.

Comment 6

[John Daggett (:jtd)]

I really don't think this is a great idea. What's being called "CoreGraphics fonts" vs. "Skia fonts" here are simply the way the two API's *render* the same fonts. I think more effort should instead be put into having Skia match CoreGraphics, since that's by definition the standard for the platform. Jiggling the weights will be very tricky as the UI font varies across release and in the case of OSX 10.11 the way the system San Francisco family is rendered has a lot of tricky subtleties to it. Maintaining an explicit weight hack like this is going to be a real headache over the long run.

One other thing to note here is that recent versions of CoreText include more controls for tweaking the gamma value used for font rasterization, among other things. I think it would be better to look at what Webkit is doing and try to match those results within Skia code.

Comment 7

[:Gijs (he/him)]

(In reply to John Daggett (:jtd) from comment #6)
> I really don't think this is a great idea. What's being called "CoreGraphics
> fonts" vs. "Skia fonts" here are simply the way the two API's *render* the
> same fonts. I think more effort should instead be put into having Skia match
> CoreGraphics, since that's by definition the standard for the platform.
> Jiggling the weights will be very tricky as the UI font varies across
> release and in the case of OSX 10.11 the way the system San Francisco family
> is rendered has a lot of tricky subtleties to it. Maintaining an explicit
> weight hack like this is going to be a real headache over the long run.
> 
> One other thing to note here is that recent versions of CoreText include
> more controls for tweaking the gamma value used for font rasterization,
> among other things. I think it would be better to look at what Webkit is
> doing and try to match those results within Skia code.

I don't really have enough knowledge of either CoreText/CoreGraphics or Skia to comment usefully here. Obviously based on this description I would prefer a fix to our Skia rendering. However, it seems bug 1230366 was purposefully wontfixed. Was this not done after consultation with other gfx/font folks?

I'm going to leave needinfo for Stephen to see what he thinks about the differences either way.

Comment 8

[John Daggett (:jtd)]

(In reply to :Gijs Kruitbosch from comment #7)

> I don't really have enough knowledge of either CoreText/CoreGraphics or Skia
> to comment usefully here. Obviously based on this description I would prefer
> a fix to our Skia rendering. However, it seems bug 1230366 was purposefully
> wontfixed. Was this not done after consultation with other gfx/font folks?
> 
> I'm going to leave needinfo for Stephen to see what he thinks about the
> differences either way.

Right, Mason made the call that it wasn't important enough to fix. That's fine but I don't think it makes sense to try and hack around the difference by fiddling with the UI font weight.

Comment 9

[Stephen Horlander [:shorlander]]

(In reply to John Daggett (:jtd) from comment #8)
> (In reply to :Gijs Kruitbosch from comment #7)
> 
> > I don't really have enough knowledge of either CoreText/CoreGraphics or Skia
> > to comment usefully here. Obviously based on this description I would prefer
> > a fix to our Skia rendering. However, it seems bug 1230366 was purposefully
> > wontfixed. Was this not done after consultation with other gfx/font folks?
> > 
> > I'm going to leave needinfo for Stephen to see what he thinks about the
> > differences either way.
> 
> Right, Mason made the call that it wasn't important enough to fix. That's
> fine but I don't think it makes sense to try and hack around the difference
> by fiddling with the UI font weight.

I agree that it doesn't make sense to hack around a font rendering regression by trying to manually tweak font weights.

This approach doesn't address the general problem that this affects all weights and fonts. All fonts regardless of weight are blurrier and more indistinct. You can see it in the site identity panel where we use a few weights and sizes: http://cl.ly/021q0o0r013Q

It also won't address thinner fonts on websites where we can't hack around it: http://cl.ly/1h2y1Z0S2G3g

There are also some specific problems with trying to do this on OS X just in our UI:

• OS X does not have a good range of default font weights on all versions that we support
  - Lucida Grande only has two and Helvetica Neue and SF are pretty different

• There is no guarantee that the next weight up is going to be the right weight. 
  - E.g. the new Skia Regular weight might be too thin, but Medium too thick

• We also appear to lose sub-pixel AA in some places (Sheets, Panels, Menus)?

Comment 10

[:Gijs (he/him)]

(In reply to Stephen Horlander [:shorlander] from comment #9)
> (In reply to John Daggett (:jtd) from comment #8)
> > (In reply to :Gijs Kruitbosch from comment #7)
> > 
> > > I don't really have enough knowledge of either CoreText/CoreGraphics or Skia
> > > to comment usefully here. Obviously based on this description I would prefer
> > > a fix to our Skia rendering. However, it seems bug 1230366 was purposefully
> > > wontfixed. Was this not done after consultation with other gfx/font folks?
> > > 
> > > I'm going to leave needinfo for Stephen to see what he thinks about the
> > > differences either way.
> > 
> > Right, Mason made the call that it wasn't important enough to fix. That's
> > fine but I don't think it makes sense to try and hack around the difference
> > by fiddling with the UI font weight.
> 
> I agree that it doesn't make sense to hack around a font rendering
> regression by trying to manually tweak font weights.
> 
> This approach doesn't address the general problem that this affects all
> weights and fonts. All fonts regardless of weight are blurrier and more
> indistinct. You can see it in the site identity panel where we use a few
> weights and sizes: http://cl.ly/021q0o0r013Q
> 
> It also won't address thinner fonts on websites where we can't hack around
> it: http://cl.ly/1h2y1Z0S2G3g
> 
> There are also some specific problems with trying to do this on OS X just in
> our UI:
> 
> • OS X does not have a good range of default font weights on all versions
> that we support
>   - Lucida Grande only has two and Helvetica Neue and SF are pretty different
> 
> • There is no guarantee that the next weight up is going to be the right
> weight. 
>   - E.g. the new Skia Regular weight might be too thin, but Medium too thick
> 
> • We also appear to lose sub-pixel AA in some places (Sheets, Panels, Menus)?

At least some of these seem like they need to be fixed outside of Fx::Theme, and in our Skia rendering code instead. Mason, would you agree? Maybe some of this (sub-pixel AA and such) is filed elsewhere?

Comment 11

[Mason Chang [Inactive] [:mchang]]

(In reply to John Daggett (:jtd) from comment #8)
> (In reply to :Gijs Kruitbosch from comment #7)
> 
> > I don't really have enough knowledge of either CoreText/CoreGraphics or Skia
> > to comment usefully here. Obviously based on this description I would prefer
> > a fix to our Skia rendering. However, it seems bug 1230366 was purposefully
> > wontfixed. Was this not done after consultation with other gfx/font folks?
> > 
> > I'm going to leave needinfo for Stephen to see what he thinks about the
> > differences either way.
> 
> Right, Mason made the call that it wasn't important enough to fix. That's
> fine but I don't think it makes sense to try and hack around the difference
> by fiddling with the UI font weight.

I also don't think that without a big rewrite of Skia internals, we'd be able to fix it. With CG, we use OS X APIs for every call to render text. CG handles all the gamma and blending of the text with the background correctly for us, which is great. However, Skia tries to not rely on CG very much. It renders each glyph once with white on black, and tries to correct the gamma such that the glyph will work in all scenarios, and caches that. It's not ideal, but overall as a whole, Skia is up to 30% faster at rendering.

I do think that one of the consequences of moving to Skia will be that the UI looks too thin / fragile for many users not on a retina display. It's not the best solution, but I'm not against it. We already set fonts/UI to look "good" for the user, they were just dependent on how OS X rendered the font. I agree we're definitely losing the purity / some native look feel here. However, the user won't care we didn't fiddle with the fonts because we render them differently, they'll care that the UI is harder to read. I think this bug should be to just make it easier for the user here. But I'm not a UI designer, so I would be very happy to be wrong here :).

Comment 12

[Mason Chang [Inactive] [:mchang]]

(In reply to Stephen Horlander [:shorlander] from comment #9)
> (In reply to John Daggett (:jtd) from comment #8)
> > (In reply to :Gijs Kruitbosch from comment #7)
> > 
> > > I don't really have enough knowledge of either CoreText/CoreGraphics or Skia
> > > to comment usefully here. Obviously based on this description I would prefer
> > > a fix to our Skia rendering. However, it seems bug 1230366 was purposefully
> > > wontfixed. Was this not done after consultation with other gfx/font folks?
> > > 
> > > I'm going to leave needinfo for Stephen to see what he thinks about the
> > > differences either way.
> > 
> > Right, Mason made the call that it wasn't important enough to fix. That's
> > fine but I don't think it makes sense to try and hack around the difference
> > by fiddling with the UI font weight.
> 
> I agree that it doesn't make sense to hack around a font rendering
> regression by trying to manually tweak font weights.

I agree I don't like it very much either, but this is a font rendering regression we're probably going to have to take. Our fonts will look like Chrome, which doesn't have many users leaving due to this issue.

> 
> This approach doesn't address the general problem that this affects all
> weights and fonts. All fonts regardless of weight are blurrier and more
> indistinct. You can see it in the site identity panel where we use a few
> weights and sizes: http://cl.ly/021q0o0r013Q
>
> It also won't address thinner fonts on websites where we can't hack around
> it: http://cl.ly/1h2y1Z0S2G3g
>

Yes, instead our fonts will look like Chrome. We'll just look like Chrome instead of Safari. I think this is acceptable and users seem to be happy with Chrome on OS X as well.
 
> There are also some specific problems with trying to do this on OS X just in
> our UI:
> 
> • OS X does not have a good range of default font weights on all versions
> that we support
>   - Lucida Grande only has two and Helvetica Neue and SF are pretty different
> 

Is it possible to change fonts based on OSX versions?

> • There is no guarantee that the next weight up is going to be the right
> weight. 
>   - E.g. the new Skia Regular weight might be too thin, but Medium too thick

Can we explore if we can pick a font that would work?

> • We also appear to lose sub-pixel AA in some places (Sheets, Panels, Menus)?

Yes, fonts on transparent backgrounds lose sub-pixel AA because at the moment, we rely on a private OS X API added in 10.9 to draw these. See bug 1055622. We haven't added support for this yet in Skia. [1].

[1] https://dxr.mozilla.org/mozilla-central/source/gfx/2d/DrawTargetCG.cpp?case=true&from=DrawTargetCG.cpp#1516

Comment 13

[Mason Chang [Inactive] [:mchang]]

I also wanted to note that even with the CG backend, we don't always correctly render font the same way as Safari. See macrumors.com, the white bold text in front of the red background on each posts' title. We're slightly thicker than Safari and Chrome more closely resembles Safari.

Comment 14

[Mason Chang [Inactive] [:mchang]]

(In reply to John Daggett (:jtd) from comment #6)
> I really don't think this is a great idea. What's being called "CoreGraphics
> fonts" vs. "Skia fonts" here are simply the way the two API's *render* the
> same fonts. I think more effort should instead be put into having Skia match
> CoreGraphics, since that's by definition the standard for the platform.
> Jiggling the weights will be very tricky as the UI font varies across
> release and in the case of OSX 10.11 the way the system San Francisco family
> is rendered has a lot of tricky subtleties to it. Maintaining an explicit
> weight hack like this is going to be a real headache over the long run.

Sorry for my bad terminology. You're right, it's getting the same underlying font data, just rendering them differently. Skia renders each glyph through CG with white text on black background, then does some custom gamma correction, and caches that image to render the glyphs. I think the only way we could really fix this is if we rewrote large parts of Skia to stop caching glyphs or rewrite it to get new glyphs if the background color changed, which I'm not sure is worth it. Would it be better to just change the font to something slightly thicker in general? That means pre yosemite wouldn't look as good, but post it would?

> 
> One other thing to note here is that recent versions of CoreText include
> more controls for tweaking the gamma value used for font rasterization,
> among other things. I think it would be better to look at what Webkit is
> doing and try to match those results within Skia code.

I was looking through the CTFont APIs and couldn't find control gamma, either through attributes or traits. Google didn't bring up much neither did ADC searches for gamma. Sorry, can you please point to which API's you're thinking of? Thanks!

Comment 15

[Stephen Horlander [:shorlander]]

(In reply to Mason Chang [:mchang] from comment #12)
> I agree I don't like it very much either, but this is a font rendering
> regression we're probably going to have to take. Our fonts will look like
> Chrome, which doesn't have many users leaving due to this issue.> 

It's hard to draw any conclusions about what factors drive people away from a product without research. But it's worth noting that our fonts, without any adjustment, will look thinner and harder to read than Chrome's: http://cl.ly/1i0P452g281o


> Can we explore if we can pick a font that would work?

Changing the font-weight to 400 with Skia rendering produces roughly the same visual weight as it currently does unmodified with CG (CG behind, Skia in front): http://cl.ly/3D1Q3A120e0v/o

But it does have some undesirable side-effects (see purple arrows):
- The character width and/or tracking are wider creating overflow issues
- Copy and Paste in the menu seem heavier than Cut for some reason

This is only on 10.11 with San Francisco. Not sure how this will work with Helvetica Neue on 10.10 and it won't work at all with Lucida Grande on 10.9 and under.


> > • OS X does not have a good range of default font weights on all versions
> > that we support
> >   - Lucida Grande only has two and Helvetica Neue and SF are pretty different
> > 
> 
> Is it possible to change fonts based on OSX versions?

I think we have a media-query that we could use to specify different font-weights per platform. My bigger concern is that if Lucida Grande also appears thinner we have no recourse there.

Comment 16

[John Daggett (:jtd)]

I'm assuming everyone commenting here is referring to screenshots with gfx.azure.content.backends set to 'skia' (why the plural on backends?!?). If that's different from Chrome that should be the first thing to figure out.

If there's a difference that's enough to consider swizzling around font weights then I think it would be much better to fix the underlying problem in Skia, including reopening and fixing bug 1230366. Fiddling with font weights is going to be tricky to implement and hard to maintain. Many parts of our UI use system font keywords (i.e. font: menu) to explicitly set the weight/style based on underlying OS settings. You would also need to hack around that stuff which is kind of defeating the purpose of using those special UI font keywords.

In case this helps, I should also point out that you can figure out the exact font used for a given piece of UI text by enabling textrun logging for UI text:

  export NSPR_LOG_MODULES=textrunui:5
  export NSPR_LOG_FILE=/path/to/textrunui.log

This will log the exact font (i.e. Postscript name of the font) used for a given textrun.

Both Gecko and Webkit nightly implement support for the '-apple-system' font family keyword. Under OSX 10.11 this will pick up the appropriate San Francisco system UI font based on the size of the text, since 10.11 uses size-based font selection.

As for OSX system font API's, I tend to read through Webkit code since that will tell you where CoreText is heading in the way they use private SPI's. Not an ideal state of the world but that's life. I did notice that there where some things in there related to rasterization that appear to be in 10.11. Something to experiment with I think.

Comment 17

[Mason Chang [Inactive] [:mchang]]

Attachment: Font logging enabled

(In reply to John Daggett (:jtd) from comment #16)
> I'm assuming everyone commenting here is referring to screenshots with
> gfx.azure.content.backends set to 'skia' (why the plural on backends?!?).

Other platforms have more than one backend. Changing the pref to "skia" just forces skia as a way to test at the moment.

> If
> that's different from Chrome that should be the first thing to figure out.

Sorry, what is "that" here? Skia is the backend from Chrome. Content on Firefox with Skia looks like Content on Chrome. The UI can be different due to differnet font settings on Chrome's UI.

> 
> If there's a difference that's enough to consider swizzling around font
> weights then I think it would be much better to fix the underlying problem
> in Skia, including reopening and fixing bug 1230366.

Unfortunately, there's no real "fixing" Skia. Skia directly uses CoreGraphics via public APIs to render the actual glyph. Then it adjusts it so it can do pre-multiplied alpha. It also seems to use this gamma correction table on Windows as well, which fonts also look slightly different than Edge / IE (due to lots of other issues as well such as subpixel AA). It seems that Chrome specifically made these decisions, which creates a relatively consistent font across platforms. To really get Skia to look like Safari all the time would involve lots of rewriting of Skia and to disable their glyph cache and soley rely on CoreGraphics and render all text through CoreGraphics directly.

Doing such a task would inversely impact what we're trying to do wi th Skia. We're specifically looking to delete our CoreGraphics backend due to the maintenance cost of having another backend. We spend a disproportionate amount of engineering time on a platform only 2.4% of our users are on. [1]

> Fiddling with font
> weights is going to be tricky to implement and hard to maintain. Many parts
> of our UI use system font keywords (i.e. font: menu) to explicitly set the
> weight/style based on underlying OS settings. You would also need to hack
> around that stuff which is kind of defeating the purpose of using those
> special UI font keywords.

Sorry, I don't understand why we would have to hack around it? If we're already using weight/style based on the underlying OS settings, what's different? None of the OS settings would have to change. If we're already fiddling with the font weights/style based on the underlying OS, what's the difference?

> 
> In case this helps, I should also point out that you can figure out the
> exact font used for a given piece of UI text by enabling textrun logging for
> UI text:
> 
>   export NSPR_LOG_MODULES=textrunui:5
>   export NSPR_LOG_FILE=/path/to/textrunui.log
> 
> This will log the exact font (i.e. Postscript name of the font) used for a
> given textrun.

Attached is a font log, I'm not quite sure what I'm looking for here though. Any insight is appreciated!

> 
> Both Gecko and Webkit nightly implement support for the '-apple-system' font
> family keyword. Under OSX 10.11 this will pick up the appropriate San
> Francisco system UI font based on the size of the text, since 10.11 uses
> size-based font selection.

That seems to be the case according to this log.

> 
> As for OSX system font API's, I tend to read through Webkit code since that
> will tell you where CoreText is heading in the way they use private SPI's.
> Not an ideal state of the world but that's life. I did notice that there
> where some things in there related to rasterization that appear to be in
> 10.11. Something to experiment with I think.

I'd really really really not want to have us dig through WebKit to implement private CoreText APIs to get the font looking like Safari. Nightly using public CoreGraphics APIs already look different than Safari which already looks different than Chrome. All three browsers look slightly different depending on the text, so we're already in a non-uniform case. Digging through WebKit to implement private APIs would also be a headache as they aren't documented, can change with any release, and would be a maintenance nightmare.

[1] https://people.mozilla.org/~danderson/moz-gfx-telemetry/www/

Comment 18

[John Daggett (:jtd)]

Attachment: screenshot, quartz/skia/chrome

Rendering with the skia backend is clearly *not* what Chrome is using, as this screenshot attests to. Text rendering in Chrome tabs is similar to the way they display in Firefox with the current cg backend.

Comment 19

[Mason Chang [Inactive] [:mchang]]

(In reply to John Daggett (:jtd) from comment #18)
> Created attachment 8721519 [details]
> screenshot, quartz/skia/chrome
> 
> Rendering with the skia backend is clearly *not* what Chrome is using, as
> this screenshot attests to. Text rendering in Chrome tabs is similar to the
> way they display in Firefox with the current cg backend.

Had a conversation with John. This is only for tabs and we think Chrome is doing something different with their UI fonts. Content, such as web pages, we both agree text rendering with Skia is the same as Chrome. We just need to figure out what Chromium is doing for their UI fonts to see if they are system fonts or not.

Also some comments about gamma at Chromium here - https://bugs.chromium.org/p/chromium/issues/detail?id=581166

Comment 20

[Mason Chang [Inactive] [:mchang]]

Attachment: Chrome vs Nightly New Tab

Looks like John was right, in that Chrome directly makes CoreGraphics calls for text for non-client area UI elements. Chrome has both "Web Elements" and "UI Elements". Web Elements are things that are webpages such as "Hamburger Menu -> Settings". "UI Elements" are part of the widget itself such as the title above tabs. For Web Elements, those elements are drawn via Skia. For "UI Elements", Chrome directly calls into CG.

For example, New Tab titles are actually NSTextFields that select the system font of size 11 here:

https://code.google.com/p/chromium/codesearch#chromium/src/chrome/browser/ui/cocoa/tabs/tab_view.mm&q=tab_view.mm&sq=package:chromium&l=99

If I print the font, I get his:

2016-02-19 20:59:15.558 Chromium[62144:963562] ".AppleSystemUIFont 11.00 pt. P [] (0x7f9523636b20) fobj=0x7f9523636b60, spc=3.17"

Chrome them also does LCD font smoothing before rendering the font:
https://code.google.com/p/chromium/codesearch#chromium/src/chrome/browser/ui/cocoa/tabs/tab_view.mm&q=tab_view.mm&sq=package:chromium&l=406

For Web Elements, Chrome actually doesn't use the system font, it actually uses Helvetica Neue explicitly:
https://code.google.com/p/chromium/codesearch#chromium/src/ui/strings/app_locale_settings.grd&q=Helvetica%20Neue&sq=package:chromium&type=cs&l=189

I should also note that on a non-retina display, Nightly with a CG backend still looks slightly less heavy than Chrome. See the attached screenshot, Chrome on the bottom. I also changed the font-size to 11px on Nightly just to make sure and it's still slightly less heavy than Chrome.

It looks like Chrome doesn't render large parts of their UI with HTML/CSS unlike us and instead is rendered via Cocoa, so that's why their UI looks more like the underlying platform. So we can either accept Skia rendering, change the weight of the UI fonts, or rewrite some of the UI to use the native platform.

Comment 21

[John Daggett (:jtd)]

Attachment: testcase, apple system font waterfall with blink keyword

Includes BlinkMacSystemFont (gak!) in the font list so that Chrome picks up the equivalent of -apple-system.

Tabs are using 11px normal text so I included some weight variations around that size.

Comment 22

[John Daggett (:jtd)]

Attachment: screenshot, 11px normal text for various OSX browser versions

Screenshots under OSX 10.10, MacBook Pro with Retina display

From top to bottom:

FF Nightly with CG backend
Safari
Chrome
FF Nightly with Skia backend
FF Nightly with Skia backend, using font-weight: 500

Chrome/Firefox are different from Safari mainly due to the use of tracking data in the system font, in this case the UI version of Helvetica Neue. Neither our CoreText or Harfbuzz shapers currently handle this. Chrome doesn't do subpixel AA for Retina displays, while Gecko does.

The key point here is that altering the weight is a dramatic change, much more dramatic than the subtle difference between Skia and CG text rendering. I don't think it's a good idea so I would *strongly* suggest that this bug be closed as WONTFIX.

As for the differences in text rendering for tabs, either accept the difference or fix it within Skia. Or maintain the CG backend and strive to keep text rendering as close as possible to the text rendering for other apps under OSX.

Comment 23

[Mason Chang [Inactive] [:mchang]]

From comment 22, resolving as WONTFIX. We'll accept this visual regression.

Comment 24

[:Gijs (he/him)]

(In reply to John Daggett (:jtd) from comment #22)
> As for the differences in text rendering for tabs, either accept the
> difference or fix it within Skia. Or maintain the CG backend and strive to
> keep text rendering as close as possible to the text rendering for other
> apps under OSX.

Mason, is work on this being done in other bugs, or are you suggesting we "accept the difference" for this too, in addition to wontfixing this? Per comment 9 I don't think the status quo and switching to skia is acceptable from a "Firefox shouldn't look worse" UX / frontend perspective.

Comment 25

[Stephen Horlander [:shorlander]]

(In reply to :Gijs Kruitbosch from comment #24)
> (In reply to John Daggett (:jtd) from comment #22)
> > As for the differences in text rendering for tabs, either accept the
> > difference or fix it within Skia. Or maintain the CG backend and strive to
> > keep text rendering as close as possible to the text rendering for other
> > apps under OSX.
> 
> Mason, is work on this being done in other bugs, or are you suggesting we
> "accept the difference" for this too, in addition to wontfixing this? Per
> comment 9 I don't think the status quo and switching to skia is acceptable
> from a "Firefox shouldn't look worse" UX / frontend perspective.

I would agree.

Comment 26

[Mason Chang [Inactive] [:mchang]]

(In reply to :Gijs Kruitbosch from comment #24)
> (In reply to John Daggett (:jtd) from comment #22)
> > As for the differences in text rendering for tabs, either accept the
> > difference or fix it within Skia. Or maintain the CG backend and strive to
> > keep text rendering as close as possible to the text rendering for other
> > apps under OSX.
> 
> Mason, is work on this being done in other bugs, or are you suggesting we
> "accept the difference" for this too, in addition to wontfixing this? Per
> comment 9 I don't think the status quo and switching to skia is acceptable
> from a "Firefox shouldn't look worse" UX / frontend perspective.

I was suggesting that we "accept the difference". IIRC, most of Firefox's frontend UI is done via HTML/CSS. Is that correct? If so, they'll all go down the same rendering pipeline as web content, and thus use Skia. Chrome gets around this problem by explicitly building their UI in native widget code (cocoa in this case), and web content / web UI elements use Skia. I think everything below should be reconsidered if our front end UI can easily be replaced with cocoa.

Unfortunately we can't really fix Skia such that no artifacts will ever appear. Even if we fixed it such that tabs looked roughly the same, there would be another test case that looked worse. Also, we really don't want to keep CoreGraphics around. I agree it would be great if we could use CG for everything and maintain both, but that's been a losing proposition for us for a while both in terms of rendering performance and engineering resources of maintaining another graphics backend.

Only 2.4% [1] of our users are on OS X, they will get up to a 30% performance increase. Users on a retina display will probably not notice and Apple is moving to all retina displays in general. I did a trial visual test in Mozlando, and 8/10 graphics folks couldn't tell the difference between CG / Skia, 1 preferred Skia, 1 preferred CG.

I do not see a clean way to keep CG fonts the way they are now for UI elements without rewriting our front end UI in cocoa, which I'm also unsure is worth the engineering effort. Please feel free to correct me on any part of this as I'm probably still missing some other pieces of info.

[1] https://people.mozilla.org/~danderson/moz-gfx-telemetry/www/

Comment 27

[:Gijs (he/him)]

(In reply to Mason Chang [:mchang] from comment #26)
> (In reply to :Gijs Kruitbosch from comment #24)
> > (In reply to John Daggett (:jtd) from comment #22)
> > > As for the differences in text rendering for tabs, either accept the
> > > difference or fix it within Skia. Or maintain the CG backend and strive to
> > > keep text rendering as close as possible to the text rendering for other
> > > apps under OSX.
> > 
> > Mason, is work on this being done in other bugs, or are you suggesting we
> > "accept the difference" for this too, in addition to wontfixing this? Per
> > comment 9 I don't think the status quo and switching to skia is acceptable
> > from a "Firefox shouldn't look worse" UX / frontend perspective.
> 
> I was suggesting that we "accept the difference". IIRC, most of Firefox's
> frontend UI is done via HTML/CSS. Is that correct?

XUL + CSS, but it's not cocoa, which presumably is the relevant bit here. Can we continue to use CG for parent process rendering? That seems like it'll get us a lot of the benefits of using Skia and not regress our UI painting.

> Unfortunately we can't really fix Skia such that no artifacts will ever
> appear.

Maybe not (I'm not a gfx person), but (eg.) the loss of subpixel AA for panels, popups and menus seems pretty substantial, and (naively speaking) seems fixable, too. Users file bugs on that kind of thing because they notice. Why can't we address that?

> Only 2.4% [1] of our users are on OS X, they will get up to a 30%
> performance increase.

But just of the rendering of the page/UI, right, which is comparably a small portion of the time it takes for the average person to "load page X"...

> Users on a retina display will probably not notice and
> Apple is moving to all retina displays in general.

I would question that second assertion. The only reason I'm on a retina display is because Mozilla bought me this machine. I wouldn't have paid extra for it, and I know very few people who would. Apple still sells iMacs and Macbook Airs without retina support, and most users tend to keep their machines for pretty long (compared to PCs) periods of time.

> I did a trial visual test
> in Mozlando, and 8/10 graphics folks couldn't tell the difference between CG
> / Skia, 1 preferred Skia, 1 preferred CG.

On retina or non-retina machines? :-)

Comment 28

[Mason Chang [Inactive] [:mchang]]

(In reply to :Gijs Kruitbosch from comment #27)
> (In reply to Mason Chang [:mchang] from comment #26)
> > (In reply to :Gijs Kruitbosch from comment #24)
> > > (In reply to John Daggett (:jtd) from comment #22)
> > > > As for the differences in text rendering for tabs, either accept the
> > > > difference or fix it within Skia. Or maintain the CG backend and strive to
> > > > keep text rendering as close as possible to the text rendering for other
> > > > apps under OSX.
> > > 
> > > Mason, is work on this being done in other bugs, or are you suggesting we
> > > "accept the difference" for this too, in addition to wontfixing this? Per
> > > comment 9 I don't think the status quo and switching to skia is acceptable
> > > from a "Firefox shouldn't look worse" UX / frontend perspective.
> > 
> > I was suggesting that we "accept the difference". IIRC, most of Firefox's
> > frontend UI is done via HTML/CSS. Is that correct?
> 
> XUL + CSS, but it's not cocoa, which presumably is the relevant bit here.
> Can we continue to use CG for parent process rendering? That seems like
> it'll get us a lot of the benefits of using Skia and not regress our UI
> painting.

Not really, our architecture isn't really set up to have different rendering pipelines in different processes. That also sounds like a stability nightmare on Windows if we allowed such situations. We already have crashes / problems related to this happening on Windows due to d3d sometimes failing on different processes, which leads to random configurations and finally crash :(.

> 
> > Unfortunately we can't really fix Skia such that no artifacts will ever
> > appear.
> 
> Maybe not (I'm not a gfx person), but (eg.) the loss of subpixel AA for
> panels, popups and menus seems pretty substantial, and (naively speaking)
> seems fixable, too. Users file bugs on that kind of thing because they
> notice. Why can't we address that?

We could address that, but I'm not sure it's worth it. Even CG doesn't have public support for subpixel AA on transparent backgrounds, which is what's happening on popups. We currently hack around it by using a private CG API. I'm very very concerned about using a private API that could go away at anytime (although it's only available on 10.9+ ATM). See bug 1055622. Do you know why we have slight transparency on menus?

> 
> > Only 2.4% [1] of our users are on OS X, they will get up to a 30%
> > performance increase.
> 
> But just of the rendering of the page/UI, right, which is comparably a small
> portion of the time it takes for the average person to "load page X"...

That'll change quite a bit with APZ / e10s. We've seen significant drawing time of consistent 10-20+ms across various pages, especially on hidpi devices. This leads to checkerboarding. If we paint faster, we'll have less checkerboarding and without e10s, generally smoother scrolling since there will be more frames that are rendered fast and within a vsync interval.

> 
> > Users on a retina display will probably not notice and
> > Apple is moving to all retina displays in general.
> 
> I would question that second assertion. The only reason I'm on a retina
> display is because Mozilla bought me this machine. I wouldn't have paid
> extra for it, and I know very few people who would. Apple still sells iMacs
> and Macbook Airs without retina support, and most users tend to keep their
> machines for pretty long (compared to PCs) periods of time.

True, good point.

> 
> > I did a trial visual test
> > in Mozlando, and 8/10 graphics folks couldn't tell the difference between CG
> > / Skia, 1 preferred Skia, 1 preferred CG.
> 
> On retina or non-retina machines? :-)

I tried on both, well kind of both. I had a retina macbook pro and changed the resolution to not be scaled (e.g. 1400x900 w/o scaling) and tested both.

Comment 29

[:Gijs (he/him)]

(In reply to Mason Chang [:mchang] from comment #28)
> (In reply to :Gijs Kruitbosch from comment #27)
> > (In reply to Mason Chang [:mchang] from comment #26)
> > > (In reply to :Gijs Kruitbosch from comment #24)
> > > > (In reply to John Daggett (:jtd) from comment #22)
> > > > > As for the differences in text rendering for tabs, either accept the
> > > > > difference or fix it within Skia. Or maintain the CG backend and strive to
> > > > > keep text rendering as close as possible to the text rendering for other
> > > > > apps under OSX.
> > > > 
> > > > Mason, is work on this being done in other bugs, or are you suggesting we
> > > > "accept the difference" for this too, in addition to wontfixing this? Per
> > > > comment 9 I don't think the status quo and switching to skia is acceptable
> > > > from a "Firefox shouldn't look worse" UX / frontend perspective.
> > > 
> > > I was suggesting that we "accept the difference". IIRC, most of Firefox's
> > > frontend UI is done via HTML/CSS. Is that correct?
> > 
> > XUL + CSS, but it's not cocoa, which presumably is the relevant bit here.
> > Can we continue to use CG for parent process rendering? That seems like
> > it'll get us a lot of the benefits of using Skia and not regress our UI
> > painting.
> 
> Not really, our architecture isn't really set up to have different rendering
> pipelines in different processes. That also sounds like a stability
> nightmare on Windows if we allowed such situations. We already have crashes
> / problems related to this happening on Windows due to d3d sometimes failing
> on different processes, which leads to random configurations and finally
> crash :(.

I don't really understand how Windows is related, as we were talking about CoreGraphics. In any case, I don't see how this would be any worse than using Skia in both processes. If CG crashes in the chrome process, the entire app will crash. We would always use Skia in the content process, so the situation there would be no worse than using Skia everywhere...

> > > Unfortunately we can't really fix Skia such that no artifacts will ever
> > > appear.
> > 
> > Maybe not (I'm not a gfx person), but (eg.) the loss of subpixel AA for
> > panels, popups and menus seems pretty substantial, and (naively speaking)
> > seems fixable, too. Users file bugs on that kind of thing because they
> > notice. Why can't we address that?
> 
> We could address that, but I'm not sure it's worth it. Even CG doesn't have
> public support for subpixel AA on transparent backgrounds, which is what's
> happening on popups. We currently hack around it by using a private CG API.
> I'm very very concerned about using a private API that could go away at
> anytime (although it's only available on 10.9+ ATM). See bug 1055622. Do you
> know why we have slight transparency on menus?

It's platform convention for menus on OS X to have vibrancy, if that's what you mean, so "to make us not look out of place and/or ugly", essentially.

> > > Only 2.4% [1] of our users are on OS X, they will get up to a 30%
> > > performance increase.
> > 
> > But just of the rendering of the page/UI, right, which is comparably a small
> > portion of the time it takes for the average person to "load page X"...
> 
> That'll change quite a bit with APZ / e10s. We've seen significant drawing
> time of consistent 10-20+ms across various pages, especially on hidpi
> devices. This leads to checkerboarding. If we paint faster, we'll have less
> checkerboarding and without e10s, generally smoother scrolling since there
> will be more frames that are rendered fast and within a vsync interval.

It probably impacts things like page scrolling, but not page or process load times, which is what I meant. A 30% performance increase as a general term is a very strong claim, and one that I think isn't really warranted by what is happening here.

Ultimately I think this needs conversation between the Firefox UX and the graphics folks. As I'm mostly a frontend engineer, I'm not a good representative of either of those, so at the moment it feels like I'm proxying a conversation that other people should be having, which is not helpful. Please set up a meeting with Stephen, :jtd and some other gfx and UX folks to discuss what you want to do here.

Comment 30

[Mason Chang [Inactive] [:mchang]]

I talked with :shorlander, we're going to take a look at making all text a bit bigger or keeping around DrawTargetCG for chrome only. Re-opening bug 1230366.