Closed Bug 1353639 Opened 3 years ago Closed 3 years ago

4.74% tp5n main_normal_fileio (windows7-32) regression on push b7a6ec5eb1e1ff88d9ebc278de9ea3cd761f763f (Tue Apr 4 2017)

Categories

(Firefox :: Untriaged, defect)

Unspecified
Windows 7
defect
Not set

Tracking

()

RESOLVED INVALID
Tracking Status
firefox55 --- affected

People

(Reporter: igoldan, Assigned: masayuki)

References

Details

(Keywords: perf, regression, talos-regression)

Talos has detected a Firefox performance regression from push b7a6ec5eb1e1ff88d9ebc278de9ea3cd761f763f. As author of one of the patches included in that push, we need your help to address this regression.

Regressions:

  5%  tp5n main_normal_fileio windows7-32 opt      64393253.58 -> 67442373.67


You can find links to graphs and comparison views for each of the above tests at: https://treeherder.mozilla.org/perf.html#/alerts?id=5845

On the page above you can see an alert for each affected platform as well as a link to a graph showing the history of scores for this test. There is also a link to a treeherder page showing the Talos jobs in a pushlog format.

To learn more about the regressing test(s), please see: https://wiki.mozilla.org/Buildbot/Talos/Tests

For information on reproducing and debugging the regression, either on try or locally, see: https://wiki.mozilla.org/Buildbot/Talos/Running

*** Please let us know your plans within 3 business days, or the offending patch(es) will be backed out! ***

Our wiki page outlines the common responses and expectations: https://wiki.mozilla.org/Buildbot/Talos/RegressionBugsHandling
Jonathan:

Do you know if same performance regression occurred at changing default fonts of Chinese and/or Korean from legacy bitmap fonts to modern fonts?

I heard from some people of MS that Meiryo has a lot of hinting information than usual fonts to render Kanji characters (Chinese characters) in narrow space. I guess that it could cause this performance issue.
Flags: needinfo?(jfkthame)
Hmm, looks like that only Simplified Chinese's (zh-CN) default font has changed to modern font, Microsoft YaHei, but zh-TW, zh-HK and ko still use legacy fonts. zh-CN was changed by bug 1096800.
tp5n test has at least 7 Japanese websites, but there are 3 Chinese sites (I check with .jp and .cn, though). The difference of websites may have not caused performance regression at changing Chinese default font. Or, simply, some of the websites may have specified modern font with font-family.
Hmm, although, I don't know how Chromium developers detect their performance regression, I don't see any performance regression at changing their default CJK fonts from the comments:
https://bugs.chromium.org/p/chromium/issues/detail?id=368568
https://bugs.chromium.org/p/chromium/issues/detail?id=579842
IIUC, the regression here is that around 3MB more i/o is happening. Looking at some example font file sizes (these are from a variety of Windows releases, depending what I have on hand, but give an idea...):

   9209540  6 Jun  2013 msgothic.ttc
  10080360  6 Jun  2013 msmincho.ttc
   9521084 26 Mar  2013 meiryo.ttc
   9737580 26 Mar  2013 meiryob.ttc
  13758988 29 May  2013 yugothib.ttf
  12812828 29 May  2013 yugothic.ttf
  12795552 29 May  2013 yugothil.ttf

  14664688 23 May  2015 YuGothB.ttc
  13725364 23 May  2015 YuGothL.ttc
  13940612 23 May  2015 YuGothM.ttc
  13933692 23 May  2015 YuGothR.ttc
  13107212 22 Aug  2013 yumin.ttf
  13249660 22 Aug  2013 yumindb.ttf
   5487844 22 Aug  2013 yuminl.ttf

we can see that typical CJK fonts on Windows are multiple MB *per face*, with individual files often exceeding 10MB.

As such, I think it's totally reasonable that a change to the default CJK font preferences will have an effect on the amount of font data we end up reading. If we can trace exactly which files are being accessed (do we have that data?) I'm sure this would confirm that we're simply loading a somewhat larger font than before -- e.g. YuGothic is several MB larger than MSGothic, and that's before we add in the fact that it comes in more weights -- and there's nothing to do here; the change is expected.

If -- and only if -- the increased i/o is resulting in a significant user-visible performance regression (slower launch? UI jank?) then I guess there'd be a decision to be made as to whether the change to default fonts is important enough to justify the cost. But the increase in i/o by itself is within expectations and does not mean anything is wrong.
Flags: needinfo?(jfkthame)
Thank you for the investigation.

msgothic.ttc is only 9,209,540, but, meiryo and meiryob are 19,258,664 (9,521,084 + 9,737,580) according to your list.

And this test runs on Win7. Win7 doesn't have Yu Gothic nor Yu Mincho. So, MS PMincho -> Yu Mincho change shouldn't affect the performance (except that if the fallback is too slow).

I don't think that initial cost of loading Meiryo doesn't matter actually because many websites specify Meiryo as default font. So, Meiryo is usually loaded if users load major websites.
I'm still not sure if the increased time is the difference of loading time, though.

I tried to check some cases:

1. Not adding Meiryo Bold into the font list <https://treeherder.mozilla.org/#/jobs?repo=try&revision=28a026866c0ae566e6e0b77343cb7fa6dab2a12a>
  In this case, I don't see the difference of the result. It might be the font list to have loaded Meiryo Bold when listing up. I'll check it tomorrow.

2. Using Yu Gothic as the default font <https://treeherder.mozilla.org/#/jobs?repo=try&revision=e7cf94ad9f1e4c8111b349801811bbf662a1aaf0>
  In this case, Yu Gothic isn't available on Win7. Therefore, I don't see any difference of the score.

3. Avoiding to use Yu Mincho as default serif font <https://treeherder.mozilla.org/#/jobs?repo=try&revision=4a6b6c7416c1338f54ae55f45f8b0e735da67058>
  In this case, if the test pages uses "serif" for Japanese text, the fallback hits unavailable font every time at font.name-list.serif.ja. However, I don't see any difference of the score. So, it's not problem of fallback cost of serif font on Win7.

4. Changing default Simplified Chinese default to the legacy font <https://treeherder.mozilla.org/#/jobs?repo=try&revision=69252139bca5db02ca9348ea9ba4415dd77bd37d>
  In this case, the score is improved similar to before bug 548311. From this experiment, I can say that setting default font to modern CJK font needs performance regression anyway.


So, I'll check more about #1, though, looks like that the runtime cost is necessary to change default font from legacy font.
(In reply to Jonathan Kew (:jfkthame) from comment #6)
> As such, I think it's totally reasonable that a change to the default CJK
> font preferences will have an effect on the amount of font data we end up
> reading. If we can trace exactly which files are being accessed (do we have
> that data?) I'm sure this would confirm that we're simply loading a somewhat
> larger font than before and there's nothing to do here; the change is expected.

Do you have some ideas to check if the performance difference is caused only by the font file size? I guess you have something because you said "if we can trace", not "if we could trace" (but I'm not sure due to my poor English skill).
Okay, I changed western and unicode default fonts to Meiryo and measured the performance:
https://treeherder.mozilla.org/#/jobs?repo=try&revision=74fec8a73bc6497f3b21de56442862c72a26469c

Then, the result is same as (or a little bit faster than) the performance of current autoland.
https://treeherder.mozilla.org/perf.html#/graphs?timerange=604800&series=%5Btry,98c1fb4c39de2dad63358b4195ba65dbf0810ab9,1,1%5D&series=%5Bautoland,98c1fb4c39de2dad63358b4195ba65dbf0810ab9,1,1%5D&highlightedRevisions=74fec8a73bc6&zoom=1491268949301.6924,1491438915000,55000000,75000000

So, this means that rendering with Meiryo isn't slower than usual western fonts. The slowdown should be caused by the font file size difference at first use of Japanese default font. Additionally, Meiryo is specified explicitly in some major websites. So, even if Meiryo is not default Japanese font, Japanese website users usually load Meiryo in every browser session. Therefore, the slowdown doesn't mean new additional cost for most Japanese users.

In other words, the rendering cost of legacy Japanese fonts (MS PGothic and MS PMincho) was just much cheaper than modern fonts (including western fonts). So, marking this bug as INVALID.
Status: NEW → RESOLVED
Closed: 3 years ago
Flags: needinfo?(jfkthame)
Resolution: --- → INVALID
You need to log in before you can comment on or make changes to this bug.