785707 - [redesign] non-ascii chars are rendered incorrectly

Status: VERIFIED DUPLICATE of bug 806550

(support.mozilla.org :: Localization, task, P2)

Description

[Pavel Cvrcek [:JasnaPaka]]

Attachment: Screenshot

Maybe it is soon to report such bugs but I have very bad experience about it in Mozilla projects. 

1) Visit http://support.allizom.org/cs/home
2) See non-ascii chars which are incorrectly rendered.

See screenshot.

Related bug: Bug 776905

Comment 1

[Ricky Rosario [:rrosario, :r1cky]]

Font issue?

Adding this to the next sprint as it probably blocks rollout.

Comment 2

[Bram Pitoyo [:bram]]

Definitely a font issue. More specifically, it shows that Open Sans doesn’t have the Latin Extended A coverage.

I will investigate whether Open Sans has Extended A and Extended B covered. The webfonts we use may be using a subset that has only Basic Latin covered, to minimize filesize.

Comment 3

[Bram Pitoyo [:bram]]

I have confirmed that all weights of Open Sans have the Extended Latin character set. It’s just a matter of including them.

There are two solutions to this problem:
* Load the full Open Sans font that contains a complete character set. Each font is around 213–225 KB in size.
* Load Open Sans that just contains Basic Latin. If a character from the Latin Extended range is requested, a second font file that just contains Latin Extended characters is requested to complement the Basic Latin. This will save heaps of bandwidth.

In solution #2, first we split each font into multiple files and name it differently
* Basic Latin: open-sans-regular.ttf – name: 'Open Sans Regular'
* Extended Latin: open-sans-regular-extended.ttf – name: 'Open Sans Regular Extended'
* Greek: open-sans-regular-greek.ttf – name: 'Open Sans Regular Greek'
* Cyrillic: open-sans-regular-cyrillic.ttf – name: 'Open Sans Regular Cyrillic'

Method:
1. In the CSS, we load the Basic Latin character set in every page
2. We figure out that, if a page’s HTML contains Unicode characters that ranges from 0080-024F, we call the Extended Latin font
3. If an HTML contains Unicode 0374-03E1, we call the Greek font
4. If an HTML contains Unicode 0400-04F9, we call the Cyrillic font

I am not sure, though, if this is possible to achieve within CSS or not.

This solution will not cover Japanese, Chinese and Korean, for which Open Sans has no coverage of, and we must fallback to use the system font because the file size needed to cover all three is around 30 MB.

Comment 4

[Ricky Rosario [:rrosario, :r1cky]]

Can we use the existing font for en-US and the more complete one for l10n? How much heavier is it?

It looks like this is what is being done in bug 776905. They mention that the full one is 60kb in bug 776905 comment 6. Does that sound right?

Comment 5

[Bram Pitoyo [:bram]]

(In reply to Ricky Rosario [:rrosario, :r1cky] from comment #4)

Yes, please. And I am not sure if this is possible or not, but if we can use the same method (Option B) that bug 776905 uses, that would be great. That way, we don’t have to serve non-Latin font to English users.

The file size for just Basic Latin is 20 KB, like bug 776905 comment 6

The file size for Basic Latin, Latin Extended, Greek and Cyrillic, is anywhere between 213–225 KB for each font.

I am not sure how they were able to get the 60 KB figure. Maybe 60 is if we strip everything else but the Cyrillic character set?

Comment 6

[Ricky Rosario [:rrosario, :r1cky]]

Assuming that we are flipping the font for en-US vs l10n, this is a 1pt.

Comment 8

[Ricky Rosario [:rrosario, :r1cky]]

Rehan landed and I deployed yesterday:

https://github.com/mozilla/kitsune/commit/7ef5d71c91a33a3d687a14dadfe8185ecddacce8

Comment 9

[Pavel Cvrcek [:JasnaPaka]]

(In reply to Ricky Rosario [:rrosario, :r1cky] from comment #8)
> Rehan landed and I deployed yesterday:

If it does mean this should work at this moment then it doesn't work. I tried https://support.mozilla.org/ and https://support.allizom.org/

Comment 10

[Ricky Rosario [:rrosario, :r1cky]]

(In reply to Pavel Cvrcek (Mozilla.cz) [:JasnaPaka] from comment #9)
> If it does mean this should work at this moment then it doesn't work. I
> tried https://support.mozilla.org/ and https://support.allizom.org/

I guess we had the wrong files. Now that Bug 776905 landed, we are going to point straight to the fonts hosted on mozilla.org. Any issues after that will be an issue with the font itself.

Comment 11

[Ricky Rosario [:rrosario, :r1cky]]

Rehan landed and I deployed:
https://github.com/mozilla/kitsune/commit/22e57fc6acffa453b1815106da0251da088d464b

We should be good now. We'll switch tot he moz.org fonts when they are all set for others to use.

Comment 12

[Pavel Cvrcek [:JasnaPaka]]

Thanks. I can confirm it works fine for czech.

Comment 13

[Leszek Zyczkowski]

For Polish also. Thanks.

Comment 14

[Pavel Cvrcek [:JasnaPaka]]

Problem with font is back. See Bug 806550.

Comment 15

[Bram Pitoyo [:bram]]

That bug has been fixed, so this bug is now resolved.