[webvtt] German umlauts leave black border which makes them unreadable on white background

NEW
Unassigned

Status

()

P3
normal
a year ago
a year ago

People

(Reporter: kontakt, Unassigned)

Tracking

({compat})

55 Branch
x86_64
Mac OS X
compat
Points:
---

Firefox Tracking Flags

(firefox57 wontfix, firefox58 affected)

Details

Attachments

(1 attachment)

(Reporter)

Description

a year ago
Created attachment 8896937 [details]
Illustrating the problem on different backgrounds

User Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.12; rv:55.0) Gecko/20100101 Firefox/55.0
Build ID: 20170809080250

Steps to reproduce:

I watched a video that shows a VTT-track for subtitles. The VTT-track contains german umlauts (ÄÖÜ).

Here is an example video. I modified the demo files I found (https://developer.mozilla.org/en-US/Apps/Fundamentals/Audio_and_video_delivery/Adding_captions_and_subtitles_to_HTML5_video) to include capitalized umlauts: 

http://temp.fetzer.me/vtt/video-with-captions.html


Actual results:

The subtitles "hide" german umlauts (ÄÖÜ), because the dots above the letters aren't drawn with a black background color. Therefore the dots are unrecognizable on light video backgrounds. It seems as this bug only appears on macOS (10.12.6 here) and not on Windows.


Expected results:

The dots should be surrounded by a black background so they are legible on light video backgrounds.
(Reporter)

Updated

a year ago
OS: Unspecified → Mac OS X
Hardware: Unspecified → x86
Summary: [webvtt] German umlauts leave the black border which makes them unreadable on white background → [webvtt] German umlauts leave black border which makes them unreadable on white background
Component: Untriaged → Audio/Video: Playback
Product: Firefox → Core
Priority: -- → P1
To solve the bug, we need someone who know the layout/font.

The flow of the vtt.jsm is a little complicated, but the major functionality is based on the result of
|getBoundingClientRect| function to get a rectangle to wrap the text, and also set background color on it.

So it looks like on Mac, the getBoundingClientRect returns a rectangle that is not bigger enough to wrap the text.
I can't think of a general Web API to get the precise font metric data.

I'm not familiar with webvtt implementation, but if you try to get the precise font metric data, you might want to use the internal API nsIFrame::GetVisualOverflowRect [1]. So, this probably can't be fixed in the vtt.jsm.

Some other options might be use <canvas> or <textarea> instead, but these might need more complex/huge implementation change. <canvas> provides TextMetrics Web API which seems exactly what you need, and it seems that <textarea> always contains the complete font bounding box, including the german umlauts (ÄÖÜ).


[1] https://searchfox.org/mozilla-central/rev/48ea452803907f2575d81021e8678634e8067fc2/layout/generic/nsIFrame.h#3042-3066
[2] https://html.spec.whatwg.org/multipage/canvas.html#textmetrics
This problem doesn't happen on Chrome on my Macbook Pro.

Benjamin,
Per comment 2, is this something we can try? Or it would be better to have layout team to help us out?
Flags: needinfo?(bechen)
Keywords: compat
(In reply to Blake Wu [:bwu][:blakewu] from comment #3)
> This problem doesn't happen on Chrome on my Macbook Pro.
> 
> Benjamin,
> Per comment 2, is this something we can try? Or it would be better to have
> layout team to help us out?

As Jeremy's suggestion, we can try the <textarea>, but I guess it won't actually help vtt.jsm to have a solution. Because now we create a <div> to wrap the text and apply psuedo-element style on it. If <textarea> works for "german umlauts", I'm not sure how much effort we still need to do.
We must have layout team's support to fix this.
Flags: needinfo?(bechen)
Ni Astley since this is not only a layout bug but also a web comat bug. :-)
Flags: needinfo?(aschen)
Jonathan, as it only happens on macOS build, would be it a font issue from the first sight of you? Thanks.
Status: UNCONFIRMED → NEW
Ever confirmed: true
Flags: needinfo?(aschen) → needinfo?(jfkthame)
Hardware: x86 → x86_64
This is happening because our default sans-serif font on macOS is Helvetica, and Helvetica's font metrics (ascent and descent) do not extend far enough to include accents on uppercase letters. So capitals with umlauts (for example) will project beyond the font's nominal ascent line.

You can see this in a simple HTML example such as:

  data:text/html;charset=utf-8,<span style="font:36px Helvetica;background:yellow">Test %C3%84%C3%9C%C3%96</span>

where the umlauts fall outside the yellow background of the <span>.

The Arial font, in contrast, has a more generous ascent value and so there is enough space within the font's vertical bounds for the umlauts to be painted without "overflow":

  data:text/html;charset=utf-8,<span style="font:36px Arial;background:yellow">Test %C3%84%C3%9C%C3%96</span>

Note that the problem would still show up with Arial (and on other platforms) in the case of "stacked" diacritics:

  data:text/html;charset=utf-8,<span style="font:36px Arial;background:yellow">Test %C3%8B%CC%81%C3%9C%CC%82</span>

but such examples are much rarer than simple German umlauts etc.

The simplest workaround -- though it is not a complete solution -- would probably be to change the default font used for the captions to either Arial or Helvetica Neue, both of which have metrics that include enough space for most common accents.
Flags: needinfo?(jfkthame)
Thanks for Jonathan's feedback, that helps a lot.

An alternative solution could be adding reasonable padding to top as Chrome does to avoid specific font issue.
Safari also adds even larger paddings around captions so that it can avoid this problem too.

Benjamin, WDYT?
Flags: needinfo?(bechen)
(In reply to Jonathan Kew (:jfkthame) from comment #7)
> This is happening because our default sans-serif font on macOS is Helvetica,
> and Helvetica's font metrics (ascent and descent) do not extend far enough
> to include accents on uppercase letters. So capitals with umlauts (for
> example) will project beyond the font's nominal ascent line.
> 
> You can see this in a simple HTML example such as:
> 
>   data:text/html;charset=utf-8,<span style="font:36px
> Helvetica;background:yellow">Test %C3%84%C3%9C%C3%96</span>
> 
> where the umlauts fall outside the yellow background of the <span>.
> 
> The Arial font, in contrast, has a more generous ascent value and so there
> is enough space within the font's vertical bounds for the umlauts to be
> painted without "overflow":
> 
>   data:text/html;charset=utf-8,<span style="font:36px
> Arial;background:yellow">Test %C3%84%C3%9C%C3%96</span>
> 
> Note that the problem would still show up with Arial (and on other
> platforms) in the case of "stacked" diacritics:
> 
>   data:text/html;charset=utf-8,<span style="font:36px
> Arial;background:yellow">Test %C3%8B%CC%81%C3%9C%CC%82</span>
> 
> but such examples are much rarer than simple German umlauts etc.
> 
> The simplest workaround -- though it is not a complete solution -- would
> probably be to change the default font used for the captions to either Arial
> or Helvetica Neue, both of which have metrics that include enough space for
> most common accents.

The webvtt spec says the default font is "sans-serif", so is it possible to change the mapping between "sans-serif" and "Helvetica" on macOS?


(In reply to Astley Chen [:astley] from comment #8)
> Thanks for Jonathan's feedback, that helps a lot.
> 
> An alternative solution could be adding reasonable padding to top as Chrome
> does to avoid specific font issue.
> Safari also adds even larger paddings around captions so that it can avoid
> this problem too.
> 
> Benjamin, WDYT?

I'm afraid that adding padding will break the wpt tests. The reference of wpt tests usually use a <span> as text container.
Flags: needinfo?(bechen)
(In reply to Benjamin Chen [:bechen] from comment #9)
> The webvtt spec says the default font is "sans-serif", so is it possible to
> change the mapping between "sans-serif" and "Helvetica" on macOS?

That would mean changing the various font.name-list.sans-serif.* prefs that currently specify Helvetica as the default for many languages/scripts.

This would of course affect content everywhere, so there's a risk that it would have undesired side-effects...we'd need pretty extensive testing of such a far-reaching change.

(FWIW, I believe both Safari and Chrome also use Helvetica as the default sans-serif on macOS, so if we changed, we'd be breaking consistency with them. The design differences between Helvetica and either Arial or Helvetica Neue is subtle enough that I doubt most users would ever notice -- except for a minority of font nerds! -- but it's possible that slight changes in metrics would cause layout breakage on pages that have made fragile assumptions about the precise size of text.)
These possible solutions are:
1. Change the sans-serif not mapping to Helvetica => no good, comment 10
2. Change the font in webvtt => no good, break spec
3. Add padding in webvtt => no good, complicated implementation, break wpt tests
...

So, now I prefer not to fix this bug.
(Reporter)

Comment 12

a year ago
Thanks for your effort so far. I'm deeply irritated that difficulties regarding this bug might lead to it being unfixed. This is in my eyes no question of preference. It's a crystal clear bug and it also affects accessibility of videos in Firefox.
Mass change P1->P2 to align with new Mozilla triage process
Priority: P1 → P2
(In reply to Jonathan Kew (:jfkthame) from comment #7)
> You can see this in a simple HTML example such as:
> 
>   data:text/html;charset=utf-8,<span style="font:36px
> Helvetica;background:yellow">Test %C3%84%C3%9C%C3%96</span>
> 
> where the umlauts fall outside the yellow background of the <span>.

Hi Jonathan, the example above seems to work fine on Chrome.
The default san-serif font loaded on Chrome is Helvetica as well, however, background covers umlauts.
Dunno whether there is any tweak behind their implementation.
Chrome is clearly using a different basis to compute the (typographic) bounding rect of the element.
Looking at a testcase like

  data:text/html;charset=utf-8,<span style="font:100px Helvetica;background:yellow">Test %C3%84%C3%9C%C3%96</span> <span style="font:100px Helvetica Neue;background:yellow">Test %C3%84%C3%9C%C3%96</span>

we can see that in Firefox, the bounds of the Helvetica span extend only slightly higher than the ascent of the capital letters, while in Helvetica Neue there is a lot more "padding" (i.e. space beyond the capital letters, but still within the ascent that we used to compute the rect).

In Chrome, OTOH, the Helvetica span gets an ascent that is almost (not quite) the same as that of Helvetica Neue.

So why the difference? Unfortunately, fonts (for various legacy reasons) contain several sets of metrics, which may disagree (greatly) about basics such as "ascent" and "descent":

* Helvetica, from 'hhea' and 'OS/2' tables (these are in terms of a unitsPerEm value of 2048):
  <hhea>
    <ascent value="1577"/>
    <descent value="-471"/>
    <lineGap value="0"/>
  <OS_2>
    <sTypoAscender value="1536"/>
    <sTypoDescender value="-348"/>
    <sTypoLineGap value="61"/>
    <usWinAscent value="1946"/>
    <usWinDescent value="461"/>
    <sCapHeight value="1469"/>

* Corresponding values from Helvetica Neue (unitsPerEm is 1000):
  <hhea>
    <ascent value="975"/>
    <descent value="-217"/>
    <lineGap value="29"/>
  <OS_2>
    <sTypoAscender value="975"/>
    <sTypoDescender value="-217"/>
    <sTypoLineGap value="29"/>
    <usWinAscent value="975"/>
    <usWinDescent value="217"/>
    <sCapHeight value="714"/>

So in Helvetica, we have a capHeight of 0.717em, while in HelvNeue it's 0.714em. Pretty similar (and indeed, the capitals look about the same height).

But what about ascent? Well, that depends... Helvetica's 'hhea' ascent is 0.77em, or about 7% larger than the capHeight; and its sTypoAscender is a mere 0.75em (4.56% above the capHeight). The only significantly larger "ascent" offered is usWinAscent, which works out at 0.95em. But this field is a legacy Windows value that is not recommended for default line-spacing calculations, per the OpenType spec.

In contrast, HelvNeue has an ascent (regardless of which field we consider) of 0.975em, 36.5% larger than its capHeight. That's why it gets a bounding rect that leaves plenty of room for diacritics above the capital letters.

Looking at the Chrome result, though, it certainly appears that they are basing things on the usWinAscent field in Helvetica. As a result, their 'normal' line-height for Helvetica works out much larger than in Firefox; compare the renderings of:

  data:text/html;charset=utf-8,<div style="font:32px Helvetica; line-height:normal">this<br>is<br>a<br>test

and this is very clear.

So what's "right" and what's "wrong" here? It's far from clear! We are currently using either the 'hhea' values or the OS/2 'sTypo*' values (I forget exactly; the code is probably in thebes/gfxFont.cpp) to compute the ascent and descent that determine the bounding rect of the element, as well as default line-height. We could look into changing this to check the usWinAscent field, although in principle that is discouraged by the spec -- it might improve interoperability with what Chrome seems to be doing.

However, any change to how we compute font metrics will have far-reaching impact on spacing and alignment of text everywhere, both in the UI and on web pages, so it's not something to attempt lightly. Changing the default sans-serif from Helvetica to Arial (comment 10) would be a lower-risk option than revising font metric computations (as it wouldn't affect metrics of every other font, particularly in UI elements).
Change the component to "Layout" since this is more related to it.
Component: Audio/Video: Playback → Layout
(In reply to Benjamin Chen [:bechen] from comment #11)

> 3. Add padding in webvtt => no good, complicated implementation, break wpt tests

I'm missing how other browsers are able to pass these tests and not have this bug. 

Chromium seems to just hard-wire 100 for a size:
https://cs.chromium.org/chromium/src/third_party/WebKit/Source/core/html/track/vtt/VTTCue.cpp?l=240

I'm willing to take a similar fix here if it's a reasonable default that gets us the desired result. We're certainly not changing all serif font mapping for this bug. We can fix tests as needed.
status-firefox57: --- → wontfix
status-firefox58: --- → affected
Component: Layout → Audio/Video: Playback
Flags: needinfo?(bechen)
Priority: P2 → P3
(In reply to Jet Villegas (:jet) from comment #17)
> (In reply to Benjamin Chen [:bechen] from comment #11)
> 
> > 3. Add padding in webvtt => no good, complicated implementation, break wpt tests
> 
> I'm missing how other browsers are able to pass these tests and not have
> this bug. 
> 
> Chromium seems to just hard-wire 100 for a size:
> https://cs.chromium.org/chromium/src/third_party/WebKit/Source/core/html/
> track/vtt/VTTCue.cpp?l=240
> 
> I'm willing to take a similar fix here if it's a reasonable default that
> gets us the desired result. We're certainly not changing all serif font
> mapping for this bug. We can fix tests as needed.

(In reply to Jet Villegas (:jet) from comment #18)
> Actually, the magic number is 5.33 (LOL!)
> https://cs.chromium.org/chromium/src/third_party/WebKit/Source/core/html/
> track/vtt/VTTRegion.cpp?l=67

No, the 5.33vh is only for the "region" pseudo element. Now the scenario is not pseudo element.

I don't know how Google Chrome does, need some time to study.
Flags: needinfo?(bechen)
You need to log in before you can comment on or make changes to this bug.