Open Bug 1502718 Opened 6 years ago Updated 2 years ago

Unicode code point FE0E text variant selector (instead of emoji) ignored

Categories

(Core :: Layout: Text and Fonts, defect, P3)

Product:
Core
Component:
Layout: Text and Fonts
Platform:
Unspecified
Linux
Type:
defect

Tracking

()

People

(Reporter: Christian.Hujer, Unassigned)

Details

Attachments

(1 file)

Capture.PNG
25.99 KB, image/png
Details 
User Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Ubuntu Chromium/70.0.3538.67 Chrome/70.0.3538.67 Safari/537.36

Steps to reproduce:

Create a page that contains the following HTML:

<!DOCTYPE html>
<figure>
    <table border="border">
        <thead>
        <tr><th rowspan="2">Emoji</th><th>Text</th><th>Emoji</th></tr>
        <tr><th colspan="2">Mode</th></tr>
        </thead>
        <tbody>
        <tr><td>Email</td><td>&#x1f4e7;&#xFE0E;</td><td>&#x1f4e7;&#xFE0F;</td></tr>
        <tr><td>Penguin</td><td>&#x1f427;&#xFE0E;</td><td>&#x1f427;&#xFE0F;</td></tr>
        </tbody>
    </table>
    <figcaption>Examples for emojis with explicitly selected text and emoji modes.</figcaption>
</figure>



Actual results:

Firefox always shows the emoji.


Expected results:

Firefox shows a normal font symbol instead of the emoji when the emoji code point is followed by the text variant selector code point FE0F.
User Agent: Mozilla/5.0 (X11; Linux x86_64; rv:63) Gecko/20100101 Firefox/63.0

Hi Christian,

Thank you for taking the time to add this report. 

I verified this using the latest Fx Versions ( 63.0.1 Release Build/20181030165643; 64.0b5 DevEdition Build/ 20181029164536; 65.0a1 Nightly Build: 20181101100640) on Windows 7, Windows 10, MacOSX and Ubuntu 18.04. 

I'm not exactly sure if I understood the expected result properly, but if it should look like the 1st 2 images in the attachment ( that are from Win 7 and Win 10 ) and  the actual looks like the 3rd one (from Linux), then I'm assuming it is reproducible on all Fx Versions on Linux and Mac OSX and not reproducible on Windows.

I'm going to add this to Core:Editor component for a more advised input, but if this is not the proper component, please feel free to change it.
Status: UNCONFIRMED → NEW
Component: Untriaged → Editor
Ever confirmed: true
OS: Unspecified → Linux
Product: Firefox → Core
Version: 63 Branch → Trunk
Attached image Capture.PNG — 

    
    

    
  
This depends on fontconfig and etc.  If not Linux, it will work well because these OS has mono font.

the latest fontconfig has emoji font family, but I don't know for monochrome emoji family.
Component: Editor → Layout: Text and Fonts

    
    

    
  
Problem occurs both on linux and android platform

Reference to the standard : http://unicode.org/emoji/charts/emoji-variants.html

    
    

    
  
Does the system even have a font available that provides text-mode glyphs for these characters? If so, which font?
Priority: -- → P3

    
    

    
  
FE0F doesn't work as well for me. I have a webfont (WOFF). It contains a gylph for the blue right arrow (https://emojipedia.org/right-arrow/). It consists of the normal arrow (27A1) and the variation selector (FE0F). The unicode value is -1 and Alternate Unicode has the entry 27A1 => FE0F.


In Chrome I can see my glyph without a problem. Firefox always insists on showing the blue arrow from its built-in Twemoji font.


If I output the normal arrow (27A1) without variation selector, I can see the webfont is used. Really confusing behavior which I have a hard time making sense of.


Are these variation selectors supported properly at all? It seems this is not the case...



    
    

    
  
(In reply to r from comment #6)




FE0F doesn't work as well for me. I have a webfont (WOFF). It contains a gylph for the blue right arrow (https://emojipedia.org/right-arrow/). It consists of the normal arrow (27A1) and the variation selector (FE0F). The unicode value is -1 and Alternate Unicode has the entry 27A1 => FE0F.


In Chrome I can see my glyph without a problem. Firefox always insists on showing the blue arrow from its built-in Twemoji font.


If I output the normal arrow (27A1) without variation selector, I can see the webfont is used. Really confusing behavior which I have a hard time making sense of.


Are these variation selectors supported properly at all? It seems this is not the case...




Could you point to a specific example page that uses your webfont and exhibits this problem?


Flags: needinfo?(r)

    
    

    
  
Here it is:


https://mozfontbug.pw6.de


Flags: needinfo?(r)

    
    

    
  
I'm not seeing the webfont used at all when I view that page. For me, the first line of arrows are rendered from DejaVu Serif (on Ubuntu) or Zapf Dingbats (on macOS), and the second line are rendered from Twemoji Mozilla (Ubuntu) or Apple Color Emoji (macOS).


The webfont is being downloaded, and incidentally logs a warning in the console:




downloadable font: OS/2: usFirstCharIndex 65535 > usLastCharIndex 0 (font-family: "Icons" style:normal weight:400 stretch:100 src index:0) source: https://mozfontbug.pw6.de/Icons.woff




Examining it with TTX, the cmap doesn't quite look like I would expect:


  <cmap>
    <tableVersion version="0"/>
    <cmap_format_4 platformID="0" platEncID="3" language="0">
    </cmap_format_4>
    <cmap_format_14 platformID="0" platEncID="5">
      <map uv="0x27a1" uvs="0xfe0f" name="uni27A1"/>
    </cmap_format_14>
    <cmap_format_0 platformID="1" platEncID="0" language="0">
      <map code="0x0" name="uni27A1"/>
    </cmap_format_0>
    <cmap_format_4 platformID="3" platEncID="1" language="0">
    </cmap_format_4>
  </cmap>



In particular, it seems odd that (a) the subtable with platform=1, encoding=0 (MacRoman) tries to map character code zero to the uni27A1 glyph (surely that's wrong?); and (b)  in the Unicode and MS subtables (0/3 and 3/1), there is no mapping at all for the U+27A1 character. I think that's why Firefox is not attempting to use this font. It won't get as far as considering the 0/5 (format-14) subtable that tries to map the <arrow, VS-16> combination, because the font does not appear to support the individual codepoints at all. (That's also probably why the tool that created the font ended up setting bogus usFirstCharIndex and usLastCharIndex values, which imply the font has no characters.)


A more complete/coherent cmap would probably allow the font to work; specifically, I think the format-4 subtables should explicitly map the individual codepoints to something, even if in the case of the variation selector, it's simply an invisible null glyph.



    
    

    
  
I created another example:


https://mozfontbug.pw6.de/index2.html


In this case, one character without the variation selector is also visible in Firefox. But not the one with variation selector. In all other browsers I could get hold of, this works.


But, still, I absolutely got your valid concerns about invalid cmap etc. and will try to check these. Thank you so much! I created this icon font with fontforge BTW and this script: https://github.com/pteromys/svgs2ttf


Perhaps this is a bug in FontForge when generating this font? Nevertheless it's a real-life font that works in all other browsers - so I thought I'd mention this here.



    
    

    
  
(In reply to r from comment #10)




I created another example:


https://mozfontbug.pw6.de/index2.html


In this case, one character without the variation selector is also visible in Firefox. But not the one with variation selector.




The cmap here still does not include any mapping for the individual right-arrow codepoint (in the format-4 and format-12 subtables):


  <cmap>
    <tableVersion version="0"/>
    <cmap_format_4 platformID="0" platEncID="3" language="0">
      <map code="0x22" name="quotedbl"/><!-- QUOTATION MARK -->
      <map code="0x40" name="at"/><!-- COMMERCIAL AT -->
      <map code="0x2b07" name="uni2B07"/><!-- DOWNWARDS BLACK ARROW -->
    </cmap_format_4>
    <cmap_format_12 platformID="0" platEncID="4" format="12" reserved="0" length="148" language="0" nGroups="11">
      <map code="0x22" name="quotedbl"/><!-- QUOTATION MARK -->
      <map code="0x40" name="at"/><!-- COMMERCIAL AT -->
      <map code="0x2b07" name="uni2B07"/><!-- DOWNWARDS BLACK ARROW -->
      <map code="0x1f1f1" name="u1F1F1"/><!-- REGIONAL INDICATOR SYMBOL LETTER L -->
      <map code="0x1f37c" name="u1F37C"/><!-- BABY BOTTLE -->
      <map code="0x1f466" name="u1F466"/><!-- BOY -->
      <map code="0x1f467" name="u1F467"/><!-- GIRL -->
      <map code="0x1f46a" name="u1F46A"/><!-- FAMILY -->
      <map code="0x1f476" name="u1F476"/><!-- BABY -->
      <map code="0x1f4c5" name="u1F4C5"/><!-- CALENDAR -->
      <map code="0x1f50e" name="u1F50E"/><!-- RIGHT-POINTING MAGNIFYING GLASS -->
      <map code="0x1f930" name="u1F930"/><!-- PREGNANT WOMAN -->
    </cmap_format_12>
    <cmap_format_14 platformID="0" platEncID="5">
      <map uv="0x260e" uvs="0xfe0f" name="NameMe.9742"/>
      <map uv="0x27a1" uvs="0xfe0f" name="uni27A1"/>
    </cmap_format_14>
    <cmap_format_0 platformID="1" platEncID="0" language="0">
      <map code="0x0" name="quotedbl"/>
      <map code="0x22" name="quotedbl"/>
      <map code="0x40" name="at"/>
    </cmap_format_0>
    <cmap_format_4 platformID="3" platEncID="1" language="0">
      <map code="0x22" name="quotedbl"/><!-- QUOTATION MARK -->
      <map code="0x40" name="at"/><!-- COMMERCIAL AT -->
      <map code="0x2b07" name="uni2B07"/><!-- DOWNWARDS BLACK ARROW -->
    </cmap_format_4>
    <cmap_format_12 platformID="3" platEncID="10" format="12" reserved="0" length="148" language="0" nGroups="11">
      <map code="0x22" name="quotedbl"/><!-- QUOTATION MARK -->
      <map code="0x40" name="at"/><!-- COMMERCIAL AT -->
      <map code="0x2b07" name="uni2B07"/><!-- DOWNWARDS BLACK ARROW -->
      <map code="0x1f1f1" name="u1F1F1"/><!-- REGIONAL INDICATOR SYMBOL LETTER L -->
      <map code="0x1f37c" name="u1F37C"/><!-- BABY BOTTLE -->
      <map code="0x1f466" name="u1F466"/><!-- BOY -->
      <map code="0x1f467" name="u1F467"/><!-- GIRL -->
      <map code="0x1f46a" name="u1F46A"/><!-- FAMILY -->
      <map code="0x1f476" name="u1F476"/><!-- BABY -->
      <map code="0x1f4c5" name="u1F4C5"/><!-- CALENDAR -->
      <map code="0x1f50e" name="u1F50E"/><!-- RIGHT-POINTING MAGNIFYING GLASS -->
      <map code="0x1f930" name="u1F930"/><!-- PREGNANT WOMAN -->
    </cmap_format_12>
  </cmap>



I believe this is why Firefox ignores it, because it uses this character coverage data to determine whether the font supports that codepoint.


The description of the cmap subtables in the OpenType spec does not explicitly say (as far as I can tell) whether it is expected that the base character of a UVS will always be present in its own right in the Unicode cmap (as Firefox is assuming), although the examples given seem to consistently follow this pattern.


Arguably, perhaps Firefox should also scan the format-14 (platform 0, encoding 5) subtable and consider any character codes found there to be "supported" by the font. But if they're only supported as part of a variation sequence and not as individual codepoints, I'm not sure offhand if that might cause other issues.



    
    

    
  
How about my 3rd attempt:


https://mozfontbug.pw6.de/index3.html


According to ttx (which I guess is the tool you are using), the "black rightwards arrow" is now in format-4 and format-12. Still, it doesnt work with the UVS. Proves to be really difficult to find a delicious font meal for our beloved fox that is appreciated.



    
    

    
  
(In reply to r from comment #12)




Still, it doesnt work with the UVS.




That is because the variation selector 0xFE0F explicitly asks for an emoji-style rendering of the character; but your font doesn't include any of the color-glyph technologies that Firefox supports. So it's seen as being a text-style-only font, and when the Unicode content explicitly asks for emoji-style rendering, Firefox will try to find and prefer a font that supports this.


If your font actually provided a color glyph for the arrow, it would be used there.



    
    

    
  
Okay, thank you for the detailed explanation. I understand now and if that's what Mozilla has decided - then so be it. Nevertheless, I have to say I find this a really, really weird decision. It makes it impossible to have a simple icon font and assign arbitrary well known emojis to it that resemble the real icon. You can assign a (colored!) baby bottle, you can assign a (colored!) red circle. But you cannot assign a blue right arrow. So you always have to watch out that you do not use these special emojis which just look like any other emojis. Other browsers handle this much better IMHO and I find it completely counter-intuitive. I specify a webfont for a reason. I think there's just too much weight given the 0xFE0F. 0xFE0F means "emoji" style, yes, but not colored emoji. Anyway, it's good to finally know the root cause of this. Thank's again for helping me getting to the bottom of this - really appreciate it.



    
    

    
  
I'm a bit confused by "you cannot assign a blue right arrow". Yes, you can: if you create a colored font with a blue right arrow, that will work fine. Your icon font here doesn't have a blue right arrow in it. It has a monochrome right arrow that will display in whatever the current foreground color is, just like any normal text-style glyph. And that's what doesn't get used when the text specifically asks for emoji-style presentation.



    
    

    
  
Well, what I meant is: If you look at e.g. the WhatsApp emoji selector you see different emojis: Baby bottles, flags, red circles and blue right arrows. They all look alike. Now you create an icon font. Not a colored font. There are many tools available that create monochrome(!) fonts. I'm unsure at the moment if the WOFF file format even supports colored fonts, for example. FontForge doesnt support colored fonts. And the SVG2TTFS generator I am using is based on FontForge (https://github.com/pteromys/svgs2ttf). So it's actually quite difficult at the moment to generate these colored fonts - especially in a scripted environment. Now, if you want to assign above-mentioned various emojis to your icons you cannot do that for some - seemingly arbitrary - emojis which happen to consist of this variation selector.


Now, it obviously depends on how you look at the "variation selector" idea in general. Why exactly has it been introduced? Well, my take on this is: It makes sense to not occupy unicode codepoints needlessly. If we already have a right pointing arrow, why not add a special "token" (FE0F) to specify a different variation. It happend for some emojis where there was already a basic monochrome version. It didn't happen for others like the red circle. From a user point of view or even from a normal developer's POV who isn't studying unicode manuals during their leisure time, this is not really obvious. Personally, I was quite surprised to find out that the blue right arrow is constructed fundamentally different than a red circle. It shouldn't matter that much in practice.


For me, the combination of those two unicodes just mean "blue arrow" like the single unicode means "red circle". And just like I can assign a monochrome icon to the red circle without any further complications, I'd expect that I can override the blue arrow with a web font icon - even, yes, if it's not actually blue.


But, well, I understand your approach - just find it quite weird and TBH not really developer-friendly. As web developers we just want to create nice icon fonts and assign some emojis to them for better lookup. And then are completely puzzled why some of them don't show - and only don't show in Firefox. The fact that a colored font is required to do this greatly restricts the number of tools you have available to generate these fonts.



    
  
Severity: normal → S3

          You need to log in
          before you can comment on or make changes to this bug.
        






  

    
  







  

    

      
Attachment

      

      
      
      
      
    

    

      

        

          

            
General

            

              Creator: 



            

            
Created: 

            
Updated: 

            
Size: