Closed Bug 1231701 Opened 9 years ago Closed 8 years ago

Ship an emoji font on Windows XP-7

Categories

(Firefox :: General, defect, P2)

Product:
Firefox
Component:
General
Platform:
Unspecified
Windows
Type:
defect

Tracking

()

Status:
RESOLVED FIXED
Milestone:
Firefox 50
Tracking Flags:
Tracking Status
firefox45 --- affected
relnote-firefox --- 50+
firefox50 --- fixed

People

(Reporter: phlsa, Assigned: jfkthame)

References

Details

(Whiteboard: [qx][This issue is fixed and is not a forum post on Emoji support in Firefox])

Attachments

(8 files)

Bug 1231701 - Ship our own variant of EmojiOne on Windows and Linux,
58 bytes, text/x-review-board-request
Dolske
: review+
glandium
: review+
gerv
: review+
jfkthame
: review+
Details
VirtualBox_IE6 - WinXP_28_04_2016_16_19_48.png
58.95 KB, image/png
Details
VirtualBox_Windows 7_28_04_2016_16_37_00.png
178.98 KB, image/png
Details
Emoji One on Ubuntu
174.26 KB, image/png
Details
VirtualBox_IE6 - WinXP_05_05_2016_11_05_23 (with ClearType).png
126.33 KB, image/png
Details
Firefox with bundled EmojiOne on WinXP
186.37 KB, image/png
Details
fontconfig
902 bytes, text/plain
Details
Add valgrind suppression for uninitialized memory access and leak in old fontconfig
1.48 KB, patch
n.nethercote
: review+
Details | Diff | Splinter Review 
Windows XP has close to no native support for emoji. Even on Windows 7 the set is very limited and not very appealing.

We have a very well developed emoji font from Firefox OS. We should bundle that font with the browser on those platforms.
The (complete?) TTF in https://github.com/mozilla/fxemoji/tree/gh-pages/dist/FirefoxEmoji is 990K. That's a notable fraction of the current full installer (44MB), but mostly moot with the stub-installer.

I assume we wouldn't want to include this with OS X (which has excellent Emoji support). Not sure what the state of affairs is like on Linux.
Before we start down this path, we should check with the platform folks to see if there are any lurking gotchas we'll need to consider. Such as the issues we ran into with L10N and Fira in Reader View (which I don't think will apply here) or the issue in bug 1239380 (which does make me wonder).

Also, with FirefoxOS itself winding down, I'd like to understand if FirefoxEmoji is going to have anyone maintaining it (and if not, how we might be able to keep it alive). It's not a huge deal, but with new Emoji being added on a regular basis, it'll fall behind.
Flags: needinfo?(jfkthame)
As far as I'm aware, bundling FirefoxEmoji for older Windows systems should work fine. The size is obviously one issue to consider.

I suspect we'd want to load the font privately within Firefox rather than install it to the Windows system fonts directory (as often happens with "bundled fonts", e.g. those delivered with MSOffice or similar products). This is because FirefoxEmoji won't work well in other apps: the OS font-rendering APIs on these Windows systems lack color-font support, so the glyphs will mostly appear as ugly black blobs. Better not to expose the font to other apps at all.

This in turn means that we'd need to make Firefox look for fonts within its own resources directory and activate (privately) at startup. We have code for that, which is currently enabled on desktop only for Mulet builds, so it's easy to do, but there will be some (small) startup cost; we should confirm just how small.

The issue in bug 1239380 could occur, though no worse than elsewhere; it's not specific to any particular platform. Given that in general those older platforms simply don't have any font coverage for the emoji characters, I think overall this would be an improvement, and "unexpected" results should be minimal.

I'd be more concerned about the future of FirefoxEmoji, and whether it's going to be maintained as new emoji characters are added to the Unicode repertoire. AIUI, the design was originally done by a contractor, and that contract is presumably long-since finished. So we'd need to have some kind of plan in place here, if we want to keep abreast of the standard.
Flags: needinfo?(jfkthame)
Oh, another question to consider is what we'd do on newer Windows systems that already include Segoe UI Emoji. Do we bundle FirefoxEmoji and prefer it (via font prefs) so as to get "our" look across all Windows systems, or do we defer to the system's emoji font so as to harmonize with the rest of the OS?

I'd generally favor the latter -- I think following a platform's "native" look & feel, which would include the default visual appearance of emoji, is usually the right thing to do -- but that will raise issues of font prefs, as we don't currently have a mechanism to ship with different defaults per Windows version.

As for Linux: I think many distros probably offer Noto Color Emoji these days, so we may not need to do anything there. Though we should check whether/how well it works in Firefox; I haven't looked into this lately, and there have been major changes in the Linux font back-end....
Blocks: 1248735
No longer blocks: fx-qx
Bug 1194338 may or may not be relevant here -- Android is now downloading fonts post-install, to keep the APK size small. Per previous comments I don't think the size is a significant issue for Desktop, but thought I'd mention it.
Here is my recommended approach:

1.
```
if test "$OS_ARCH" = "WINNT"; then
  MOZ_BUNDLED_FONTS=1
fi
```
in browser/confvars.sh (feature implemented in bug 998844)

2. Check-in FirefoxEmoji.ttf into ./browser/fonts/
3. Figure out how moz.build works and make the ttf included in the package under ./fonts/
4. Patch the [1] like bug 1193481 so "Firefox Emoji" would get picked up on Windows builds when "Segoe UI Emoji" does not exist.

[1] https://dxr.mozilla.org/mozilla-central/rev/fc15477ce628599519cb0055f52cc195d640dc94/gfx/thebes/gfxWindowsPlatform.cpp#710

I am not sure about 4 since it is unclear to me if a bad version of Segoe UI Emoji is shipped on Windows 7, and whether or not we want to overwrite it per comment 0.
With the pivot away from shipping FirefoxOS phones (which were the driver for developing the Firefox Emoji font), I'm concerned that it will languish unmaintained, and become increasingly out-of-date as the Unicode emoji repertoire grows.

We might be better-advised to consider using an actively-maintained emoji collection such as "Emoji One", unless we're going to put some ongoing resources into maintaining the Firefox Emoji set.
(In reply to Jonathan Kew (:jfkthame) from comment #7)
> With the pivot away from shipping FirefoxOS phones (which were the driver
> for developing the Firefox Emoji font), I'm concerned that it will languish
> unmaintained, and become increasingly out-of-date as the Unicode emoji
> repertoire grows.
> 
> We might be better-advised to consider using an actively-maintained emoji
> collection such as "Emoji One", unless we're going to put some ongoing
> resources into maintaining the Firefox Emoji set.

That would be a product/UX question them. The original designer of Firefox Emoji was contracted by us (I am sorry -- I lost her name in my inbox) by the Firefox OS visual team.

Madhava, is Firefox Emoji something your team will be looking forward to maintain in the future?
Flags: needinfo?(madhava)
(In reply to Tim Guan-tin Chien [:timdream] (please needinfo) from comment #6)
> I am not sure about 4 since it is unclear to me if a bad version of Segoe UI
> Emoji is shipped on Windows 7, and whether or not we want to overwrite it
> per comment 0.

This shouldn't be a problem according to https://msdn.microsoft.com/en-us/goglobal/bb688099.aspx#W7

It says black-and-white Emoji font was shipped as "Segoe UI Symbols", not "Segoe UI Emoji". So we can simply get away from it by avoiding the font name.
(In reply to Tim Guan-tin Chien [:timdream] (please needinfo) from comment #8)
> (In reply to Jonathan Kew (:jfkthame) from comment #7)
> > With the pivot away from shipping FirefoxOS phones (which were the driver
> > for developing the Firefox Emoji font), I'm concerned that it will languish
> > unmaintained, and become increasingly out-of-date as the Unicode emoji
> > repertoire grows.
> > 
> > We might be better-advised to consider using an actively-maintained emoji
> > collection such as "Emoji One", unless we're going to put some ongoing
> > resources into maintaining the Firefox Emoji set.
> 
> That would be a product/UX question them. The original designer of Firefox
> Emoji was contracted by us (I am sorry -- I lost her name in my inbox) by
> the Firefox OS visual team.

The designer was Sabrina Smelko, http://sabrinasmelko.com/portfolio-item/emojis/
(In reply to Tim Guan-tin Chien [:timdream] (please needinfo) from comment #10)
> (In reply to Tim Guan-tin Chien [:timdream] (please needinfo) from comment
> #6)
> > 4. Patch the [1] like bug 1193481 so "Firefox Emoji" would get picked up on
> > Windows builds when "Segoe UI Emoji" does not exist.

Correction: Segoe UI Emoji would need to be a superset so Firefox Emoji will *never* show up on Window 8.1+, if that's the desired effect.
(In reply to Tim Guan-tin Chien [:timdream] (please needinfo) from comment #6)
> 3. Figure out how moz.build works and make the ttf included in the package
> under ./fonts/

FINAL_TARGET_FILES.fonts += ['FirefoxEmoji.ttf']
Assignee: nobody → timdream
Status: NEW → ASSIGNED
(In reply to Tim Guan-tin Chien [:timdream] (please needinfo) from comment #13)
> (In reply to Tim Guan-tin Chien [:timdream] (please needinfo) from comment
> #6)
> > 3. Figure out how moz.build works and make the ttf included in the package
> > under ./fonts/
> 
> FINAL_TARGET_FILES.fonts += ['FirefoxEmoji.ttf']

This should be 

FINAL_TARGET_FILES['...'].fonts += ['FirefoxEmoji.ttf']

I've also realized there is no Emoji font shipped in Ubuntu 12. I wonder if we want to ship the font too on Linux builds.
(In reply to Tim Guan-tin Chien [:timdream] (please needinfo) from comment #14)
> I've also realized there is no Emoji font shipped in Ubuntu 12. I wonder if
> we want to ship the font too on Linux builds.

Yes, please.
(In reply to Jonathan Kew (:jfkthame) from comment #7)
> We might be better-advised to consider using an actively-maintained emoji
> collection such as "Emoji One", unless we're going to put some ongoing
> resources into maintaining the Firefox Emoji set.

All for it, since it's a very comprehensive set, but we'd need to engineer a build system to turn that set into fonts for Windows and Linux.
Patch on Windows XP -- Emoji renders correctly.
This is how the patch looks like on Windows 7.

I am a bit confused by why logic in [1] selects Segoe UI Symbol over Firefox Emoji -- I can see there is no VS15 (text-style) on that page so they should be shown in Emoji. Maybe I misread the `if`s?

[1] https://dxr.mozilla.org/mozilla-central/rev/fc15477ce628599519cb0055f52cc195d640dc94/gfx/thebes/gfxWindowsPlatform.cpp#717-731
(In reply to Tim Guan-tin Chien [:timdream] (please needinfo) from comment #19)
> Created attachment 8746450 [details]
> VirtualBox_Windows 7_28_04_2016_16_37_00.png
> 
> This is how the patch looks like on Windows 7.
> 
> I am a bit confused by why logic in [1] selects Segoe UI Symbol over Firefox
> Emoji -- I can see there is no VS15 (text-style) on that page so they should
> be shown in Emoji. Maybe I misread the `if`s?
> 
> [1]
> https://dxr.mozilla.org/mozilla-central/rev/
> fc15477ce628599519cb0055f52cc195d640dc94/gfx/thebes/gfxWindowsPlatform.
> cpp#717-731

And if you look at "Emoji Wrap Issue #1 is out!" line of the page, Firefox Emoji are correctly selected; also the tab bar. This is very strange and I would need some help (& proofread) here.

Let's wait for product/UX response before working on this further. It's been fun.

(The Windows builds are generated by
https://treeherder.mozilla.org/#/jobs?repo=try&revision=d69262d35d183ec9f0aaf8c55ad509f8e8d7d4d2&selectedJob=20078255 if people want to take a look at it.)
(In reply to Mike de Boer [:mikedeboer] from comment #17)
> All for it, since it's a very comprehensive set, but we'd need to engineer a
> build system to turn that set into fonts for Windows and Linux.

I don't think you need to convert the font format as long as the font is load and rendered by Gecko. It displays fine on Windows, as shown.
(In reply to Tim Guan-tin Chien [:timdream] (please needinfo) from comment #19)
> Created attachment 8746450 [details]
> VirtualBox_Windows 7_28_04_2016_16_37_00.png
> 
> This is how the patch looks like on Windows 7.
> 
> I am a bit confused by why logic in [1] selects Segoe UI Symbol over Firefox
> Emoji

The page explicitly uses

  font-family: "Apple Color Emoji","Segoe UI Emoji","NotoColorEmoji","Segoe UI Symbol","Android Emoji","EmojiSymbols";

so it'll choose Segoe UI Symbol and never hit Gecko's fallback code path.
Verdi, is maintaining a distinct Firefox Emoji something we are willing to commit? Please comment 7. If we want to move away from it, I can prepare a patch shipping the Emoji One font instead.

It's probably a problem between investment v.s. newsbuzz v.s. product experience.
Flags: needinfo?(mverdi)
(In reply to Jonathan Kew (:jfkthame) from comment #22)
> The page explicitly uses
> 
>   font-family: "Apple Color Emoji","Segoe UI Emoji","NotoColorEmoji","Segoe
> UI Symbol","Android Emoji","EmojiSymbols";
> 
> so it'll choose Segoe UI Symbol and never hit Gecko's fallback code path.

Ah, thank you for spotting that. I misinterpret the "system" label on the DevTools font inspector.
(In reply to Tim Guan-tin Chien [:timdream] (please needinfo) from comment #21)
> (In reply to Mike de Boer [:mikedeboer] from comment #17)
> > All for it, since it's a very comprehensive set, but we'd need to engineer a
> > build system to turn that set into fonts for Windows and Linux.
> 
> I don't think you need to convert the font format as long as the font is
> load and rendered by Gecko. It displays fine on Windows, as shown.

That's Firefox Emoji, though. I believe Mike's comment was referring to the alternative of the Emoji One collection, which is primarily a collection of images (available as scalable SVG images and as pre-rasterized PNGs in several sizes).
  http://emojione.com/

They also provide font files targeted at Android, OS X, iOS; these are embedded-PNG-bitmap fonts, supported on those OSes but not universally.

However, there's also someone packaging the collection as an SVG-in-OpenType font. That should work in Gecko across all platforms, AFAIK.
  https://github.com/eosrei/emojione-color-font

Another option -- possibly lighter-weight than SVG-in-OT -- would be to repackage the glyphs as a layered color font using the COLR/CPAL tables; that's how FirefoxEmoji is done. But I haven't seen any existing project doing that, so we might need to come up with that build system ourselves if we want to pursue it.
(In reply to Tim Guan-tin Chien [:timdream] (please needinfo) from comment #23)
> Verdi, is maintaining a distinct Firefox Emoji something we are willing to
> commit? Please comment 7. If we want to move away from it, I can prepare a
> patch shipping the Emoji One font instead.
> 
> It's probably a problem between investment v.s. newsbuzz v.s. product
> experience.

BTW you can find the full set of Firefox Emoji at https://mozilla.github.io/fxemoji/

(In reply to Jonathan Kew (:jfkthame) from comment #25)
> However, there's also someone packaging the collection as an SVG-in-OpenType
> font. That should work in Gecko across all platforms, AFAIK.
>   https://github.com/eosrei/emojione-color-font

Sounds like a good plan also!
(In reply to Tim Guan-tin Chien [:timdream] (please needinfo) from comment #26)
> (In reply to Jonathan Kew (:jfkthame) from comment #25)
> > However, there's also someone packaging the collection as an SVG-in-OpenType
> > font. That should work in Gecko across all platforms, AFAIK.
> >   https://github.com/eosrei/emojione-color-font
> 
> Sounds like a good plan also!

The downside of this option is that the EmojiOneColor-SVGinOT.ttf font file is around 6.5MB. (That would compress by 50% or so in a .zip download, I guess, but it's still an awful lot to add to the Firefox package.)

My guess is that converting the glyphs to COLR/CPAL layered format would come out considerably smaller, but I don't think there's any way to know just how much, short of actually doing the work.
(In reply to Jonathan Kew (:jfkthame) from comment #27)
> The downside of this option is that the EmojiOneColor-SVGinOT.ttf font file
> is around 6.5MB. (That would compress by 50% or so in a .zip download, I
> guess, but it's still an awful lot to add to the Firefox package.)

It's <3MB zipped. (See https://github.com/eosrei/emojione-color-font/releases), which looks like a less costly option than engineering the COLR/CPAL thing, no?
OK, so just under 3MB. That's still a 7+% increase in the size of the Windows package, which is pretty substantial. (Especially sad for the users on Windows 8.1 and 10 who won't see any benefit from it, as they've got Segoe UI Emoji anyway.)

Yes, using the EmojiOneColor-SVGinOT.ttf would be less costly up-front than engineering a COLR/CPAL version. Whether the engineering cost of the latter would be worthwhile over the longer term is another question, though. It would probably be more performant at runtime, in addition to reducing the package footprint. But by how much? That's unknown....
(In reply to Tim Guan-tin Chien [:timdream] (please needinfo) from comment #23)
> Verdi, is maintaining a distinct Firefox Emoji something we are willing to
> commit? Please comment 7. If we want to move away from it, I can prepare a
> patch shipping the Emoji One font instead.

The Firefox OS emoji font is beautiful and doesn't add as much to the Firefox download so I'm inclined to go with that. What is involved with maintaining this vs. EmojiOne? Is it mainly adding new illustrations for new emojis?
Flags: needinfo?(mverdi) → needinfo?(timdream)
(In reply to Verdi [:verdi] from comment #31)
> (In reply to Tim Guan-tin Chien [:timdream] (please needinfo) from comment
> #23)
> > Verdi, is maintaining a distinct Firefox Emoji something we are willing to
> > commit? Please comment 7. If we want to move away from it, I can prepare a
> > patch shipping the Emoji One font instead.
> 
> The Firefox OS emoji font is beautiful and doesn't add as much to the
> Firefox download so I'm inclined to go with that. What is involved with
> maintaining this vs. EmojiOne? Is it mainly adding new illustrations for new
> emojis?

Other than the new images I don't think there are a lot of efforts. But I might be wrong and :jfkthame can correct me.
Flags: needinfo?(timdream)
Flags: needinfo?(mverdi)
Flags: needinfo?(jfkthame)
There are basically two things that need doing, I guess: designing new images as things get added to the emoji character repertoire in Unicode; and building the collection of images into a colored font. For FxOS, that was done by Pavel; I don't know if he's available on an ongoing basis to re-do/maintain that process as needed, or if it's something that could be made more automated.

Ideally, we'd have a repository where a designer could check in new images as needed, with some kind of metadata (even just a filename convention) specifying what character they correspond to, and there'd be a build process that automatically generates an updated FirefoxEmoji.ttf font. But I don't know how close we are to being able to do that... AFAIK, so far it's been a manually-run process.
Flags: needinfo?(jfkthame) → needinfo?(pivanov)
Comment on attachment 8746430 [details]
Bug 1231701 - Ship our own variant of EmojiOne on Windows and Linux,

Review request updated; see interdiff: https://reviewboard.mozilla.org/r/49427/diff/1-2/
Attachment #8746430 - Attachment description: MozReview Request: Bug 1231701 - Ship Firefox Emoji font v1.7.9 on Windows → MozReview Request: Bug 1231701 - Ship Firefox Emoji font v1.7.9 on Windows and Linux, r=glandium, r=dolske, r=jfkthame
Attachment #8746430 - Flags: review?(mh+mozilla)
Attachment #8746430 - Flags: review?(jfkthame)
Attachment #8746430 - Flags: review?(dolske)
Patch updated to include Linux per comment 16. Keep using Firefox Emoji per comment 31. Disclaimer: the size of the font has grown to 1.1MB (uncompressed) since comment 1. 

I was intend to make sure I get a Try pass before asking for review, but I am not sure how I could do that on mozreview (publish a patch w/o actually request for review?). Here is the Try run anyway:

https://treeherder.mozilla.org/#/jobs?repo=try&revision=147220a8a7b6
And it's 596825 bytes, or 582.8KB compressed.

$ zip -9 - FirefoxEmoji.ttf | wc
  adding: FirefoxEmoji.ttf (deflated 48%)
    2360   17482  596825
Comment on attachment 8746430 [details]
Bug 1231701 - Ship our own variant of EmojiOne on Windows and Linux,

Review request updated; see interdiff: https://reviewboard.mozilla.org/r/49427/diff/3-4/
(In reply to Verdi [:verdi] from comment #31)
> The Firefox OS emoji font is beautiful and doesn't add as much to the
> Firefox download so I'm inclined to go with that. What is involved with
> maintaining this vs. EmojiOne? Is it mainly adding new illustrations for new
> emojis?

I agree with the size issue; it's definitely much smaller. However, have you looked at the screenshots to compare the two? The anti-aliasing on these emojis is horrible. I'd say that in its current bitmap-source state, it's not beautiful. The nice thing about the Apple Emoji font is that it's vector-sourced, just like EmojiOne.

Playing devils' advocate for a bit: for an extra 1.9MB we get a complete, scalable font which Fx can render on all platforms and has been used and tested on many more platforms than the Firefox Emoji font. I'm having a hard time believing that the Firefox Emoji font will be extended with different skin tones, which is the most important addition to the emoji code points since its inception.

I'm very proud of having our own emoji font and having a choice to start with, but I'm afraid it can't objectively win from EmojiOne.

And Tim, <3 for picking this up!
https://reviewboard.mozilla.org/r/49427/#review46627

::: gfx/thebes/gfxPlatformGtk.cpp:216
(Diff revision 4)
>      aFontList.AppendElement(kFontFreeSerif);
>      aFontList.AppendElement(kFontDejaVuSans);
>      aFontList.AppendElement(kFontFreeSans);
>  
> +    if (!IS_IN_BMP(aCh)) {
> +        uint32_t p = aCh >> 16;

You can initialise `uint32_t p = aCh >> 16;` outside of this conditional, because it's used below too:
```c
if (!IS_IN_BMP(aCh) && p == 1) {
  aFontList.AppendElement(kFontFirefoxEmoji);
}
...
(p == 2)) {
...
```

It probably doesn't hurt to have the VS15 exception in there, but isn't that mobile/ FxOS specific? I don't think we have another font that can translate these two. But, I'm not an expert at all ;-)
(In reply to Mike de Boer [:mikedeboer] from comment #42)
> It probably doesn't hurt to have the VS15 exception in there, but isn't that
> mobile/ FxOS specific? I don't think we have another font that can translate
> these two. But, I'm not an expert at all ;-)

VS15/VS16 is rather independent of mobile/desktop, characters must (must not) render in color when there is a VS16 (VS15).
What's worth considering is the case when there is no VS in the source text. See:

http://unicode.org/reports/tr51/#Presentation_Style

To implement this by the book we would have to save the entire emoji-data table into Gecko, at least know exactly which characters are text-default and which are emoji-default.
(In reply to Tim Guan-tin Chien [:timdream] (please needinfo) from comment #43)
> To implement this by the book we would have to save the entire emoji-data
> table into Gecko, at least know exactly which characters are text-default
> and which are emoji-default.

... and that's exactly what bug 1032671 is doing.
Comment on attachment 8746430 [details]
Bug 1231701 - Ship our own variant of EmojiOne on Windows and Linux,

Review request updated; see interdiff: https://reviewboard.mozilla.org/r/49427/diff/4-5/
Made some adjustment according to test failures. I however couldn't figure out why this fails, it looks like either Firefox Emoji nor Segoe UI Emoji are applied on our Windows 8 test machines:

https://treeherder.mozilla.org/logviewer.html#?job_id=20199346&repo=try

even if I comment out the Segoe UI Emoji from the code:

https://treeherder.mozilla.org/logviewer.html#?job_id=20208014&repo=try (from https://treeherder.mozilla.org/#/jobs?repo=try&revision=1442bed89145&selectedJob=20208014)

and all while Windows 7 tests passes.
Comment on attachment 8746430 [details]
Bug 1231701 - Ship our own variant of EmojiOne on Windows and Linux,

Review request updated; see interdiff: https://reviewboard.mozilla.org/r/49427/diff/5-6/
Comment on attachment 8746430 [details]
Bug 1231701 - Ship our own variant of EmojiOne on Windows and Linux,

I've overlook the following: all Win8 debug build crashed on this one assertion.

==

05:14:31     INFO -  Assertion failure: int32_t(mRefCnt) > 0 (dup release), at c:/builds/moz2_slave/try-w64-d-00000000000000000000/build/src/gfx/thebes/gfxDWriteFontList.cpp:1730
05:14:31     INFO -  #01: dwrite.dll + 0x61df9
05:14:31     INFO -  #02: dwrite.dll + 0x3954a
05:14:31     INFO -  #03: dwrite.dll + 0x394b9
05:14:31     INFO -  #04: dwrite.dll + 0x39329
05:14:31     INFO -  #05: dwrite.dll + 0x3925d
05:14:31     INFO -  #06: dwrite.dll + 0x39166
05:14:31     INFO -  #07: dwrite.dll + 0x5f323
05:14:31     INFO -  #08: dwrite.dll + 0x87682
05:14:47     INFO -  #09: gfxDWriteFontList::CreateBundledFontsCollection(IDWriteFactory *) [gfx/thebes/gfxDWriteFontList.cpp:1854]
05:14:47     INFO -  #10: gfxDWriteFontList::InitFontList() [gfx/thebes/gfxDWriteFontList.cpp:917]
05:14:47     INFO -  #11: gfxWindowsPlatform::CreatePlatformFontList() [gfx/thebes/gfxWindowsPlatform.cpp:576]
05:14:47     INFO -  #12: gfxPlatformFontList::Init() [gfx/thebes/gfxPlatformFontList.h:103]
05:14:47     INFO -  #13: gfxPlatform::Init() [gfx/thebes/gfxPlatform.cpp:686]
05:14:47     INFO -  #14: gfxPlatform::GetPlatform() [gfx/thebes/gfxPlatform.cpp:510]
05:14:47     INFO -  #15: mozilla::widget::GfxInfo::GetD2DEnabled(bool *) [widget/windows/GfxInfo.cpp:49]
Attachment #8746430 - Flags: review?(mh+mozilla)
Attachment #8746430 - Flags: review?(jfkthame)
Attachment #8746430 - Flags: review?(dolske)
(In reply to Jonathan Kew (:jfkthame) from comment #7)
> With the pivot away from shipping FirefoxOS phones (which were the driver
> for developing the Firefox Emoji font), I'm concerned that it will languish
> unmaintained, and become increasingly out-of-date as the Unicode emoji
> repertoire grows.

I don't think this is a major concern. AIUI we should only be using this on platforms that have no (or very poor) emoji support. So even if our emoji font is never updated again, it's still a strict improvement on what users would otherwise experience, no?


(In reply to Tim Guan-tin Chien [:timdream] (please needinfo) from comment #36)
> And it's 596825 bytes, or 582.8KB compressed.

The font size should not be a major concern either, so long as we're aware of it and not doing anything crazy. 580K doesn't seem like something I'd worry about.

Size has usually been a concern for either consuming storage space or download time. For 600K storage space is a complete non-issue. Download time turns out to not be an issue either -- AIUI the insights team has done a number of experiments (such as artificially increasing the download size), and it doesn't really have and significant effect on our funnel rates. Especially with the stub installer. (Both of these issues matter more on Android, but we're not shipping this there...)
(In reply to Mike de Boer [:mikedeboer] from comment #41)
> (In reply to Verdi [:verdi] from comment #31)
> > The Firefox OS emoji font is beautiful and doesn't add as much to the
> > Firefox download so I'm inclined to go with that. What is involved with
> > maintaining this vs. EmojiOne? Is it mainly adding new illustrations for new
> > emojis?
> 
> I agree with the size issue; it's definitely much smaller. However, have you
> looked at the screenshots to compare the two? The anti-aliasing on these
> emojis is horrible. I'd say that in its current bitmap-source state, it's
> not beautiful. The nice thing about the Apple Emoji font is that it's
> vector-sourced, just like EmojiOne.

I'm confused -- is FirefoxEmoji not vector? The examples on http://mozilla.github.io/fxemoji/dist/FirefoxEmoji/index.html sure seem to scale smoothly to large sizes.

What screenshot are you looking at?

Is there a live page that compares the two?
Flags: needinfo?(mdeboer)
Hey all,
I'm still not sure what I will do in the feature but I will let you know when I know.

FirefoxOS Emoji Font is a vector one ... thanks to Jonathan Kew I made some small scripts who build the font from SVG's files (separate each layer from Icon to SVG). It's not a big problem to redesign the icons ... the only one problem is to prepare the SVG files because the way designers worked icons is not the most optimal to put the icon in to the font. Some of the icons had more than 100 layers which means 100 SVG files and most of the time I've converted some of these icons to 10-20 layers. If we want to continue with new designs we can talk more on this :) just ping me :)
Flags: needinfo?(pivanov)
I just saw the anti-aliasing of the emojis on the VirtualBox/ XP screenshot and didn't like what I saw... whereas the EmojiOne ones seem to lend themselves better to be rasterized on smaller sizes.
The screenshots I looked at are attached to this bug.
Flags: needinfo?(mdeboer)
(In reply to Mike de Boer [:mikedeboer] from comment #52)
> I just saw the anti-aliasing of the emojis on the VirtualBox/ XP screenshot
> and didn't like what I saw... whereas the EmojiOne ones seem to lend
> themselves better to be rasterized on smaller sizes.
> The screenshots I looked at are attached to this bug.

Ah, attachment 8746447 [details]? I'd expect that has more to do with ClearType antialiazing being disabled at certain sizes... Notice how the "Pile of Poo" title text is antialiased (as is the poo emoji next to it), whereas the "Smiling Poop Emoji" text at the bottom is not (nor is the emoji next to it there).

I'd suspect EmojiOne would look the same way.
Here is the same build on WinXP with ClearType turned on. Noted that Windows XP was never shipped with ClearType turned on, so we should not novice users runs Firefox with this configuration.
(In reply to Tim Guan-tin Chien [:timdream] (please needinfo) from comment #15)
> Created attachment 8746430 [details]
> MozReview Request: Bug 1231701 - Ship Firefox Emoji font v1.7.9 on Windows
> and Linux, r=glandium, r=dolske, r=jfkthame
> 
> Review commit: https://reviewboard.mozilla.org/r/49427/diff/#index_header
> See other reviews: https://reviewboard.mozilla.org/r/49427/

Errr, I just noticed you actually have a patch up for review here. But because you put "r=foo" in the description, it didn't actually flag anyone for review. With reviewboard you should just put in "r?foo" so the review flag get set.

I'm... not sure if it's possible to update flags through the Bugzilla UI, or if you need to do a new reviewboard push with updated flags.
Attachment #8746430 - Flags: review+
Comment on attachment 8746430 [details]
Bug 1231701 - Ship our own variant of EmojiOne on Windows and Linux,

https://reviewboard.mozilla.org/r/49427/#review47519

One small nit, but otherwise this is basically a rubberstamp review from me on the /browser changes since that's the simple part of the patch. :)

::: browser/installer/package-manifest.in:50
(Diff revision 6)
>  @RESPATH@/browser/chrome/@AB_CD@@JAREXT@
>  @RESPATH@/browser/chrome/@AB_CD@.manifest
>  @RESPATH@/chrome/@AB_CD@@JAREXT@
>  @RESPATH@/chrome/@AB_CD@.manifest
>  @RESPATH@/dictionaries/*
> +#if defined(XP_WIN) || defined(XP_LINUX) && !defined(ANDROID)

Hmm, I don't think the !defined(ANDROID) is needed. The mobile front-end is in /mobile, so it shouldn't ever even be building this code.
(In reply to Justin Dolske [:Dolske] from comment #55)
> Errr, I just noticed you actually have a patch up for review here. But
> because you put "r=foo" in the description, it didn't actually flag anyone
> for review. With reviewboard you should just put in "r?foo" so the review
> flag get set.
> 
> I'm... not sure if it's possible to update flags through the Bugzilla UI, or
> if you need to do a new reviewboard push with updated flags.

Actually, I cancelled the review because of comment 48 -- crashes on Windows 8 debug.

So far I have not been able to reproduce this with Windows 8.1 32bit not Windows 7 64bit (both in VirtualBox). Will find an actual Windows 8 machine in the office and try to debug on it.
Clearing my NI - looks like all questions are answered
Flags: needinfo?(mverdi)
Flags: needinfo?(madhava)
Comment on attachment 8746430 [details]
Bug 1231701 - Ship our own variant of EmojiOne on Windows and Linux,

Review request updated; see interdiff: https://reviewboard.mozilla.org/r/49427/diff/6-7/
Attachment #8746430 - Flags: review?(mh+mozilla)
Attachment #8746430 - Flags: review?(jfkthame)
https://reviewboard.mozilla.org/r/49427/#review48671

With help from :jfkthame on the two dependent bugs I think this is close to ship :)

Let's make sure everything works when pieced together: https://treeherder.mozilla.org/#/jobs?repo=try&revision=651b53418aeb
https://reviewboard.mozilla.org/r/49427/#review46627

> You can initialise `uint32_t p = aCh >> 16;` outside of this conditional, because it's used below too:
> ```c
> if (!IS_IN_BMP(aCh) && p == 1) {
>   aFontList.AppendElement(kFontFirefoxEmoji);
> }
> ...
> (p == 2)) {
> ...
> ```
> 
> It probably doesn't hurt to have the VS15 exception in there, but isn't that mobile/ FxOS specific? I don't think we have another font that can translate these two. But, I'm not an expert at all ;-)

I would like to keep the diff clean and consistent with where I copy this from (gfxAndroidPlatform.cpp).
https://reviewboard.mozilla.org/r/49427/#review48689

::: gfx/thebes/gfxPlatformGtk.cpp:212
(Diff revision 7)
> +      aFontList.AppendElement(kFontFirefoxEmoji);
> +    }
> +
>      aFontList.AppendElement(kFontDejaVuSerif);
>      aFontList.AppendElement(kFontFreeSerif);
>      aFontList.AppendElement(kFontDejaVuSans);

Many of the BMP smiley on Ubuntu will be kept rendered in b/w with Deja Vu Sans, includes a few SMP emojis like cat faces (U+1F63A).

This is the same behavior on other OSes, as we have not implement Unicode TR51 presentation style selection yet. I believe it will be addressed in bug 1032671.
https://reviewboard.mozilla.org/r/49427/#review48689

> Many of the BMP smiley on Ubuntu will be kept rendered in b/w with Deja Vu Sans, includes a few SMP emojis like cat faces (U+1F63A).
> 
> This is the same behavior on other OSes, as we have not implement Unicode TR51 presentation style selection yet. I believe it will be addressed in bug 1032671.

I was refering to the fact we do not know if an character is text-default or emoji-default per TR51 and we allow font fallback list to run it's course to find the glyth to render.

We surely honors VS15/VS16 (forse text-style or emoji-style).
The remaining question I'd still like us to consider here is whether we should bundle a font based on EmojiOne,[1] or the Firefox Emoji font. In order to be able to do a comparison, I hacked up a script at the weekend to convert the EmojiOne graphics to a layered-color TrueType font; this isn't 100% finished, but should be good enough for testing.

I've pushed a try build at [2], and once the build is ready I'll try running it on a WinXP VM, and get a screenshot of the results if all goes well. (I expect there may well be some test failures on the run, as tests may need to be updated to be aware of the new font.)

In the end, I think the Firefox product/design/UX people should have a look at both Firefox Emoji and EmojiOne builds, and decide which of the fonts we want to deliver. Some of the pros and cons of each of the options, as I see them:

* Firefox Emoji (TTF file: 1,137,116 bytes; zipped: 597,919 bytes)
  + our own distinctive design, matching what we shipped in Firefox OS
  + somewhat smaller font, so less impact on download size
  - no current support for emoji skin-tone modifiers ("racially diverse emoji")
  - not actively developed/maintained, afaik

* EmojiOne (TTF file: 1,717,740 bytes; zipped: 816,369 bytes)
  + high-profile, actively-developed open project (license: CC-BY-4.0)
  + more complete repertoire (e.g. all the Regional Indicator[3] combinations, not just a handful;
    includes newer emoji such as U+1F910 ZIPPER-MOUTH FACE through U+1F9C0 CHEESE WEDGE)
  + support for skin-tone modifiers
  - larger font (the impact on a zipped package will be slightly over 200K more)
  - not under our control

And I'm sure there are other factors I'm not thinking of right now...


[1] http://emojione.com/
[2] https://treeherder.mozilla.org/#/jobs?repo=try&revision=3453a8393020
[3] http://emojipedia.org/flags/
Here's an example of how things look on WinXP if we go with the EmojiOne set.
Another thing to consider is that this is basically legacy support for more or less dying OS versions. I don't think we need to find the perfect emoji font here, nor do we need to commit to updating it forever (or at all even).
Which is to say, Firefox Emoji sounds like a reasonable choice to me, and I think the download size is the most convincing argument there.
(In reply to Dão Gottwald [:dao] from comment #66)
> Another thing to consider is that this is basically legacy support for more
> or less dying OS versions.

What about Linux?
I'd assume Windows7 will be with us for at least another 5 years or so... how much effort we want to keep putting into it may be questionable, but I don't see it going away.

Size may still be the most significant factor here, of course.
(In reply to Mike de Boer [:mikedeboer] from comment #68)
> (In reply to Dão Gottwald [:dao] from comment #66)
> > Another thing to consider is that this is basically legacy support for more
> > or less dying OS versions.
> 
> What about Linux?

I'd guess Noto Color Emoji will become increasingly widespread on mainstream Linux distros, so this issue will be increasingly limited to older (or particularly minimalist) setups.
(In reply to Mike de Boer [:mikedeboer] from comment #68)
> (In reply to Dão Gottwald [:dao] from comment #66)
> > Another thing to consider is that this is basically legacy support for more
> > or less dying OS versions.
> 
> What about Linux?

I would assume that modern and relevant distros have shipped emoji fonts or are about to do so. Is this not the case?

Comment 14 mentions "Ubuntu 12", but I don't know what that is. Ubuntu 12.04 LTS from 2012? I think we can safely ignore that here.
(In reply to Dão Gottwald [:dao] from comment #71)
> Comment 14 mentions "Ubuntu 12", but I don't know what that is. Ubuntu 12.04
> LTS from 2012? I think we can safely ignore that here.

I am sorry, my desktop should have been upgraded to 14.04 LTS already. So, no, Ubuntu does not install an Emoji font for their users by default on 14.04 LTS.
Okay, but then 14.04 is also two years old and I don't think we should optimize for Ubuntu's LTS support cycles. 16.04 seems to have basic support.
Is there a side-by-side comparison page of the two fonts in consideration? (Something akin to http://unicode.org/emoji/charts/full-emoji-list.html) They both seem high-quality, but it would seem useful to be able to compare them (especially to see them at smaller sizes and without the benefit of ClearType).

Jonathan's summary in comment 64 seems compelling to me...

The main goal here is to improve Emoji support on XP/7 -- either font would be adequate.

But the positives for Firefox Emoji do seem somewhat weak -- the design/control is not a major consideration here as a stopgap, and size difference is small enough to not really matter. In contrast, having an actively maintained font that already is more complete and with more features sounds pretty nice.

Not to mention what a huge win it is to have U+1F9C0 CHEESE WEDGE ;-)


Seems like, these kinds of considerations aside, switching from one to the other is a trivial change to the patch here? I think we should proceed doing the review and getting a final UX decision in parallel.
(In reply to Justin Dolske [:Dolske] from comment #74)
> Is there a side-by-side comparison page of the two fonts in consideration?
> (Something akin to http://unicode.org/emoji/charts/full-emoji-list.html)
> They both seem high-quality, but it would seem useful to be able to compare
> them (especially to see them at smaller sizes and without the benefit of
> ClearType).

I've put a page at http://people.mozilla.org/~jkew/tests/emoji-list.html that shows the two fonts side-by-side (as well as the system's default, if available; expect a lot of hexboxes on WinXP). It uses @font-face, so you don't need a bundled-fonts build to view them on various systems & settings; just load it in a standard Firefox.

They're pretty small, for text-size comparison, but you can hover over them to see them bigger.

(From this list, I see that U+1F60E in Emoji One apparently hasn't converted well via my script. So I'll need to look into that, and any other such glitches, if we decide to use that collection.)
Candid question: why not download the font at runtime when there is no emoji font detected on the system, instead of making the installer bigger for everyone while only benefiting a small-ish percentage of them?
(In reply to Jonathan Kew (:jfkthame) from comment #75)
> I've put a page at http://people.mozilla.org/~jkew/tests/emoji-list.html
> that shows the two fonts side-by-side (as well as the system's default, if
> available; expect a lot of hexboxes on WinXP). It uses @font-face, so you
> don't need a bundled-fonts build to view them on various systems & settings;
> just load it in a standard Firefox.
> 
> They're pretty small, for text-size comparison, but you can hover over them
> to see them bigger.

FWIW, at that small size, on a desktop screen, for me, some are really hard to distinguish from others or even tell what they are in the Firefox Emoji font, while that's not true of Emoji One. (Linux)
Comment 64 make sense. Initially I was worry about the effort it would take to convert the Emoji One font to the smaller-size format, given that :jfkthame is already doing it, it shouldn't be a problem anymore. I am happy to convert my patch to use Emoji One once it's ready.

I do have second thought on Linux, though: It was included under the suggestion of :mikedeboer in comment 16. However, the situration compared to Windows <8.0 is completely different:

- Per :dao's comment on comment 73, Ubuntu 16.04 has it's own built-in emoji support.
- As of older versions (or any other distros), Emoji font is a simple |sudo apt-get| away. Linux users should be able to do that easily if s/he really cares about Emoji rendering, while the same approach results ugly blobs for Windows <8.0 users (comment 3).
- Since we will be hard-code the font name in gfxPlatformGtk.cpp, Gecko will prefer the bundled Emoji font over system-supplied Emoji font. This will likely cause problems on new versions of many distros. We avoid that problem on Windows >=8.1 because we know the name of the system Emoji font determinstially -- it's "Segoe UI Emoji".
- This is probably subjective: Linux users hate duplications and package and package management are the thing to solve that very problem right? Ignore that and bundle an Emoji font might not be best treatment to those small but highly vocal Firefox users.
(In reply to Mike Hommey [:glandium] from comment #76)
> Candid question: why not download the font at runtime when there is no emoji
> font detected on the system, instead of making the installer bigger for
> everyone while only benefiting a small-ish percentage of them?

Is there an download-on-demend system inside Firefox? What do we use that for already? Can we share that file across the profiles or it would be a seperate download for every profile created? If there is and there isn't a lot of effort to hook this up with it, I agree it make sense to use that. It sould not make sense to implement that feature from the beginning, just to save 800k of disk space for Windows >=8.1 users.
Although using an automatically downloaded add-on is an option (and I have some experience to share from Valence and ADB helper), the modern facility to use for that would be Kinto:

https://wiki.mozilla.org/Firefox/Go_Faster#III:_Kinto

It is already being used by Fennec to download fonts and by desktop for other things, so it's a perfect fit.
Downloading the fonts for fennec was implemented in bug 1194338 and dependencies; if we can do a similar thing for the emoji font for old Windows versions, that would be great. (Some of the fennec implementation of this is native Java, so it'll need to be re-done in some other form for a desktop version.)

:past, can you point us at examples where we're using this on desktop already?
I believe Mark did the OneCRL work. Mark, can you point to examples in the tree where we contact the Kinto server to fetch the CRL?
Flags: needinfo?(mgoodwin)
(In reply to Panos Astithas [:past] from comment #80)
> https://wiki.mozilla.org/Firefox/Go_Faster#III:_Kinto

I've found a document being said as the main document of OneCRL and title "Go Faster / Security Policy Settings" [1]. This may be the better infra on Desktop we could reuse, although not font related.

[1] https://docs.google.com/document/d/1XCtVh8fDFbg_yPr1EHrx8Ry61lQI4bfVdWelKQfhopc/edit#

The diff between this and download-and-install/update-a-font is probably only the frequency of the update check and the actual font installation?
Comment on attachment 8746430 [details]
Bug 1231701 - Ship our own variant of EmojiOne on Windows and Linux,

https://reviewboard.mozilla.org/r/49427/#review49325

From a build system perspective, this is r=me with the nits fixed. Remains the question of where the file(s) should be in the firefox tree and whether it's desirable to ship it/them at all.

::: browser/fonts/moz.build:7
(Diff revision 7)
> +# vim: set filetype=python:
> +# This Source Code Form is subject to the terms of the Mozilla Public
> +# License, v. 2.0. If a copy of the MPL was not distributed with this
> +# file, You can obtain one at http://mozilla.org/MPL/2.0/.
> +
> +if CONFIG['OS_ARCH'] == 'WINNT' or CONFIG['OS_ARCH'] == 'Linux':

if CONFIG['OS_ARCH'] in ('WINNT', 'Linux')

::: browser/fonts/moz.build:8
(Diff revision 7)
> +# This Source Code Form is subject to the terms of the Mozilla Public
> +# License, v. 2.0. If a copy of the MPL was not distributed with this
> +# file, You can obtain one at http://mozilla.org/MPL/2.0/.
> +
> +if CONFIG['OS_ARCH'] == 'WINNT' or CONFIG['OS_ARCH'] == 'Linux':
> +    FINAL_TARGET_FILES['..'].fonts += [

DIST_SUBDIR = ''
FINAL_TARGET_FILES.fonts += [
]

::: browser/installer/package-manifest.in:51
(Diff revision 7)
>  @RESPATH@/browser/chrome/@AB_CD@.manifest
>  @RESPATH@/chrome/@AB_CD@@JAREXT@
>  @RESPATH@/chrome/@AB_CD@.manifest
>  @RESPATH@/dictionaries/*
> +#if defined(XP_WIN) || defined(XP_LINUX)
> +@RESPATH@/fonts/*

Do we really want to create a new "top-level" directory in the firefox tree? Also, do we really want those files to be outside omni.ja?
Attachment #8746430 - Flags: review?(mh+mozilla) → review+
(In reply to Mike Hommey [:glandium] from comment #85)
> Do we really want to create a new "top-level" directory in the firefox tree?
> Also, do we really want those files to be outside omni.ja?

We "activate" bundled fonts by passing their paths to OS APIs that expect to handle a standard font file (in OpenType/TrueType format); if we package them into omni.ja, we wouldn't be able to do that, but would have to explicitly extract/decompress them first, which would add to the startup impact/memory footprint. So I don't think we want to do that.
(In reply to Panos Astithas [:past] from comment #82)
> I believe Mark did the OneCRL work. Mark, can you point to examples in the
> tree where we contact the Kinto server to fetch the CRL?

The services collections code is in services/common - currently here: https://dxr.mozilla.org/mozilla-central/source/services/common/KintoBlocklist.js <- I know :leplatrem is doing some work around here so there may be some short-term churn.

If you want to use this for fonts, it's probably worth speaking with :leplatrem and Sebastian Kaspari (who did something similar for Fennec).
Flags: needinfo?(mgoodwin)
(In reply to Mark Goodwin [:mgoodwin] from comment #87)
> (In reply to Panos Astithas [:past] from comment #82)
> > I believe Mark did the OneCRL work. Mark, can you point to examples in the
> > tree where we contact the Kinto server to fetch the CRL?
> 
> The services collections code is in services/common - currently here:
> https://dxr.mozilla.org/mozilla-central/source/services/common/
> KintoBlocklist.js <- I know :leplatrem is doing some work around here so
> there may be some short-term churn.
> 
> If you want to use this for fonts, it's probably worth speaking with
> :leplatrem and Sebastian Kaspari (who did something similar for Fennec).

Thanks for the reply. This at least confirmed we would be able to rely on Kinto to sync a file manifest -- looks like there will still be work on creating a firefox-dlc (akin to bug 1197720) to do the actual downloading and installation of the fonts.

There is no point for me to guess here -- I will reach out to :leplatrem and Sebastian Kaspari to find out the details.
(In reply to Tim Guan-tin Chien [:timdream] (please needinfo) from comment #88)
> This at least confirmed we would be able to rely on
> Kinto to sync a file manifest -- looks like there will still be work on
> creating a firefox-dlc (akin to bug 1197720) to do the actual downloading
> and installation of the fonts.
> 
> There is no point for me to guess here -- I will reach out to :leplatrem and
> Sebastian Kaspari to find out the details.

FYI, Sebastian confirmed w/ me in an e-mail confirmed my assessment here. We would need a firefox-dlc that's reacts to the category files synced from Kinto, to download and install the Emoji font.

I am a bit confused on who has the final say here -- I can see the value of firefox-dlc for future use other than the Emoji font, but I am not sure if this should be depend on that.
I'd be more inclined to keep this bug focused on getting an emoji font in.
AFAICT the choice which one of the two options to pick hasn't been made, so that's priority number 1.
A desktop kinto client for this sounds like a perfect fit, but not for this bug, rather a follow-up. We could think about 1) getting this ready and land on 49 branch and prepare for backout before branching, 2) re-land on 50 branch which will grant us plenty of time to work on bells and whistles like deferred installation.
Sounds like a lot of complication when you could just land something that won't, automatically, ride the trains (by adding some checks on NIGHTLY_BUILD)
(In reply to Mike Hommey [:glandium] from comment #91)
> Sounds like a lot of complication when you could just land something that
> won't, automatically, ride the trains (by adding some checks on
> NIGHTLY_BUILD)

I don't see the value of shipping the bundled-to-installer patch to Nightly only... Aurora 49 is still weeks away and this should be able to ride that train to Fx49 release.

It's either we don't ship this patch and start implementing firefox-dlc, or we ship it to release and replace it with firefox-dlc in later versions.
(In reply to Mike de Boer [:mikedeboer] from comment #90)
> I'd be more inclined to keep this bug focused on getting an emoji font in.
> AFAICT the choice which one of the two options to pick hasn't been made, so
> that's priority number 1.

Yes; we need this to finalize any patch, as it determines the name that we'll be adding to the font fallback code, for example.

(Just FTR, I fixed a few issues in the version of the EmojiOne font at http://people.mozilla.org/~jkew/tests/emoji-list.html, which provides a visual comparison of the two options.)

Who's responsible to drive this to a decision?


(In reply to Tim Guan-tin Chien [:timdream] (please needinfo) from comment #92)
> I don't see the value of shipping the bundled-to-installer patch to Nightly
> only... Aurora 49 is still weeks away and this should be able to ride that
> train to Fx49 release.
> 
> It's either we don't ship this patch and start implementing firefox-dlc, or
> we ship it to release and replace it with firefox-dlc in later versions.

I agree. If we think it's not a good idea to ship this form of the patch (primarily because it means we'll be bloating Firefox for users on Win8.1 and later, who don't need the bundled emoji font because they have Segoe UI Emoji as standard), then there's little point in landing it as Nightly-only; we should just put it on hold for now, and get someone working on the deferred-installation support.

Alternatively, we move forward and ship this in Firefox 49, accepting that we're adding somewhere around 600-800K to our installer (and twice that to our post-install disk footprint) for -all- Windows users, whether they need it or not; and we hope to reverse that at some point in the future.

Again, who's going to make this call? To me, these sound like questions for product management rather than us engineers.
(In reply to Jonathan Kew (:jfkthame) from comment #93)
> (In reply to Mike de Boer [:mikedeboer] from comment #90)
> > I'd be more inclined to keep this bug focused on getting an emoji font in.
> > AFAICT the choice which one of the two options to pick hasn't been made, so
> > that's priority number 1.
> 
> Yes; we need this to finalize any patch, as it determines the name that
> we'll be adding to the font fallback code, for example.
> 
> (Just FTR, I fixed a few issues in the version of the EmojiOne font at
> http://people.mozilla.org/~jkew/tests/emoji-list.html, which provides a
> visual comparison of the two options.)
> 
> Who's responsible to drive this to a decision?

This is probably a question again for Verdi. Verdi, there are additional discussions happens around using Emoji One v.s. Firefox Emoji. Thankfully Jonathan has cleared the technical path for the first option (and you can read his argument at comment 64). Would you be able to make a design decision (and design support decision again) on what font we should ship?

> I agree. If we think it's not a good idea to ship this form of the patch
> (primarily because it means we'll be bloating Firefox for users on Win8.1
> and later, who don't need the bundled emoji font because they have Segoe UI
> Emoji as standard), then there's little point in landing it as Nightly-only;
> we should just put it on hold for now, and get someone working on the
> deferred-installation support.
> 
> Alternatively, we move forward and ship this in Firefox 49, accepting that
> we're adding somewhere around 600-800K to our installer (and twice that to
> our post-install disk footprint) for -all- Windows users, whether they need
> it or not; and we hope to reverse that at some point in the future.
> 
> Again, who's going to make this call? To me, these sound like questions for
> product management rather than us engineers.

:dolske, is this something you could make a call on how to move forward? Technical path are mostly figured out, it's just a matter of a balance b/t time-to-ship and compromises. Thanks!
Flags: needinfo?(mverdi)
Flags: needinfo?(dolske)
Ok lets go with EmojiOne. Also make sure we do the proper attribution somewhere http://cl.ly/3B0x190L313d
Flags: needinfo?(mverdi)
(In reply to Verdi [:verdi] from comment #95)
> Ok lets go with EmojiOne. Also make sure we do the proper attribution
> somewhere http://cl.ly/3B0x190L313d

A new section in about:license should be sufficient.
Comment on attachment 8746430 [details]
Bug 1231701 - Ship our own variant of EmojiOne on Windows and Linux,

Let's wait for the availability of the right format of the Emoji One font.
Attachment #8746430 - Flags: review?(jfkthame)
A project is set up on GitHub to convert the EmojiOne font to check-in: https://github.com/mozilla/emojione-colr
The patch as been converted to use EmojiOne instead. There is only one defect in our EmojiOne conversion script; the patch will be ready to land after that.

https://treeherder.mozilla.org/#/jobs?repo=try&revision=b6790e2351a6
Don't forget to add appropriate attribution in about:license.
Comment on attachment 8746430 [details]
Bug 1231701 - Ship our own variant of EmojiOne on Windows and Linux,

Review request updated; see interdiff: https://reviewboard.mozilla.org/r/49427/diff/7-8/
Attachment #8746430 - Attachment description: MozReview Request: Bug 1231701 - Ship Firefox Emoji font v1.7.9 on Windows and Linux, r=glandium, r=dolske, r=jfkthame → MozReview Request: Bug 1231701 - Ship our own variant of EmojiOne on Windows and Linux, r=glandium, r=dolske, r=jfkthame, r=gerv
Attachment #8746430 - Flags: review?(jfkthame)
Attachment #8746430 - Flags: review?(gerv)
Attachment #8746430 - Flags: review?(gerv) → review+
Comment on attachment 8746430 [details]
Bug 1231701 - Ship our own variant of EmojiOne on Windows and Linux,

https://reviewboard.mozilla.org/r/49427/#review53070
Comment on attachment 8746430 [details]
Bug 1231701 - Ship our own variant of EmojiOne on Windows and Linux,

https://reviewboard.mozilla.org/r/49427/#review53088

This seems fine to me, as far as the patch is concerned. It's not clear to me whether we've actually made a decision about shipping in this form for the time being, or holding off until we have a download-on-demand mechanism in place.
Attachment #8746430 - Flags: review?(jfkthame) → review+
Let's go ahead and land this...

* 800K is a ~1.8% increase on the current 42MB full-installer size, which doesn't seem alarming as a one-off (i.e. this is not a slippery slope where we're planning on adding more fonts in every release).

* It would be good to do a follow-up on making this a downloadable component (via Kinto, system addon, or whatever), but I don't think that needs to block a landing for Nightly. This has come up before for other things, and so such a capability will have usefulness beyond this particular font.

* If there's more discussion to be had over the impact on download size, it's easy enough to back this out (or add a build flag to disable it) so it doesn't ship past Beta. This lets us start getting some testing of the actual feature ASAP.

I floated this with a couple product people this morning, and it sounded reasonable to them.


(In reply to Justin Dolske [:Dolske] from comment #49)

> [...]AIUI the insights team has done a
> number of experiments (such as artificially increasing the download size),
> and it doesn't really have and significant effect on our funnel rates.
> Especially with the stub installer.

I did a little follow up on this too. This experiment was run back in Firefox 25 (with a 3MB increase to the then-22MB installer), and so strictly speaking the results may not apply to a larger installer. But I think it does indicate the situation is more complex than every byte being sacred.
Flags: needinfo?(dolske)
Comment on attachment 8746430 [details]
Bug 1231701 - Ship our own variant of EmojiOne on Windows and Linux,

Review request updated; see interdiff: https://reviewboard.mozilla.org/r/49427/diff/8-9/
Attachment #8746430 - Attachment description: MozReview Request: Bug 1231701 - Ship our own variant of EmojiOne on Windows and Linux, r=glandium, r=dolske, r=jfkthame, r=gerv → Bug 1231701 - Ship our own variant of EmojiOne on Windows and Linux,
Attachment #8746430 - Flags: review?(timdream)
Attachment #8746430 - Flags: review?(mdeboer)
The patch is already pushed to try again w/ bug 1276594 here:

https://treeherder.mozilla.org/#/jobs?repo=try&revision=2fdc5dda3e2e

No more action required from me to on this patch. If both bugs lands today we would be able to ship this in Fx49.
Comment on attachment 8746430 [details]
Bug 1231701 - Ship our own variant of EmojiOne on Windows and Linux,

MozReview was drunk...
Attachment #8746430 - Flags: review?(timdream)
Attachment #8746430 - Flags: review?(mdeboer)
Comment on attachment 8746430 [details]
Bug 1231701 - Ship our own variant of EmojiOne on Windows and Linux,

Review request updated; see interdiff: https://reviewboard.mozilla.org/r/49427/diff/9-10/
Attachment #8746430 - Flags: review?(timdream)
Attachment #8746430 - Flags: review?(mdeboer)
Attachment #8746430 - Flags: review?(timdream)
Attachment #8746430 - Flags: review?(mdeboer)
Comment on attachment 8746430 [details]
Bug 1231701 - Ship our own variant of EmojiOne on Windows and Linux,

Review request updated; see interdiff: https://reviewboard.mozilla.org/r/49427/diff/10-11/
Pushed by cbook@mozilla.com:
https://hg.mozilla.org/integration/fx-team/rev/6ba1398db335
Ship our own variant of EmojiOne on Windows and Linux, r=glandium, r=dolske, r=jfkthame, r=gerv
Keywords: checkin-needed
sorry had to back this out for valgrind failures like https://treeherder.mozilla.org/logviewer.html#?job_id=10001453&repo=fx-team
Flags: needinfo?(timdream)
Backout by cbook@mozilla.com:
https://hg.mozilla.org/integration/fx-team/rev/942a42c22de7
Backed out changeset 6ba1398db335 for causing valgrind failures
This looks like it's coming from deep inside fontconfig. Can we find out exactly what fontconfig version is present on the test systems?
Flags: needinfo?(cbook)
Attached file fontconfig — 
talked to garndt from the taskcluster team and got this information. does this help
Flags: needinfo?(cbook) → needinfo?(jfkthame)
Wow. That's _OLD_ .... afaict, fontconfig 2.8.0 dates from November 2009.[1]

I wonder if this still shows up with a newer fontconfig? I'll try to test but it probably won't be until after work-week. My guess is that this is purely a fontconfig issue, and we should either update the version we're using or simply suppress the valgrind error.


[1] https://www.freedesktop.org/software/fontconfig/release/ChangeLog-2.8.0
Flags: needinfo?(jfkthame)
I won't be helpful on that test failure and thankfully Jonathan has already jumped on it. 

(I will be away after work week so I hope this gets resolved this week or someone would have to re-land this for me...)
Flags: needinfo?(timdream)
That gets installed here:

  https://dxr.mozilla.org/mozilla-central/source/testing/docker/centos6-build/system-setup.sh#57

so it's using the "latest", but latest from the ancient CentOS 6 we build on.  So, updating requires finding or building a set of RPMs that can be linked into that environment.
The valgrind failures (comment 113) don't occur on systems with a newer fontconfig (tested locally on Ubuntu 14.04 LTS, which has fontconfig 2.11.0). So I think we should simply suppress them here and move on. Try: https://treeherder.mozilla.org/#/jobs?repo=try&revision=28d0074ba54d
Attachment #8763219 - Flags: review?(n.nethercote)
Assignee: timdream → jfkthame
Attachment #8763219 - Flags: review?(n.nethercote) → review+
https://hg.mozilla.org/integration/mozilla-inbound/rev/d96389755efc2b8d99d80ad408f994e32d20dee5
Bug 1231701 - Add valgrind suppression for uninitialized memory access and leak in old fontconfig. r=njn

https://hg.mozilla.org/integration/mozilla-inbound/rev/232f75ef4109ccfede162d427a224d22bb8d3857
Bug 1231701 - Ship our own variant of EmojiOne on Windows and Linux, r=glandium, r=dolske, r=jfkthame, r=gerv
Blocks: 1280698
https://hg.mozilla.org/mozilla-central/rev/d96389755efc
https://hg.mozilla.org/mozilla-central/rev/232f75ef4109
Status: ASSIGNED → RESOLVED
Closed: 8 years ago
status-firefox50: --- → fixed
Resolution: --- → FIXED
Target Milestone: --- → Firefox 50
\o/ Thanks to all for this wonderful effort. I hope our Windows and Linux users will like this a lot!
Sorry for the spam but a BIG THANK YOU from an avid firefox/emoji user! 
Fantastic choice by implementing the EmojiOne icon set!  
Few are missing on Linux dont know why - http://imgur.com/wybkLpj but thank you everyone!
(In reply to bull500 from comment #124)
> Sorry for the spam but a BIG THANK YOU from an avid firefox/emoji user! 
> Fantastic choice by implementing the EmojiOne icon set!  
> Few are missing on Linux dont know why - http://imgur.com/wybkLpj but thank
> you everyone!

I suspect what you're seeing are (b/w) glyphs that are present in your default font (DejaVu?), so they are used rather than falling back to the EmojiOne set. If you explicitly style the content with 'font-family: EmojiOne Mozilla', you should see the color glyphs.
> I suspect what you're seeing are (b/w) glyphs that are present in your
> default font (DejaVu?), so they are used rather than falling back to the
> EmojiOne set. If you explicitly style the content with 'font-family:
> EmojiOne Mozilla', you should see the color glyphs.

The default font set in Firefox is the factory set - serif; i havn't changed this.   
Fedora system font defaults 
Windows Titles/Interface - Cantarell; 
Documents - Sans Regular; 
Monospace - Monospace  
These are default as well i believe 

On Fedora the font pack for emoji ive installed is gdouros-symbola-fonts   
On ubuntu i've got ttf-ancient-fonts    

Both show the exact same results (as in missing color emoji's)   

Setting to EmojiOne Mozilla from serif does show the color fonts! :) 
I think most Linux's default to serif
Special thanks to bull500 for giving us a heads up.

We’re very grateful to be in the conversation and thrilled to be included within Firefox’s latest nightly build.  One of the concerns brought up was overall file size and make no mistake this is also a major priority for us as well.  We look for every opportunity to fine tune each file on a regular basis, and of course it makes it more difficult when Unicode releases additional emoji code points.  This is one of the major reasons our file is significantly larger than the Firefox OS emoji set (almost twice as many total files).  Our upcoming Summer update will include another significant reduction in overall byte size.

Other goodness as some are aware is how close we pay attn to the other emoji families.  We take great pride in producing original creative works which also interoperate well with Google/Apple.  It’s a delicate task, and something we improve on with each release.  We take every design very seriously, and listen to the input given to our audience primarily through Github and social media.

EmojiOne is officially committed to serving Mozilla anyway we can.  Please reach out to us anytime, and much love to everyone here for your valuable contributions here at Bugzilla.  :bug:
Thanks, Casey (also for your prompt responses when I've filed issues on github).

Regarding size, note that the figures in comment 64 are somewhat out of date; with further work on the conversion to TrueType/COLR format, the .ttf version of EmojiOne that we're currently using is down to 1231096 bytes, which is only a shade larger than FirefoxEmoji was -- and it's a much more complete set, of course.

Many thanks for your awesome work, and for sharing it so freely.
(In reply to bull500 from comment #126)
> > I suspect what you're seeing are (b/w) glyphs that are present in your
> > default font (DejaVu?), so they are used rather than falling back to the
> > EmojiOne set. If you explicitly style the content with 'font-family:
> > EmojiOne Mozilla', you should see the color glyphs.
> 
> The default font set in Firefox is the factory set - serif; i havn't changed
> this.   
> Fedora system font defaults 
> Windows Titles/Interface - Cantarell; 
> Documents - Sans Regular; 
> Monospace - Monospace  
> These are default as well i believe 

If I read the code correctly, this can be fixed in bug 1032671 by use emoji font families (instead of serif) to render all emoji-default characters without VS15.
Thanks :kohei for flagging relnote-firefox.

[Why is this notable]: Adding new feature, allowing Windows <=8.0 and Linux Firefox users to see Emoji characters.
[Suggested wording]: Emoji for everyone! Firefox will use built-in Emoji on operating systems without native Emoji fonts (Windows 8.0 and lower and Liunx).
[Links (documentation, blog post, etc)]: None, but it's probably worthy to write one.

Note: I can't input an Emoji on Bugzilla, but it would be nice if the relnote itself comes with a Emoji smiling face :)
Thank you :timdream for the bugreport link!
Also :timdream i believe EmojiOne can be implemented on websites as well(https://github.com/Ranks/emojione). Would be nice to provide a link to their official site in rel-note/description.
 
It'll be great if this can be ported over to android as well. 
Many external apps bundle with EmojiOne.
Lots of users on older versions of android are left with boxes because the emoji font files are never updated unless you are rooted. 
This could be a win on that platform as well.
(In reply to bull500 from comment #131)
> It'll be great if this can be ported over to android as well. 
> Many external apps bundle with EmojiOne.
> Lots of users on older versions of android are left with boxes because the
> emoji font files are never updated unless you are rooted. 
> This could be a win on that platform as well.

Could you clone another bug under Firefox for Android: General? apk size is a problem though so we would definitely do a dynamic download solution instead. Thanks.
(In reply to Tim Guan-tin Chien [:timdream] (please needinfo) from comment #132)
> (In reply to bull500 from comment #131)
> > It'll be great if this can be ported over to android as well. 
> > Many external apps bundle with EmojiOne.
> > Lots of users on older versions of android are left with boxes because the
> > emoji font files are never updated unless you are rooted. 
> > This could be a win on that platform as well.
> 
> Could you clone another bug under Firefox for Android: General? apk size is
> a problem though so we would definitely do a dynamic download solution
> instead. Thanks.

Done :) 
https://bugzilla.mozilla.org/show_bug.cgi?id=1284531
Asked for another user, is there anyway to disable shipped emoji font?
(In reply to Benjamin Peng from comment #134)
> Asked for another user, is there anyway to disable shipped emoji font?

You could manually delete the font file in the Firefox directory.
This user is complaining that Firefox is overriding their installed emojis.

https://www.reddit.com/r/firefox/comments/4u6ajn/change_emojis/
Depends on: 1292886
See Also: → 1295029
Blocks: 1299556
I saw this announced on OMG! Ubuntu! and I'm super curious: Why did you go with EmojiOne rather than Fxemoji? I read through this thread but I wasn't able to find the actual reason...
Comment 64 mentioned some points to consider; it looks like a decision was made in comment 95, though it's not stated exactly how the various issues were weighed up. I'd guess that greater completeness and ongoing maintenance were probably key factors.
Thanks for shipping this on Linux too, it fills a very empty space. I hoped GNOME would have got us sooner to satisfying system-wide emoji picking/display, but it remains at the design phase for now: https://wiki.gnome.org/Design/OS/Emoji

At least in-house Firefox support means we get a correct viewing experience on the web, and as for picking we can keep using things http://emoji.codes/
Running latest Nightly 53.0a1 (aka firefox-trunk) from https://launchpad.net/~ubuntu-mozilla-daily/+archive/ubuntu/ppa in Ubuntu 16.04 here and there doesn't seem to be any embedded Emoji One font.

I also can not select the font in the font defaults setting in Firefox.

Looks like the font is not there?

Can someone comment?

Haven't checked it with Firefox 50 yet, since Firefox 50 has not yet been released for Ubuntu 16.04...
By the way:

Really good choice to go with Emoji One instead of the FxEmoji.

IMHO FxEmoji should not have been created in the first place, I had already complained about FxEmoji a lot earlier for several reasons, see:

https://bugzilla.mozilla.org/show_bug.cgi?id=1053216#c10

;)

So, it's good to see that FxEmoji has finally been abandoned.

And:

(In reply to Jonathan Kew (:jfkthame) from comment #75)
> I've put a page at http://people.mozilla.org/~jkew/tests/emoji-list.html
> that shows the two fonts side-by-side (as well as the system's default

The Emoji One developers already have put up something like that for a long time and it includes pretty much every Emoji set available, see:

http://emoji.codes/family

;)
With the exception of a few emoji's that look too different, Fxemoji is not bad at all, pretty nice actually. Going with EmojiOne instead of expanding Fxemoji is a very worrisome decision. Less font diversity is not a good thing.
(In reply to jop from comment #144)
> With the exception of a few emoji's that look too different, Fxemoji is not
> bad at all, pretty nice actually. Going with EmojiOne instead of expanding
> Fxemoji is a very worrisome decision. Less font diversity is not a good
> thing.

Wrong. It's a good thing for Emojis.

If you want to know why, then please read this:

http://www.huffingtonpost.com/2014/06/27/emoji-meaning_n_5530638.html

FxEmoji probably is the single best (or rather worst) example for this.
(In reply to nw9165-3201 from comment #145)
> Wrong. It's a good thing for Emojis.
> 
> If you want to know why, then please read this:
> 
> http://www.huffingtonpost.com/2014/06/27/emoji-meaning_n_5530638.html
> 
> FxEmoji probably is the single best (or rather worst) example for this.

I know about that problem, but it's an issue of poor font design, not of font diversity. Well-designed fonts always correspond/convey the same meaning.

Imagine a world were Times New Roman is the only serif font, Helvetica is the only sans-serif font and Consolas is the only monospace font. That's what people are pushing for when it comes to emoji...

Please invest in fixing the few wrongly conveyed emoji's and adding the missing ones in Fxemoji instead of using a ready-made emoji font.
I appreciate the emoji enthusiasm, but this bug isn't the place to be arguing about which emoji font is better.

Also, please file new bugs for any followup issues. This bug is fixed, and we can't track/fix comments added here.
(In reply to Justin Dolske [:Dolske] from comment #147)
> I appreciate the emoji enthusiasm, but this bug isn't the place to be
> arguing about which emoji font is better.
> 
> Also, please file new bugs for any followup issues. This bug is fixed, and
> we can't track/fix comments added here.

Oh, sorry for being unclear, I wasn't trying to argue which font looks better, not at all. My issue with EmojiOne is not the design, it looks fine. My issue with EmojiOne is that it's a ready-made solution. In my opinion it's sort of irresponsible to use a ready-made implementation even though a mostly done homegrown implementation is waiting to be finished and used. Unicode is a standard that needs to be protected, and implementation-monoculture is pretty bad for a standard. And an organization like Mozilla has influence in this.

Should I file a new bug for this concern?
Depends on: 1320089
(In reply to Caspy7 from comment #136)
> This user is complaining that Firefox is overriding their installed emojis.
> 
> https://www.reddit.com/r/firefox/comments/4u6ajn/change_emojis/

Huh. My problem is the exact opposite. I have a few black and white emojis making it through--the very simple ones. I don't like it. 

Sure, if I had a full COLR emoji set, I'd want to use that. But since it's just a few that came with some font (I don't even know which--looks like maybe Deja Vu Sans?), I'd rather have the color emoji. 

At the very least, complete or nearly complete COLR fonts should take precedence over old monochrome fonts. Should I file a bug?
(In reply to Terrell Kelley from comment #149)
> Huh. My problem is the exact opposite. I have a few black and white emojis
> making it through--the very simple ones. I don't like it. 
> 
> Sure, if I had a full COLR emoji set, I'd want to use that. But since it's
> just a few that came with some font (I don't even know which--looks like
> maybe Deja Vu Sans?), I'd rather have the color emoji. 
> 
> At the very least, complete or nearly complete COLR fonts should take
> precedence over old monochrome fonts. Should I file a bug?

Known DejaVu Sans issue, indeed. Other Linux color emoji solutions have been bumping into this as well. Try removing DejaVu from your system and replacing it with BitStream, which is an identical font with less characters. That should fix it.
(In reply to jop from comment #150)
> Known DejaVu Sans issue, indeed. Other Linux color emoji solutions have been
> bumping into this as well. Try removing DejaVu from your system and
> replacing it with BitStream, which is an identical font with less
> characters. That should fix it.

Or if bug 1295029 get implemented in Gecko. Bugzilla is not a user forum so please be considerate and put your voice on the right issue, as what :dolske said in comment 147.

(In reply to jop from comment #148)
> Should I file a new bug for this concern?

Yes, that would belong to a new bug too. However, getting some traction on firefox-dev or dev-platform mailing lists will be more helpful than filing an lonely issue on Bugzilla.
Whiteboard: [qx] → [qx][This issue is fixed and is not a forum post on Emoji support in Firefox]
This bug is where the emoji font backup was implemented. It is thus the correct bug to bring up problems with that implementation. Sometimes this results in the bug being reopened, and other times it results in a spin-out bug. 

If people were never supposed to respond after a bug was marked "Fixed," then all bugs would be automatically closed at that point.

And the mailing lists are always obfuscated, so that regular users don't know how to get to them. I will remedy this:

firefox-dev is https://groups.google.com/forum/#!forum/firefox-dev
dev-platform is https://groups.google.com/forum/#!forum/mozilla.dev.platform
Recently I noticed even on Win 7, Firefox starts to show emoji in Microsoft's Segoe UI Emoji instead of Mozilla's shipped one (EmojiOne?). Did Microsoft port emoji fonts back to Win 7 via Windows updates or something? I never install fonts manually, but my system is up-to-date. 

More weirdly, it only shows as colored in Firefox, but black & white in IE and Chrome. Any explain on this behavior? Just of out curiosity.
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Creator:
Created:
Updated:
Size: