Closed Bug 638335 Opened 13 years ago Closed 8 years ago

Support Japanese emoji

Categories

(Thunderbird :: Message Reader UI, enhancement)

Product:
Thunderbird
Component:
Message Reader UI
Type:
enhancement
Priority:
Not set
Severity:
normal

Tracking

(thunderbird52 fixed, thunderbird53 fixed)

Status:
VERIFIED FIXED
Milestone:
Thunderbird 53.0
Tracking Flags:
Tracking Status
thunderbird52 --- fixed
thunderbird53 --- fixed

People

(Reporter: masayuki, Assigned: Nomis101)

References

(Blocks 1 open bug)

Details

(Keywords: intl)

Attachments

(2 files)

patch
841.54 KB, patch
Paenglab
: review+
Details | Diff | Splinter Review
Combined bug 638335 and bug 1324555 for aurora
2.95 KB, patch
jorgk-bmo
: review+
jorgk-bmo
: approval-comm-aurora+
Details | Diff | Splinter Review 
Unicode 6.0 has Japanese emoji.
http://www.unicode.org/faq/emoji_dingbats.html

It's used in Japanese cell phone emails. So, tb should support it on reading window at least (supporting on editor may be very difficult).

We already have emoticon displaying mechanism on Thunderbird, so, Kato-san guesses we could use it.

For supporting it, we need to work on...

1. Support emoji Unicode mapping.
2. Create our own emoji icons.
3. Make new pref for replacing the emoji characters with images.
4. Make new pref which allows/disallows to animate emoji images. (unnecessary?)
5. Add new font language prefs for emoji character for Mac OS X 10.7 and later and if it's supported, we shouldn't replace the emoji characters with our images.

http://www.appleinsider.com/articles/11/02/27/inside_mac_os_x_10_7_lion_font_book_3_emoji_support.html

Anything else?
FYI: this is different from bug 42017. Emoji is a character in Unicode 6.0 but bug 42017 needs to think what string should be replaced.
Is this a Thunderbird specific thing, or should this support live in core?
(In reply to comment #2)
> Is this a Thunderbird specific thing, or should this support live in core?

If we support emoji only for reading time, this should be fixed on Tb. But if we *could* support this feature on editor (or gfx?), it's better. But I have no idea to support this on editor for now. Emoji glyphs have some colors and may be animated. I.e., if an emoji character is inserted, we need to replace it with an image ideally.

Do you have idea which can support emoji on editor?
(In reply to comment #3)
> (In reply to comment #2)
> > Is this a Thunderbird specific thing, or should this support live in core?
> 
> If we support emoji only for reading time, this should be fixed on Tb. But if
> we *could* support this feature on editor (or gfx?), it's better. But I have no
> idea to support this on editor for now. Emoji glyphs have some colors and may
> be animated. I.e., if an emoji character is inserted, we need to replace it
> with an image ideally.
> 
> Do you have idea which can support emoji on editor?

What about SVG fonts? Now we have a valid use case :)
Depends on: 119490
(In reply to comment #3)
> (In reply to comment #2)
> > Is this a Thunderbird specific thing, or should this support live in core?
> 
> If we support emoji only for reading time, this should be fixed on Tb. But if
> we *could* support this feature on editor (or gfx?), it's better. But I have no
> idea to support this on editor for now. Emoji glyphs have some colors and may
> be animated. I.e., if an emoji character is inserted, we need to replace it
> with an image ideally.
> 
> Do you have idea which can support emoji on editor?

What is emoji?  Is it an encoding system?
(In reply to comment #5)
> What is emoji?  Is it an encoding system?

http://en.wikipedia.org/wiki/Emoji. This is from Unicode 6.0
Simon is probably the best person to talk to about this.
Although we could use Symbola as a last resort, we want multicolor glyphs and animated glyphs for emoji. TTF/OTF/WOFF standard doesn't support multicolor and animations ("Apple Color Emoji" seems to use Apple proprietary extensions).
SVG fonts is the only standard which supports multicolor and animations.
Of course, we could reuse our existing emoticon support if we don't care about emoji support on editors.
Thank you. The approach of using SVG fonts sounds great. But I worry about the performance. How is it?

And if we fallback to symbol fonts, we should add new font language in prefs (but I'm not sure which is needed on UI).
(In reply to comment #10)
> But I worry about the performance. How is it?
I don't know because SVG fonts is not supported yet. Also no other browsers support multicolor and animations because they support it just to pass Acid3 and SVG 1.1 test suite.
I see. I posted a comment to bug 119490. Thanks.
(In reply to bug 119490 comment #120)
> The Apple Color Emoji font is interesting. It uses a sbix table to store PNG
> images for each glyph.
Is the 'sbix' spec open? We can use it instead of SVG fonts if the spec is available.
I don't know how important it is, but it seems to me that using scalable coloured glyphs would be better than bitmaps (whether by SVG Fonts or Robert's proposed OpenType/SVG hybrid).
Mac OS X Lion now includes an Emoji set in its Character Palette. But if you insert this into a TB message, you only get an empty gap. Would be nice if we could use this Emojis from Lion in Thunderbird.
(In reply to Ehsan Akhgari [:ehsan] from comment #2)
> Is this a Thunderbird specific thing, or should this support live in core?

These characters are mostly in use by iOS users because they have an easy way to type them in the system built-in keyboard. It should be implemented in Core and not just in Thunderbird, because these users are also typing them in social networks and chats. 

Very useful way to represent emotions in Twitter without passing the 140 characters limit. ☺ (← not an emoji character but this character can help you understand how useful it could be…)
> ☺ (← not an emoji character but this character can help you understand how useful it could be…)
Actually Apple Color Emoji will display an emoji for this character. Because some emojis are unified to existing Unicode symbols.
https://code.google.com/p/emoji4unicode/source/browse/trunk/generated/EmojiSources.txt
bug 490986 (from 2009) mentions that Gmail supports Emoji and this information should be added to this bug.

Also, bug 715798 talks about similar implementation into Core instead of Thunderbird, which might block this bug from getting implemented.
Blocks: 490986
Blocks: tb-mac
We can use embedded SVG glyphs imstead of SVG fonts.
Depends on: 719286
No longer depends on: 119490
Our OpenType SVG support supports colored glyphs now, and adding support for animation should only be a small amount of additional work.
Windows 8 (and Windows 7 with and update) have support for many of these with the addition of many glyphs to Segoe UI Symbol, which seems to be showing up places (eg. Twitter) through font fallback. 

It would be great if we could ship a free font with the full set across all platforms. MS have proved that they're still very effective in monochrome.

http://www.istartedsomething.com/20120818/microsoft-backports-windows-8-emoji-for-segoe-ui-symbol-to-windows-7/
No need to create own animated emoji icons. These characters can be added to any font that is Unicode encoded. I started adding them to fonts I was building within a week of the Unicode 6.0 announcement, and black-and-white representation (e.g. ☺♡) works just fine in most cases. First step should be just to get font substitution working properly in these ranges. I email little hearts and such to my wife and they don't display in her Tb, even though my font is installed on her Mac.

There's an effort at Wikipedia to support emoji (etc.) by specifying supporting fonts. Lists several other fonts that support these characters.

http://en.wikipedia.org/wiki/User:ChristTrekker/UnicodeSymbol

A later step _could_ be to add graphical icons (presumably color, perhaps animated) to display these chars, but that should be optional. I personally prefer email to be a textual medium, and I strip HTML/RTF email down to text-only.
(In reply to Tim Larson from comment #23)
> No need to create own animated emoji icons. These characters can be added to
> any font that is Unicode encoded. I started adding them to fonts I was
> building within a week of the Unicode 6.0 announcement, and black-and-white
> representation (e.g. ☺♡) works just fine in most cases. First step should be
> just to get font substitution working properly in these ranges. I email
> little hearts and such to my wife and they don't display in her Tb, even
> though my font is installed on her Mac.

Do they display if you set the option "gfx.font_rendering.fallback.always_use_cmaps" to true (default is false) in her Thunderbird? (You can find this via Preferences > Advanced > General > Config Editor.)
Oh, I think you might have to restart Thunderbird after changing that setting before it will take effect.
Set it to true and restarted, still nothing. Where there character should be, appears to be empty space. I can copy-and-paste from there into other applications and the characters appear.
Aha - I think I know what's happening. You're on OS X 10.7 or 10.8, so the Apple Color Emoji font is present, right? Then I think font fallback is choosing that font for the emoji characters (which is reasonable), but due to limitations of the Core Graphics APIs that we use to draw the text, it renders blanks instead of the actual (color) emoji glyphs.

That should be fixed in Firefox (and Thunderbird) 19, as per bug 715798. It's unfortunate that Apple decided to ship a new kind of font without ensuring that it works with -all- the OS X text-drawing APIs, but the fix is on its way.

Presumably if you explicitly format the emoji characters to use your emoji font (which would require using HTML rather than plain-text email), and the recipient has the same emoji font installed, it would probably work. But simply putting the characters into regular text and expecting font fallback to step in is doomed because it finds the Color Emoji font, which renders blanks.
Ah, that makes sense.  The fallback is based on just finding a font that HAS the character, not whether the app has facility for DISPLAYING it. I'll take a look at Tb19 when it's out.
With Thunderbird 19 and above I now can use Apple emoji font in my emails. It sends and receives fine. :-)
But if somebody sends me an iPhone email with an Apple iPhone emoji, than I still get this missing-glyph hexbox instead of the emoji. Will there be also any fix for this?
Could you copy and paste the hexbox here?
Sure. I hope you can see it than.
This is one hexbox from an iPhone emoji:  
And this is also one: 
And this: 
And also this: 

(In the pasted message it now only looks like a square)
These are not Unicode emoji but career-dependent emoji for compatibility with Japanese feature phone. They are not even compatible between careers. We are not going to support non-Unicode emoji.
Assuming the sender's career is Softbank, these emoji means 
Bugzilla dislike non-BMP characters :(
In the source code of the email this emojis are encoded as
=EE=90=8D
=EE=81=96
=EE=81=97
=EE=90=81=20
And it isn't a Japanese phone, it's a normal german iPhone I've got this emails from.
(In reply to Nomis101 from comment #35)
> In the source code of the email this emojis are encoded as
> =EE=90=8D
> =EE=81=96
> =EE=81=97
> =EE=90=81=20
> And it isn't a Japanese phone, it's a normal german iPhone I've got this
> emails from.

Interesting (a bit surprising, even), but the fact remains that these are -not- Unicode emoji characters, they are Private Use Area codepoints (U+E40D, E057, E058, E401), and as such there can be no expectation that they are interoperable between systems. They aren't even supported using these codepoints by the Apple Color Emoji font on OS X.

It turns out that (according to http://en.wikipedia.org/wiki/Emoji, at least), Apple has moved over to using Unicode emoji on iOS in version 5. So I'd guess your iPhone is running an older version of iOS, right?
(In reply to Jonathan Kew (:jfkthame) from comment #36)
> It turns out that (according to http://en.wikipedia.org/wiki/Emoji, at
> least), Apple has moved over to using Unicode emoji on iOS in version 5. So
> I'd guess your iPhone is running an older version of iOS, right?

Interesting to know. It's not my iPhone (I don't have any), it's the iPhone of a friend of mine (and yes, its older). I will ask my brother than, who has an iPhone 5, to send me a message with iOS emojis. I'm curious if I than see the emojis in TB.
My brother has sended me an email with iOS 5 emoticons. I see none of this emojis in Thunderbird 17 (its totally blank). But I can see all emojis in Thunderbird 19 and above. :-) This are good news, I think!
I'm not sure but maybe this project help you. 
https://github.com/Genshin/PhantomOpenEmoji
Going with Firefox OS on a Web page with Emoji and not seeing the characters.
Example: https://miketaylr.com/posts/2013/11/do-the-http-redirect-dance.html
(In reply to Karl Dubost :karlcow from comment #40)
> Going with Firefox OS on a Web page with Emoji and not seeing the characters.
> Example: https://miketaylr.com/posts/2013/11/do-the-http-redirect-dance.html

This is a Thunderbird bug. I don't think it has anything to do with Firefox OS.
(In reply to Jim Porter (:squib) from comment #41)
> (In reply to Karl Dubost :karlcow from comment #40)
> > Going with Firefox OS on a Web page with Emoji and not seeing the characters.
> > Example: https://miketaylr.com/posts/2013/11/do-the-http-redirect-dance.html
> 
> This is a Thunderbird bug. I don't think it has anything to do with Firefox
> OS.

Friday evening… brain fried. My bad. I will open a new bug.
Emoji is supported by Windows 8+ and Mac OS X 10.8+. Although, I'm not sure the Linux's state, can we mark this WONTFIX (for legacy OSes) or something now? Or, do we need some hack to the ISO-2022-JP decoder even on the latest OSes?
Flags: needinfo?(m_kato)
Flags: needinfo?(jfkthame)
Flags: needinfo?(VYV03354)
(In reply to Masayuki Nakano (:masayuki) (Mozilla Japan) from comment #43)
> Emoji is supported by Windows 8+ and Mac OS X 10.8+. Although, I'm not sure
> the Linux's state, can we mark this WONTFIX (for legacy OSes) or something
> now? Or, do we need some hack to the ISO-2022-JP decoder even on the latest
> OSes?

All platforms already support both Microsoft "COLR" format fonts and SVG fonts for color glyph/emoji even if old OSes.  So I don't think that Thunderbird needs additional code for this.
Flags: needinfo?(m_kato)
(If you add Microsoft's Segoe Color Emoji to Linux, Gecko can render color glyph using it)
Yeah, I don't think there's anything more to do here at present. There's adequate support in Gecko for displaying emoji across all platforms. It is dependent on availability of fonts on the host system, so on older OS versions that don't ship emoji fonts, they won't show up (but that's true for any characters, emoji or not, where font support is lacking).
Flags: needinfo?(jfkthame)
Flags: needinfo?(VYV03354)
Thank you everybody. Let's close this bug as WONTFIX for the old OSes.
Status: NEW → RESOLVED
Closed: 9 years ago
Resolution: --- → WONTFIX
Since version 50, Firefox includes the Firefox OS variant of EmojiOne for Windows XP and 7.
See bug 1231701.
It may be a good idea to do the same for Thunderbird, and the work would be much easier now.
Thunderbird team might be interested in this. Feel free to reopen this if somebody work on this.
(In reply to Thomas Bertels from comment #48)
> Since version 50, Firefox includes the Firefox OS variant of EmojiOne for
> Windows XP and 7.
> See bug 1231701.
> It may be a good idea to do the same for Thunderbird, and the work would be
> much easier now.

This would only work on Windows and Linux than. It seems pretty easy to port. I could attach a patch in the next days, but I will not be able to test it, I only have a Mac.
You can test on Windows 7 by downloading VirtualBox here:
https://www.virtualbox.org/wiki/Downloads

And a free Windows 7 VirtualBox image here:
https://developer.microsoft.com/en-us/microsoft-edge/tools/vms/#downloads
Attached patch patch — — Splinter Review 
This is the patch that just ports the changes from Bug 1231701 to Thunderbird. I did not test this patch.
(In reply to Thomas Bertels from comment #51)
> You can test on Windows 7 by downloading VirtualBox here:
> https://www.virtualbox.org/wiki/Downloads
> 
> And a free Windows 7 VirtualBox image here:
> https://developer.microsoft.com/en-us/microsoft-edge/tools/vms/#downloads
Thanks. But I would also need to build Thunderbird with this patch on windows, which I don't know how to do and don't have the time to find out how to do this...
Maybe Richard is interested in moving this forward.
Flags: needinfo?(richard.marti)
Status: RESOLVED → REOPENED
Flags: needinfo?(richard.marti)
Resolution: WONTFIX → ---
Comment on attachment 8819527 [details] [diff] [review]
patch

Review of attachment 8819527 [details] [diff] [review]:
-----------------------------------------------------------------

Because it's a straight port of bug 1231701 and it works, without patch the emojis are monochrome on Win7 and colored with patch, I give r+

Because XP is only supported until TB 52, you should, after landing on c-c, ask for approval-comm-aurora.
Attachment #8819527 - Flags: review+
Assignee: nobody → Nomis101
Status: REOPENED → ASSIGNED
Attachment #8819527 - Attachment description: patch (untested) → patch
(In reply to Richard Marti (:Paenglab) from comment #55)
> Because XP is only supported until TB 52, you should, after landing on c-c,
> ask for approval-comm-aurora.
Good to know, will do that.
Keywords: checkin-needed
Status: ASSIGNED → RESOLVED
Closed: 9 years ago → 8 years ago
Keywords: checkin-needed
Resolution: --- → FIXED
Target Milestone: --- → Thunderbird 53.0
Why did you copy a 1.2 MB file (EmojiOneMozilla.ttf) into comm-central?
You could just have packaged the file in mozilla-central. See SeaMonkey changeset https://hg.mozilla.org/comm-central/rev/f03dbcdc6fe9

(In reply to Mike Hommey [:glandium] from Bug 1299556 comment #4)
> Comment on attachment 8787318 [details] [diff] [review]
> Patch to move EmojiOneMozilla.ttf to /gfx/

> With all that said, I don't see why moving the font makes it easily usable
> by comm-central: You can just add DIRS += ['/browser/fonts'] in any
> moz.build file in comm-central, and that should work.

Note: from comm-central the path is '/mozilla/browser/fonts'
Flags: needinfo?(richard.marti)
Flags: needinfo?(Nomis101)
See Also: → 1280698
How about a follow-up here then?
Depends on: 1324555
Filed bug 1324555 and I am now testing a patch.
Flags: needinfo?(richard.marti)
Flags: needinfo?(Nomis101)
(In reply to Richard Marti (:Paenglab) from comment #55)
> Comment on attachment 8819527 [details] [diff] [review]
> Because XP is only supported until TB 52, you should, after landing on c-c,
> ask for approval-comm-aurora.

We would than need a combined patch with bug 1324555 for comm-aurora.
Combined patch with all check-ins from bug 1324555.

[Approval Request Comment]
User impact if declined: No colored emojis
Testing completed (on c-c, etc.): on c-c
Risk to taking this patch (and alternatives if risky): should be low when all build files are correct now.
Attachment #8820375 - Flags: review?(jorgk)
Attachment #8820375 - Flags: approval-comm-aurora?
Comment on attachment 8820375 [details] [diff] [review]
Combined bug 638335 and bug 1324555 for aurora

Confusing ;-(
I think you got it right. They are the same changes as
https://hg.mozilla.org/comm-central/rev/f03dbcdc6fe9

I'll land this now. Can you please try it out in tomorrow's Earlybird.
Attachment #8820375 - Flags: review?(jorgk)
Attachment #8820375 - Flags: review+
Attachment #8820375 - Flags: approval-comm-aurora?
Attachment #8820375 - Flags: approval-comm-aurora+
Aurora (TB 52):
https://hg.mozilla.org/releases/comm-aurora/rev/7d543345472dddafa30c78e3ea3320619897fe4c
Bug 638335 - Support Japanese emoji (merged patch from bug 638335 and bug 1324555). r=Paenglab,jorgk a=jorgk

In case anyone is interested: Since bug 1316256 has been uplifted to Aurora, the Aurora tree should be green again. That means that TB 52 Earlybird is fully functional for the first time.
Checked the aurora Linux and Windows builds. The fonts directory with the font is there it has to be.
Status: RESOLVED → VERIFIED
Um? I don't think Windows XP can display coloured emoji. Otherwise ++
(In reply to Philip Chee from comment #66)
> Um? I don't think Windows XP can display coloured emoji. Otherwise ++

WinXP doesn't have any built-in support for them, but they should work in Gecko using the bundled color font.
Thunderbird 53 doesn't support Windows XP anyway.
No longer blocks: tb-mac
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Creator:
Created:
Updated:
Size: