Closed Bug 1279925 Opened 8 years ago Closed 7 years ago

IPA Characters Overlap When Entering Text in Firefox 47

Categories

(Core :: Graphics: Text, defect)

47 Branch
defect
Not set
normal

Tracking

()

VERIFIED FIXED
mozilla54
Tracking Status
firefox50 --- wontfix
firefox51 --- wontfix
firefox52 --- verified
firefox53 --- verified
firefox54 --- verified

People

(Reporter: amagidow, Assigned: jfkthame)

References

Details

(Keywords: regression, testcase, Whiteboard: [gfx-noted])

Attachments

(7 files, 1 obsolete file)

Attached image IPAInFirefox47.png
User Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:47.0) Gecko/20100101 Firefox/47.0
Build ID: 20160604131506

Steps to reproduce:

In a text box (input type = 'text'), I tried to enter IPA characters which are NOT combining such as ː (long vowel length, ), or ˈ (primary stress) in the context of other letters, e.g. ʔaːna or ˈʔiħna.

Note that this only occurs in text boxes which use the default variable width font. In text areas, Firefox uses a fixed width font which acts correctly. 

Test in Firefox 47 on Windows 10. Error did not occur in Firefox 46. 


Actual results:

The characters ː and ˈ end up overlapping with the next character that is entered. It looks almost as though they are treated like Right-to-Left characters. 


Expected results:

The characters should show up in the sequence in which they are entered. These are not combining characters and should not overlap.
Could you attach a testcase to the bug report, please.
Flags: needinfo?(amagidow)
Keywords: testcase-wanted
Component: Untriaged → Layout: Text
Product: Firefox → Core
Attached file Testcase HTML
Note the incorrect rendering in the text box, but correct rendering in text area.
Regression range:
https://hg.mozilla.org/integration/mozilla-inbound/pushloghtml?fromchange=d8da9b
c56468f58be505e61e72e78bff1d2b4b61&tochange=2b0aa1cffeeaabb524b9d2936321af18c744
5fbc

Jonathan Kew — Bug 1249861 - Update harfbuzz to release 1.2.2 from upstream. r=jrmuizel
Blocks: 1249861
Status: UNCONFIRMED → NEW
Component: Layout: Text → Graphics: Text
Ever confirmed: true
Flags: needinfo?(amagidow) → needinfo?(jfkthame)
OS: Unspecified → All
Hardware: Unspecified → All
This looks fine here, so might be font dependent (might be a bug in the GDEF table of the font). Can you tell which font us used for rendering the overlapped characters?
I am not sure how to determine which font exactly is used within text boxes. It does not seem to use the font that is set via the Options>Content>Fonts and Colors>Advanced box, which is Arial for san-serif. Based on a comparison with my word processor, I'm fairly certain that it is Verdana. 

This was not a problem prior to the Firefox 47.0 update - I frequently enter IPA data and did not have any problems prior to the 47 update pushed to my computer on 6/10/2016. So I think it's most likely to be a change in Firefox rather than a flaw in the font, unless Firefox has changed the font used since that update.
You can use https://addons.mozilla.org/en-US/firefox/addon/fontinfo/, then select the text, right click then “Show Fonts in Selection” should tell you what fonts are used. The attached document shows fine here in Firefox 47, so it is very likely to be a font dependent issue.
I doubt it will work, because in the testcase, the text is in the placeholder and I'm not sure this add-on is able to parse such a text selection.
Fontinfo does not work to show the font used in the text boxes (the alert dialog is simply blank), as Loic cautioned. Is there a way to actually change the font used for the text boxes? That would make it easier to test whether this is a font issue.
What might work is to use a testcase such as

  data:text/html;charset=utf-8,<div style="font: caption">%CB%88%CA%94i%C4%A7na

which I'm guessing may reproduce the problem.

If so, then go to about:config and set the devtools.fontinspector.enabled option to 'true'; then right-click on the text and choose Inspect Element from the context menu, and select the Fonts panel in the inspector that opens. This should report the font(s) used to display the text.
Flags: needinfo?(jfkthame)
(In reply to amagidow from comment #10)
> Is there a way to actually
> change the font used for the text boxes?

You should be able to do this by simply setting the CSS font-family property on the element; e.g.

   <input type="text" style="font-family:arial" value="...">

Note, however, that if the IPA characters aren't actually present in the font you specify, fallback will take effect and they'll be rendered from some other font. Only a tool like fontinfo (if it works for you; it has some problems with recent Firefox versions) or the Font Inspector in the developer tools (see comment 11) will report what font is actually _used_, as opposed to what font is requested.
The data: trick did not work, it rendered fine.

The developer tools, however, revealed that the font used is Tahoma. Tahoma appears to be the culprit. I altered the test case to use different fonts for the text box, including Arial and Verdana, and those render just fine. As soon as I forced it back to Tahoma, it reproduced the error, and setting the textarea to Tahoma also produces the error. 

Not sure how to proceed - is this an error local only to my computer, or to anyone for whom Firefox defaults to Tahoma in textboxes on web forms? If so, is there a simple resolution?
Perhaps this is related to a previously reported bug with Tahoma:
https://bugzilla.mozilla.org/show_bug.cgi?id=653993
(In reply to amagidow from comment #14)
> Perhaps this is related to a previously reported bug with Tahoma:
> https://bugzilla.mozilla.org/show_bug.cgi?id=653993

Sounds like it may well be a manifestation of the same thing.

There have been some changes in how harfbuzz handles the zeroing of advance width for 'mark' glyphs, and I suspect this is a result of that. As such, the same problem will probably show up in Chrome once the harfbuzz version used there gets updated, for example.

A possibly-relevant harfbuzz change would be:

    commit b3582a8ee82942a931130a83b2867b239f8ca71a
    Author: Behdad Esfahbod <behdad@behdad.org>
    Date:   Wed Feb 10 18:10:15 2016 +0700

        Change default mark advance zeroing behavior from Unicode to GDEF
    
        See thread at:
        https://lists.freedesktop.org/archives/harfbuzz/2016-February/005462.html

Behdad, do we need to reconsider that change? Or maybe blacklist the bad Tahoma GDEF table?
Flags: needinfo?(mozilla)
> Behdad, do we need to reconsider that change? Or maybe blacklist the bad Tahoma GDEF table?

I'm leaning towards blacklisting.  Can you cook a patch?
Font "Cantarell" is probably affected too, see bug 1279875.
As is the Tibetan font is being used in the bug 1279693 example. (I'm not sure what font/version that is.)
Whiteboard: [gfx-noted]
HarfBuzz got resolved in version 1.3.0.
We updated Harfbuzz to version 1.3.0 in bug 1269343. Can anybody confirm that everything renders as expected now in a current Beta release?
Flags: needinfo?(amagidow)
Nevermind, the attached testcase still reproduces for me on Win10 with a current Nightly build.
Flags: needinfo?(amagidow)
Jonathan, is there anything we can do to help get this moving again?
Flags: needinfo?(jfkthame)
Hmm, that's unfortunate. We did blacklist tables in some Tahoma versions, but if (for example) MS has updated the font version, we might still be missing it.

I'll check my current Win10 machine and see what the fonts look like there...
Looks like the Tahoma font was updated (presumably as part of the Anniversary Update), so we need new blacklist entries. Here's a try build that adds the font version as found on my current Win10 machine: https://treeherder.mozilla.org/#/jobs?repo=try&revision=671f69993b5a3d2aa86fb7bb85ecf901fbe7adb9. Ryan, could you confirm whether this fixes the testcase for you? Assuming it helps, I'll take this upstream as well.
Assignee: nobody → jfkthame
Status: NEW → ASSIGNED
Flags: needinfo?(jfkthame)
Indeed, the gets me the same rendering as Chrome for the attached testcase.
I'll submit this as a PR upstream and it'll eventually come back to us in a harfbuzz update, but in the meantime I think we should just patch locally to add this font version to the blacklist.
Attachment #8832905 - Flags: review?(jmuizelaar)
Attachment #8832844 - Attachment is obsolete: true
Attachment #8832905 - Flags: review?(jmuizelaar) → review+
https://hg.mozilla.org/integration/mozilla-inbound/rev/22028faf1e868f3bcd82e0ce674cc733e39570ec
Bug 1279925 - Add Win10AU versions of tahoma.ttf and tahomabd.ttf to the GDEF blacklist in harfbuzz. r=jrmuizel
https://hg.mozilla.org/mozilla-central/rev/22028faf1e86
Status: ASSIGNED → RESOLVED
Closed: 7 years ago
Resolution: --- → FIXED
Target Milestone: --- → mozilla54
Thanks Jonathan! Let's request Aurora/Beta approval on this too.
Flags: needinfo?(mozilla)
Did anyone report to Microsoft about the font error? Or do we have to update the blacklist forever?
The error is in older versions of Windows, which they won't fix ever.  We keep blacklist forever.  Font is fixed in latest versions of Windows.
(In reply to Behdad Esfahbod from comment #32)
> The error is in older versions of Windows, which they won't fix ever.  We
> keep blacklist forever.  Font is fixed in latest versions of Windows.

It's not fixed AFAICT, even on my Win10 Anniversary Update / Insider channel system. A bunch of the spacing marks (U+02xx chars) etc are still assigned GDEF class 3.
Then yes, please report to Microsoft.
Comment on attachment 8832905 [details] [diff] [review]
Add Win10AU versions of tahoma.ttf and tahomabd.ttf to the GDEF blacklist in harfbuzz

Approval Request Comment
[Feature/Bug causing the regression]: some glyphs from the Tahoma font on Win10 render poorly with recent harfbuzz, due to buggy font tables

[User impact if declined]: bad spacing/positioning of some characters in the Tahoma font shipped on Win10

[Is this code covered by automated tests?]: no

[Has the fix been verified in Nightly?]: yes, comment 26

[Needs manual test from QE? If yes, steps to reproduce]: no

[List of other uplifts needed for the feature/fix]: none

[Is the change risky?]: no

[Why is the change risky/not risky?]: trivial patch, just adding another font version to a harfbuzz blacklist

[String changes made/needed]: none
Attachment #8832905 - Flags: approval-mozilla-beta?
Attachment #8832905 - Flags: approval-mozilla-aurora?
Comment on attachment 8832905 [details] [diff] [review]
Add Win10AU versions of tahoma.ttf and tahomabd.ttf to the GDEF blacklist in harfbuzz

Fix the overlapped issue of IPA chars in text box. Aurora53+.
Attachment #8832905 - Flags: approval-mozilla-aurora? → approval-mozilla-aurora+
Comment on attachment 8832905 [details] [diff] [review]
Add Win10AU versions of tahoma.ttf and tahomabd.ttf to the GDEF blacklist in harfbuzz

harfbuzz blacklist addition for beta52, should be in 52.0b5
Attachment #8832905 - Flags: approval-mozilla-beta? → approval-mozilla-beta+
Modified from Uplift Dashboard.
Modified from Uplift Dashboard.
Flagging this for verification, str in Comment 0, testcase in Comment 3.
Flags: qe-verify+
Reproducible only on Windows. Tested on Windows 10 x32 with FF Nighlty 54.0a1(2017-02-09) I can confirm the fix. 
Please note that I tested also on Ubuntu 16.04 and Mac but I couldn't reproduce the issue.
Status: RESOLVED → VERIFIED
I tested this issue on Windows 10 x64 and I’m still able to reproduce this issue on Nightly 54.0a1 (2017-03-02), Aurora 53.0a2 (2017-03-02), Beta 52.0b5. I saw in the preview comment that Ovidiu verified this using Windows 10 x32 and I think this is a specific issue only for Windows x64. Please advise me if will be a good idea to log a new bug for tracking this new issue. 
Jonathan, please tell me your opinion about this. Thanks

Note: I tested on: Firefox 52.0b5, 53.0a2, 54.0a1: Linux 16.04 -->OK
                                                   Windows 10 x32 -->OK
                                                   Windows 10 x64 -->BAD
                                                   Windows 8.1 x64 -->BAD
Flags: needinfo?(jfkthame)
This suggests that your various Windows systems probably have different versions of the Tahoma font, and we'd need to add more entries to the blacklist.

Please confirm the versions of Tahoma and Tahoma Bold on each of your systems, and for the problematic ones, report the font table details shown by running "ttx -t tahoma.ttf" (for the appropriate file name/path) in each case. The TTX tool is part of the fonttools package (https://github.com/fonttools/fonttools).

With those details, we can add the relevant font versions to the harfbuzz list.
Flags: needinfo?(jfkthame)
I download this tool from github and Python, please write more steps to run the "ttx -t tahoma.ttf" command to find the version of tahoma font.
Flags: needinfo?(jfkthame)
> "ttx -t tahoma.ttf"

Sorry, typo... that should have been -l, not -t. In more detail:

It looks like if you have Python installed and the "pip" tool (package manager) is available, you may be able to simply do

    pip install fonttools

in a command window to get the fonttools package. (I haven't tried it but it looks like that's now the standard way.)

Otherwise, in the fonttools package from github there should be a setup.py file, which you can run with

    python setup.py build
    python setup.py install

to prepare and install the package.

Either way, once it is installed you should be able to run the command-line tool "ttx" in a command window. (You might need to start a new command window before that is recognized, if it involved adding a new directory to the PATH for executable commands.)

Once ttx is available (if you enter just "ttx" at a command prompt, it should reply with a long "help" message about options, etc), then you need to locate the Tahoma font files; I expect they'll probably be at "c:\windows\fonts\tahoma.ttf" and "c:\windows\fonts\tahomabd.ttf". In that case, the two commands you need would be:

    ttx -l c:\windows\fonts\tahoma.ttf

    ttx -l c:\windows\fonts\tahomabd.ttf

Each of those should output a listing of font tables along with checksum, size, etc. That's the information we need.

To check the font version, which would also be good to know, locate the font file in the Windows explorer (or in Control Panel / Fonts / "open" the font family to see the individual faces), right-click on it, and choose Properties, then look in the Details tab of the properties panel.
Flags: needinfo?(jfkthame)
I had attached in commnet48 the list about information for this two fonts (tahomaFontInformation.txt), please let me know if you need more information.
Thanks, that's great. I'll see about getting this version added to the list. Could you please confirm exactly which version of Windows this comes from, and what the font version shows. (You can find this by viewing the font within Control Panel / Fonts; it should show a version number at the top of the font preview window. Or you can look in the File Properties panel for the font file.)

Also, you reported problems on both 8.1 and 10 (in comment 44), but this info is only from one of the systems; so please provide the same details from the other as well, so that we can fix both cases. Thanks again!
The list with information that I sent you comes from Win 8.1 x 64, and the font version here is 6.04.
On Win 10 x64, where also have the same problem, the font version is 6.91, and I send you the list of information for this fonts version too.
I verified again this issue, but is still reproduceble on 53.0b10! 
Please tell me if I have to do something else to resolve this issue.
Flags: needinfo?(jfkthame)
Sorry, a workaround for those additional font versions hasn't yet made it into the code. We need to get the blacklist entries integrated into harfbuzz (I've just opened https://github.com/behdad/harfbuzz/pull/459 to address this), and then pulled into the mozilla repository.
Flags: needinfo?(jfkthame)
This issue is still reproduceble in Firefox 54.0b1 on Windows 8.1 x64.
Could I close this bug mark it as verified and open another one to track the remaining issues for 64-bit systems? (in comment 44).
Flags: needinfo?(jfkthame)
Yes, that makes sense. We'll need to take an additional harfbuzz update (or cherry-pick the specific changeset with the added font versions), so that will involve a new gecko patch, distinct from what already landed here. It'll be easier to keep track of what changes are on which branches if we have a separate bug to track that.
Flags: needinfo?(jfkthame)
Thank you for the quick answer. I will file the follow up bug tomorrow. Marking as verified as per above comments.
You need to log in before you can comment on or make changes to this bug.