1880362 - Line break handling with smart quotes is incorrect

Status: VERIFIED FIXED

(Core :: Layout: Text and Fonts, defect)

Description

[harris]

Attachment: Untitled.jpg

User Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:122.0) Gecko/20100101 Firefox/122.0

Steps to reproduce:

On a website I'm drafting, the following string appears:

the NYPD was humiliated by the protests on January 8,” Dunlea said

The length of the column on desktop (macOS 14.1.1 (23B81), FF 122.0.1 (64-bit)) just so happens to cause the line break to fall after the comma, like so:

the NYPD was humiliated by the protests on January 8,
” Dunlea said

Typography rules dictate that the end quote should be kept on the same like as the word that precedes it and in fact using a "dumb" quote, the behavior is correct and it instead breaks:

the NYPD was humiliated by the protests on January
8," Dunlea said

Comment 1

[BugBot [:suhaib / :marco/ :calixte]]

The Bugbug bot thinks this bug should belong to the 'Core::Layout: Text and Fonts' component, and is moving the bug to that component. Please correct in case you think the bot is wrong.

Comment 2

[Daniel Holbert [:dholbert]]

Attachment: testcase 1

Thanks for the bug report. I can reproduce -- here's a testcase. This seems to be specific to cases with a digit followed by a period followed by smart-quotes.

I think this might be related to TYLin's recent segmenter work - I recall there being different linebreaking opportunities when non-ASCII characters are involved in another bug he mentioned there...

TYLin, could you take a look when you're back from vacation?

Comment 3

[Daniel Holbert [:dholbert]]

Attachment: testcase 1

(whoops, this updated testcase fixes my broken monospace-font styling in the previous attachment)

Comment 4

[Daniel Holbert [:dholbert]]

Attachment: screenshot of smart-closequote wrapped by itself after numeral/punctuation

Comment 5

[Daniel Holbert [:dholbert]]

Regression range for getting to our current behavior (in Nightly):
https://hg.mozilla.org/mozilla-central/pushloghtml?fromchange=06273ebf279aad7827a5bd6c083dfb819eb55cb1&tochange=2f07664dbc304adeae1029ead0f4ab014aaa02a8

--> This is from enabling the icu4x segmenter in bug 1719535.

Makoto, maybe you could take a look and double-check if this new behavior is per-spec or not?

Comment 6

[Jonathan Kew [:jfkthame]]

According to https://www.unicode.org/Public/UCD/latest/ucd/LineBreak.txt, both the ASCII double-quote U+0022 and the closing quote U+201D (and other related quote marks) have line-break class QU, which should prevent such a break (https://unicode.org/reports/tr14/#LB19). So the different behavior here seems surprising.

Comment 7

[Daniel Holbert [:dholbert]]

Thanks, let's call this a confirmed regression from bug 1719535 then, pending further investigation.

Comment 8

[Daniel Holbert [:dholbert]]

(And that means Firefox 122 is the first affected release, since this feature was preffed on for release users there in bug 1854032.)

Comment 9

[Makoto Kato [:m_kato]]

Maybe, this is already fixed by ICU4X next version (https://github.com/unicode-org/icu4x/pull/4389)

Comment 10

[j.j.]

I have a similar problem here, first seen with Fx 123 on Android, but probably older:
https://olafschlindwein.de/
Works in Chromium and worked in Firefox before.

There are two blue HTML tables. On narrow screens unwanted line-breaks appear in the right cells.
Breaking eg between 42, and – (–)
Is this the same Issue? Should I file a new bug?

Comment 11

[Makoto Kato [:m_kato]]

(In reply to j.j. from comment #10)

I have a similar problem here, first seen with Fx 123 on Android, but probably older:
https://olafschlindwein.de/
Works in Chromium and worked in Firefox before.

There are two blue HTML tables. On narrow screens unwanted line-breaks appear in the right cells.
Breaking eg between 42, and – (–)
Is this the same Issue? Should I file a new bug?

It will be also fixed by https://github.com/unicode-org/icu4x/pull/4389. After ICU4X 1.5 is released, we will upgrade it into Gecko.

Comment 12

[Ryan VanderMeulen [:RyanVM]]

Doesn't seem like the 1.5 release is going to be happening any time soon. Can we cherry-pick the fix in the mean time?

Comment 13

[Makoto Kato [:m_kato]]

(In reply to Ryan VanderMeulen [:RyanVM] from comment #12)

Doesn't seem like the 1.5 release is going to be happening any time soon. Can we cherry-pick the fix in the mean time?

I guess that we can fix this by updating toml file. I try it.

Comment 14

[Makoto Kato [:m_kato]]

Attachment: Bug 1880362 - Update line segmenter data file. r=TYLin

Update line segmenter data file to match with Unicode 15.0.

Comment 16

[Makoto Kato [:m_kato]]

Attachment: Bug 1880362 - Add web platform tests. r=TYLin

Comment 17

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/45b752a869b1
Update line segmenter data file. r=TYLin
https://hg.mozilla.org/integration/autoland/rev/7ae70412fd03
Add web platform tests. r=TYLin

Comment 18

[Web Platform Test Sync Bot [:wpt-sync] (Matrix: #interop:mozilla.org)]

Created web-platform-tests PR https://github.com/web-platform-tests/wpt/pull/45531 for changes under testing/web-platform/tests

Comment 19

[Atila Butkovits]

Backed out for causing failures at line-breaking-013.html.

Backout link: https://hg.mozilla.org/integration/autoland/rev/ea031eb2c3e43e739089680dc40acc6c34d5b060

Push with failures: https://treeherder.mozilla.org/jobs?repo=autoland&resultStatus=testfailed%2Cbusted%2Cexception%2Cretry%2Cusercancel&revision=7ae70412fd030cfb923bc10ced9b868cb43a4aae&selectedTaskRun=M2NjhL5US2qX0GXg9RQlSA.0

Failure log: https://treeherder.mozilla.org/logviewer?job_id=453472157&repo=autoland&lineNumber=7281

Comment 20

[Jonathan Kew [:jfkthame]]

These are "unexpected passes" in line-breaking-013.html and line-breaking-014.html, which apparently got fixed by the updated segmenter data, so we just need to remove the existing failure annotations.

Comment 21

[Web Platform Test Sync Bot [:wpt-sync] (Matrix: #interop:mozilla.org)]

Upstream PR was closed without merging

Comment 22

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/fcc8ec2728b7
Update line segmenter data file. r=TYLin
https://hg.mozilla.org/integration/autoland/rev/07bb40c2bb54
Add web platform tests. r=TYLin

Comment 23

[Sandor Molnar[:smolnar]]

https://hg.mozilla.org/mozilla-central/rev/fcc8ec2728b7
https://hg.mozilla.org/mozilla-central/rev/07bb40c2bb54

Comment 24

[Web Platform Test Sync Bot [:wpt-sync] (Matrix: #interop:mozilla.org)]

Upstream PR merged by moz-wptsync-bot

Comment 25

[Ardelean Oana, Desktop Test Engineering [:oardelean]]

Reproducible on a 2024-02-14 Nightly build on Windows 10.
Verified as fixed on Firefox Nightly 127.0a1 and Firefox 126.0b6 on Windows 10, macOS 12, Ubuntu 22.