1848491 - CSS :lang selectors never match against languages with private use tags

Status: RESOLVED FIXED

(Core :: CSS Parsing and Computation, defect)

Description

[uruwi]

Attachment: langtest.html

User Agent: Mozilla/5.0 (X11; Linux x86_64; rv:109.0) Gecko/20100101 Firefox/116.0

Steps to reproduce:

I opened the attached langtest.html file in Firefox.

Actual results:

The word ‘har’ is not in red.

Expected results:

The word ‘har’ should be shown in red, because the only CSS rule in this document has a selector of ‘:lang(art-x-v3-arx0as)’. This should match any element with a language of ‘art-x-v3-arx0as’ because ‘art-x-v3-arx0as’ is a valid BCP-47 language tag (albeit a private use one).

Replacing ‘art-x-v3-arx0as’ with a language code without a private use tag such as ‘art’ produces the expected behavior, although ‘:lang(art)’ does not match against elements with a language of ‘art-x-v3-arx0as’.

Comment 1

[BugBot [:suhaib / :marco/ :calixte]]

The Bugbug bot thinks this bug should belong to the 'Core::CSS Parsing and Computation' component, and is moving the bug to that component. Please correct in case you think the bot is wrong.

Comment 2

[Mayank Bansal]

Can repro this behaviour in Nightly. Chrome shows "har" in red.

Comment 3

[Alice0775 White]

Regression window:
https://hg.mozilla.org/integration/autoland/pushloghtml?fromchange=23fa52107ed38f5c5f06ebed5dce4bc4287143b5&tochange=f1dcf41730702a8752420abf555b734464fdf1b8

Comment 4

[BugBot [:suhaib / :marco/ :calixte]]

:jfkthame, since you are the author of the regressor, bug 1121792, could you take a look? Also, could you set the severity field?

For more information, please visit BugBot documentation.

Comment 5

[Jonathan Kew [:jfkthame]]

Interestingly, Safari likewise doesn't match the :lang() selector in the testcase from comment 0, although Chrome does.

In Gecko, this fails because the lang attribute and the :lang() selector's argument are passed to intl::ffi::unic_langid_new() here, in order to parse them and then perform matching; but unic_langid_new() doesn't handle private subtags, and so rejects them as "invalid".

For the time being I think we can consider dealing with this in nsStyleUtil for the specific case of :lang()-selector matching, by checking for the presence of private subtags (delimited by -x-) and adding custom handling for them. In the longer term, we may want to extend unic_langid_new and the underlying LanguageIdentifier (see https://searchfox.org/mozilla-central/source/third_party/rust/icu_locid/src/langid.rs) to handle private subtags in some way.

The behavior I propose to try here is that if the :lang() selector includes private subtags, then it will only match elements where the lang attribute has the same subtags; but the presence of private subtags in an element's lang attribute will not prevent it matching a selector that has the same (standard) language subtags but doesn't include private subtags. So :lang(en-x-private) would NOT match elements with lang="en" or lang="en-x-other", but a simple :lang(en) WOULD still match elements with lang="en-x-foobar".

Comment 6

[Jonathan Kew [:jfkthame]]

Attachment: Bug 1848491 - Handle private subtags in the lang attribute and :lang() selector. r=#style

Comment 7

[Jonathan Kew [:jfkthame]]

Attachment: Bug 1848491 - Add some WPT reftest for :lang() matching with private subtags. r=#style

Comment 8

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/c2e6e49bdeef
Handle private subtags in the lang attribute and :lang() selector. r=emilio
https://hg.mozilla.org/integration/autoland/rev/cc1e56d6f6ab
Add some WPT reftest for :lang() matching with private subtags. r=emilio

Comment 9

[Web Platform Test Sync Bot [:wpt-sync] (Matrix: #interop:mozilla.org)]

Created web-platform-tests PR https://github.com/web-platform-tests/wpt/pull/41493 for changes under testing/web-platform/tests

Comment 10

[Cristian Tuns]

Backed out for causing reftest failures in unicode-lang.html

• Backout link
• Push with failures
• Failure Log
• Failure line: REFTEST TEST-UNEXPECTED-FAIL | layout/reftests/unicode/unicode-lang.html == layout/reftests/unicode/unicode-ref.html | image comparison, max difference: 255, number of differing pixels: 167

Comment 11

[Jonathan Kew [:jfkthame]]

Ah, right - we're missing a return false in the case where the tag fails to parse, but there isn't a private subtag to be stripped for a re-try.

Comment 12

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/fdca3f17a750
Handle private subtags in the lang attribute and :lang() selector. r=emilio
https://hg.mozilla.org/integration/autoland/rev/1cafd355bbcf
Add some WPT reftest for :lang() matching with private subtags. r=emilio

Comment 13

[Web Platform Test Sync Bot [:wpt-sync] (Matrix: #interop:mozilla.org)]

Upstream PR was closed without merging

Comment 14

[Iulian Moraru]

https://hg.mozilla.org/mozilla-central/rev/fdca3f17a750
https://hg.mozilla.org/mozilla-central/rev/1cafd355bbcf

Comment 15

[Web Platform Test Sync Bot [:wpt-sync] (Matrix: #interop:mozilla.org)]

Upstream PR merged by moz-wptsync-bot

Comment 16

[BugBot [:suhaib / :marco/ :calixte]]

The patch landed in nightly and beta is affected.
:jfkthame, is this bug important enough to require an uplift?

• If yes, please nominate the patch for beta approval.
• If no, please set status-firefox117 to wontfix.

For more information, please visit BugBot documentation.

Comment 17

[Jonathan Kew [:jfkthame]]

I don't think this warrants uplift; it can just ride the 118 train.