1925496 - (CVE-2024-11695) Firefox Desktop Address Bar Spoofing Using Arabic Punycode and a long URL

Status: VERIFIED FIXED

(Firefox :: Address Bar, defect, P1)

Description

[Renwa]

Attachment: Screenshot 2024-10-18 at 2.38.56 PM.png

Hey FireFox Team

If a domain name contains Arabic language and punycode then with a long URL the address bar is showing the long URL and hides the origin, also found a white space \u200c we can fully spoof the URL bar and trick a user thinking they are on a legitimate site.

POC:

https://ا.pwr.wtf/?spoof=%20%E2%80%8C%20%E2%80%8C%20%E2%80%8Cwww.google.com%20%E2%80%8C%20%E2%80%8C%20%E2%80%8C%20%E2%80%8C%20%E2%80%8C%20%E2%80%8C%20%E2%80%8C%20%E2%80%8C%20%E2%80%8C%20%E2%80%8C%20%E2%80%8C%20%E2%80%8C%20%E2%80%8C%20%E2%80%8C%20%E2%80%8C%20%E2%80%8C%20%E2%80%8C%20%E2%80%8C%20%E2%80%8C%20%E2%80%8C%20%E2%80%8C%20%E2%80%8C%20%E2%80%8C%20%E2%80%8C%20%E2%80%8C%20%E2%80%8C%20%E2%80%8C%20%E2%80%8C%20%E2%80%8C%20%E2%80%8C%20%E2%80%8C%20%E2%80%8C%20%E2%80%8C%20%E2%80%8C%20%E2%80%8C%20%E2%80%8C%20%E2%80%8C%20%E2%80%8C%20%E2%80%8C%20%E2%80%8C%20%E2%80%8C%20%E2%80%8C%20%E2%80%8C%20%E2%80%8C%20%E2%80%8C%20%E2%80%8C%20%E2%80%8C%20%E2%80%8C%20%E2%80%8C%20%E2%80%8C%20%E2%80%8C%20%E2%80%8C%20%E2%80%8C%20%E2%80%8C%20%E2%80%8C%20%E2%80%8C%20%E2%80%8C%20%E2%80%8C%20%E2%80%8C%20%E2%80%8C%20%E2%80%8C%20%E2%80%8C%20%E2%80%8C%20%E2%80%8C%20%E2%80%8C%20%E2%80%8C%20%E2%80%8C%20%E2%80%8C%20%E2%80%8C%20%E2%80%8C%20%E2%80%8C%20%E2%80%8C%20%E2%80%8C%20%E2%80%8C%20%E2%80%8C%20%E2%80%8C%20%E2%80%8C%20%E2%80%8C%20%E2%80%8C%20%E2%80%8C%20%E2%80%8C%20%E2%80%8C%20%E2%80%8C%20%E2%80%8C%20%E2%80%8C%20%E2%80%8C%20%E2%80%8C%20%E2%80%8C%20%E2%80%8C%20%E2%80%8C%20%E2%80%8C%20%E2%80%8C%20%E2%80%8C%20%E2%80%8C%20%E2%80%8C%20%E2%80%8C%20%E2%80%8C%20%E2%80%8C%20%E2%80%8C%20%E2%80%8C%20%E2%80%8C%20%E2%80%8C%20%E2%80%8C%20%E2%80%8C%20%E2%80%8C%20%E2%80%8C%20%E2%80%8C%20%E2%80%8C%20%E2%80%8C%20%E2%80%8C%20%E2%80%8C%20%E2%80%8C%20%E2%80%8C%20%E2%80%8C%20%E2%80%8C%20%E2%80%8C%20%E2%80%8C%20%E2%80%8C%20%E2%80%8C%20%E2%80%8C%20%E2%80%8C%20%E2%80%8C%20%E2%80%8C%20%E2%80%8C%20%E2%80%8C%20%E2%80%8C%20%E2%80%8C%20%E2%80%8C%20%E2%80%8C%20%E2%80%8C%20%E2%80%8C%20%E2%80%8C%20%E2%80%8C%20%E2%80%8C%20%E2%80%8C%20%E2%80%8C%20%E2%80%8C%20%E2%80%8C%20%E2%80%8C%20%E2%80%8C%20%E2%80%8C%20%E2%80%8C%20%E2%80%8C%20%E2%80%8C%20%E2%80%8C%20%E2%80%8C%20%E2%80%8C%20%E2%80%8C%20%E2%80%8C%20%E2%80%8C%20%E2%80%8C%20%E2%80%8C%20%E2%80%8C%20%E2%80%8C%20%E2%80%8C%20%E2%80%8C%20%E2%80%8C%20%E2%80%8C%20%E2%80%8C%20%E2%80%8C%20%E2%80%8C%20%E2%80%8C%20%E2%80%8C%20%E2%80%8C%20%E2%80%8C%20%E2%80%8C%20%E2%80%8C%20%E2%80%8C%20%E2%80%8C%20%E2%80%8C%20%E2%80%8C%20%E2%80%8C%20%E2%80%8C%20%E2%80%8C%20%E2%80%8C%20%E2%80%8C%20%E2%80%8C%20%E2%80%8C%20%E2%80%8C%20%E2%80%8C%20%E2%80%8C%20%E2%80%8C%20%E2%80%8C%20%E2%80%8C%20%E2%80%8C%20%E2%80%8C%20%E2%80%8C%20%E2%80%8C.

Online POC: https://pwr.wtf/pwn.php

Image POC attached too

Number of whitespaces can be adjusted to the user screen size.

Thanks
Renwa

Comment 1

[Andrew McCreight [:mccr8]]

Attachment: Address bar screenshot

The address bar looks less spoofy for me. Maybe because my browser is localized for English?

Comment 2

[Andrew McCreight [:mccr8]]

Ah, I see. The way the address bar looks depends on how wide the window is. If my browser window is full width, then you can see the whole URL. If it is about 60% of the width of the screen, then the address bar is completely blank. When it is somewhere in between, I see only "www.google.com". It does look funny because it is in grey instead of white, but otherwise it looks pretty convincing to me.

So, it depends on the specific width of the window, but presumably an attacker could measure the width and put in an appropriate number of characters. I see now the reporter did allude to this at the end of their report.

Comment 3

[Andrew McCreight [:mccr8]]

I guess I wasn't able to confirm the precise behavior of the screen shot where it shows https:// and the URL, which is important, but maybe with some more fiddling that could be reproced.

Comment 4

[Renwa]

Hey Andrew, My browser is also localized for English. I also mentioned in the description that the payload is constructed for my screen size (776x1536) https://www.webfx.com/tools/whats-my-browser-size/

For the bug to work the attacker needs to adjust number of whitespaces according to width of the victim, increase it yourself and you will able to see the full spoofed URL.

Comment 5

[Andrew McCreight [:mccr8]]

Attachment: screen shot at 776 by 1536

Thank you for the additional detail. I set my window to 776x1536 and it still does not reproduce exactly like the screen shot. Maybe the presence of the saved password icon in your address bar also is affecting things or something else.

Comment 6

[Renwa]

It might be the saved password icon or an extension and other stuff that I have differs from your setup, I can try to provide a full working POC for all screen sizes if needed? but I think the bug is obvious and reproducible.

Comment 7

[Marco Bonardo [:mak]]

Thank you for the report.

From Bug 1395508 we were supposed to decode consecutive whitespaces, it looks like that's not working here.
\u200c is not part of the list of encoded characters (We encode \u2000-\u200a), I suspect because it's supposed to be zero-width? But apparently it's not Ah I see this is wisely fragmented with %20 exactly to bypass the encoding. Any zero-width would allow to bypass the encode-consecutive-spaces regex with no visible signs.

We are also supposed to ensure the origin is visible, so we must check why in this case that is failing.

Comment 8

[Marco Bonardo [:mak]]

I'll start investigating this.

Comment 9

[Marco Bonardo [:mak]]

I think we should just encode any %20 that is not followed by a printable ASCII. That is more aggressive than the current method, whose primary scope was to try to preserve some URL readability. But it's effectively easy to find chars that are either zero-width, or barely visible, and interleave %20 with them, breaking any kind of consecutive-whitespace encoding rule we may write.
Using printable ASCII the worst cases are ` ` ` ` ` ` ` or . . . . . . . . . that are visible anyway.

Comment 10

[Marco Bonardo [:mak]]

We currently rely on directionality of the domain, in this case the domain is moved to the right:
https://اسماء.شبكة/#%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%D7%A1%D7%95%D6%B9.%D7%A1%D7%97
But in this case it's not:
https://ا.pwr.wtf/?spoof=%20%E2%80%8C%20%E2%80%8C%20%E2%80%8Cwww.google.com%20%E2%80%8C%20%E2%80%8C%20%E2%80%8C%20%E2%80%8C%20%E2%80%8C20%E2%80%8C%20%E2%80%8C%20%E2%80%8C%20%E2%80%8C%20%E2%80%8C%20%E2%80%8C.

Jonathan, is there a way to tell from javascript if the domain will remain on the left or will be moved to the right? Doing further approximation here doesn't seem great. To ensure the domain is always visible we either need to know where it is, or the textbox itself should ensure a selection we set is scrolled into view (but scrollSelectionIntoView seems to only apply vertically).
I can't find a way to get the visual string, or to convince the editor to horizontally scroll to a selection.

edit: in reality, setting SELECTION_NORMAL scrolls it into view, though I think we'd prefer to not override the selection. I must double check if we ccan overwrite it.

Comment 11

[Jonathan Kew [:jfkthame]]

(In reply to Marco Bonardo [:mak] from comment #10)

We currently rely on directionality of the domain, in this case the domain is moved to the right:
https://اسماء.شبكة/#%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%D7%A1%D7%95%D6%B9.%D7%A1%D7%97

In this case, the domain is RTL and everything following it is also RTL or neutral-direction (the punctuation chars) or becomes an embedded LTR fragment (the digits "20" in each encoded space). So overall, in the rendered URL bar we have "https://" (LTR) at the left-hand end, and then everything else beginning with the Arabic-script domain اسماء.شبكة and ending with the Hebrew-script "spoofed domain" סוֹ.סח as an RTL run. The fact that this RTL run has a lot of embedded (higher-bidi-level) LTR numbers in it is unimportant; it still ends up starting (at the far right) with اسماء.شبكة and ending (at the far left, adjacent to "https://") with סוֹ.סח.

If we un-escape the Hebrew letters, and shorten the run of encoded spaces to avoid wrapping, I expect we'll see the same ordering happen here:

https://اسماء.شبكة/#%20%20%20סוֹ.סח

But in this case it's not:
https://ا.pwr.wtf/?spoof=%20%E2%80%8C%20%E2%80%8C%20%E2%80%8Cwww.google.com%20%E2%80%8C%20%E2%80%8C%20%E2%80%8C%20%E2%80%8C%20%E2%80%8C20%E2%80%8C%20%E2%80%8C%20%E2%80%8C%20%E2%80%8C%20%E2%80%8C%20%E2%80%8C.

In this case, although the domain starts with an Arabic letter, it's then followed by strongly-LTR characters. So the Arabic letter is a little RTL run all on its own, which appears immediately after the initial "https://" and is then followed by more LTR text at the base paragraph bidi level.

Jonathan, is there a way to tell from javascript if the domain will remain on the left or will be moved to the right?

If I'm thinking about this right, the domain will only end up at the right if it is itself RTL and it is not followed by any strongly-LTR text (which would end the RTL run and appear "after" it in LTR order).

Doing further approximation here doesn't seem great. To ensure the domain is always visible we either need to know where it is, or the textbox itself should ensure a selection we set is scrolled into view (but scrollSelectionIntoView seems to only apply vertically).

The domain could appear almost anywhere, I think. Most commonly it's at the left-hand edge of the text (after the initial "https://"), or in the case of a purely-RTL URL, at the right-hand edge; but if you have an RTL domain, and follow it with some more RTL text (as /path/, ?query, or #fragment) and then some strongly-LTR text, the actual domain could end up in the middle, with potentially-misleading stuff on either side of it:

https://اسماء.شبكة/%20%E2%80%8C%20%E2%80%8C%20%E2%80%8C%D7%A1%D7%95%D6%B9.%D7%A1%D7%97/%20%E2%80%8C%20%E2%80%8C%20%E2%80%8Cno.io

The actual domain here is اسماء.شبكة, but in the rendered URL bar that doesn't appear at either the left- or right-hand edge.

I can't find a way to get the visual string, or to convince the editor to horizontally scroll to a selection.

Yeah, tricky. I wonder, does Find-in-Page manage to deal with this if you have bidi content and horizontal scrolling, and the fragment it needs to highlight is scrolled out of view? If it does -- I haven't tested this yet -- we might be able to convince the editor to do something similar.

The other way forward would be to impose more structure on the URL when we display it, so that we're not just relying on the (rather hairy) Unicode bidi algorithm, especially for edge cases involving neutral characters, etc. This could be done by wrapping parts of the URL in inline elements with appropriate bidi isolation properties, or by inserting bidi control codes for display purposes. Either one is also potentially tricky, though, if we want to maintain the illusion that the URL as a whole is a simple user-editable string.

Comment 12

[Jonathan Kew [:jfkthame]]

(In reply to Marco Bonardo [:mak] from comment #10)

edit: in reality, setting SELECTION_NORMAL scrolls it into view, though I think we'd prefer to not override the selection. I must double check if we ccan overwrite it.

Ah, interesting -- maybe we could provide an extended version of scrollSelectionIntoView that can be told which type of selection we want it to consider?

[edit: looks like it already takes a type parameter; can we use that to make this work?]

Comment 13

[Marco Bonardo [:mak]]

(In reply to Jonathan Kew [:jfkthame] from comment #11)

Yeah, tricky. I wonder, does Find-in-Page manage to deal with this if you have bidi content and horizontal scrolling, and the fragment it needs to highlight is scrolled out of view? If it does -- I haven't tested this yet -- we might be able to convince the editor to do something similar.

I'm not 100% sure how to test this, but I can try.

The other way forward would be to impose more structure on the URL when we display it

Yeah, that's tricky, we tried a few things in the past, but it was breaking too many assumptions. Even if we go the Safari way of just showing the origin, I suspect we'll have to provide an option to disable it for advanced users, and we'd still not want to leave them without a visible origin.

Chrome is handling these cases better, I guess they have a way to tell the editor to scroll enough to make a part fully visible, or don't scroll at all if it's already fully visible.
That's what ideally I'd like, a new invisible SELECTION_URLORIGIN around the domain, and just call scrollSelectionIntoView for it, with a similar behavior.

Comment 14

[Jonathan Kew [:jfkthame]]

We currently have SELECTION_URLSECONDARY, which is presumably set to everything except the origin (or is that not exactly how it's used?), so this would basically be the inverse of that. Maybe if we introduce SELECTION_URLORIGIN we wouldn't need SECONDARY any more, as that would just be the default?

Comment 15

[Marco Bonardo [:mak]]

URLSECONDARY is used to de-emphasize the text that is not the origin, and it does that using opacity (alpha channel is set to 127 here).
To invert it, we should set the base color to the de-emphasized one and then set full opacity on URLORIGIN. It would probably just add some complexity to css, but it's feasible.

Though, URLSECONDARY is not working with scrollSelectionIntoView, and similarly URLORIGIN wouldn't work.
NORMAL works, any other type doesn't (I also tried FIND). I wonder if it's because it's not actually moving the caret.
If it can't be made working, we may have to use NORMAL.

Actually, we should not care about the caret when the url is formatted, but we use it because of autofill:

1. type moz
2. get moz[illa] filled
3. focus content
4. focus again the urlbar
5. at this point we check if the caret is at the end, and if it is we autofill

We could maybe store the caret position on blur and restore it on focus, that may free up the possibility to modify the selection while the field is blurred. There's also a few tests that for some reason check that we preserve the selection on blur, maybe related to the window losing/gaining focus.

Comment 16

[Marco Bonardo [:mak]]

I'm thinking to split out this into 2 bugs, in a separate one handle the whitespace encoding, for which we can easily get an immediate fix, while leave this bug to actually keep the domain visible that sounds like more complex work.
Would that be a problem to handle (re: bounty), and in case do you prefer the separate bug still be a security one, or can we land it just as a public bug fix to url encoding, making no explicit reference to the issue?

Comment 17

[Daniel Veditz [:dveditz]]

Splitting the bugs is fine; sounds like a good idea in this case. Keeping the whitespace bug a security bug will make sure we don't drop the ball on tracking it for a bounty and making the advisory/CVE.

The problem of making sure the "right parts" of the domain are visible is an already-known problem in general (see bug 1598175), or will RTL require additional changes? If you agree it's the same then we could use this bug for the whitespace issue and add some RTL examples to bug 1598175.

Comment 18

[Daniel Veditz [:dveditz]]

(In reply to Jonathan Kew [:jfkthame] from comment #11)

if you have an RTL domain, and follow it with some more RTL text (as /path/, ?query, or #fragment) and then some strongly-LTR text, the actual domain could end up in the middle, with potentially-misleading stuff on either side of it:

https://اسماء.شبكة/%20%E2%80%8C%20%E2%80%8C%20%E2%80%8C%D7%A1%D7%95%D6%B9.%D7%A1%D7%97/%20%E2%80%8C%20%E2%80%8C%20%E2%80%8Cno.io

The actual domain here is اسماء.شبكة, but in the rendered URL bar that doesn't appear at either the left- or right-hand edge.

That's bonkers—not usable by either RTL or LTR readers.

Comment 19

[Daniel Veditz [:dveditz]]

I think we should just encode any %20 that is not followed by a printable ASCII. That is more aggressive than the current method, whose primary scope was to try to preserve some URL readability.

That seems overly aggressive. It would look a bit ugly to have a random %20 here or here in a Latin-based URL:
https://bergamo.corriere.it/notizie/cronaca/24_novembre_05/Città Alta si%20è svuotata di bergamaschi (NB: the site's real URLs use underscores and only ASCII, but it's plausible). For sites using non-Latin scripts ALL the spaces get taken away at once.

It looks like \u200C and \u200D were explicitly excluded from escaping in bug 582186 because it broke (uglified?) sites. But if they're next to a space they (or the space) should be fair game.

Is the escaping you're taking about the stuff that's going on in losslessDecodeURI()? If you use unicode character sets you could simplify https://searchfox.org/mozilla-central/rev/fcf53e1685bfb990b5abc7312ac1daf617f0991f/browser/components/urlbar/UrlbarInput.sys.mjs#4903
from
/[\u0000-\u001f\u007f-\u00a0\u1680\u2000-\u200a\u2028\u2029\u202f\u205f\u2800\u3000\ufffc]|[\r\n\t]|\u0020(?=\u0020)|\s$/g
to
/[[\s--\u0020]\p{Cc}\u2800\uFFFC]|\u0020(?=\u0020)|\s$/vg

Then to fix this bug you could add the zero-width chars to the middle term ...\u0020(?=[\u0020\u200B-\u200D]).... If you wanted to be even more aggressive you could escape space in front of all Format characters: \u0020(?=[\u0020\p{Cf}])

the \s match (or \p{Z}) ends up escaping the \uFEFF zero-width space as a bonus so we don't have to worry about that one.

Comment 20

[Marco Bonardo [:mak]]

(In reply to Daniel Veditz [:dveditz] from comment #19)

I think we should just encode any %20 that is not followed by a printable ASCII. That is more aggressive than the current method, whose primary scope was to try to preserve some URL readability.

That seems overly aggressive. It would look a bit ugly to have a random %20 here or here in a Latin-based URL:

yeah, it is aggressive. My fear is that this becomes a whac-a-mole run after characters that are barely visible, or invisible.
We can try to find a set of chars a bit larger than ASCII but still made up of visible characters, or maybe go for \u0020(?!\p{L})

(one examples of using barely visible char: \u0702)

Comment 21

[Renwa]

Maybe using this list https://invisible-characters.com/ and anything from this with a space change it to %20 the list I think is from Microsoft Code https://github.com/microsoft/vscode/blob/bb5215fff67fd9f40e247a353cc0e5e84a28f49f/src/vs/base/common/strings.ts#L1175

Comment 22

[Renwa]

(In reply to Renwa from comment #21)

Maybe using this list https://invisible-characters.com/ and anything from this with a space change it to %20 the list I think is from Microsoft Code https://github.com/microsoft/vscode/blob/bb5215fff67fd9f40e247a353cc0e5e84a28f49f/src/vs/base/common/strings.ts#L1175

• All zero width characters

Comment 23

[Marco Bonardo [:mak]]

(In reply to Daniel Veditz [:dveditz] from comment #17)

The problem of making sure the "right parts" of the domain are visible is an already-known problem in general (see bug 1598175), or will RTL require additional changes? If you agree it's the same then we could use this bug for the whitespace issue and add some RTL examples to bug 1598175.

They are related, being both ways of pushing the domain out of the visible run. RTL may add some technical complexity, due to the domain position being established only in the visual string. Though, if we handle that using some selection tool (e.g. scrollSelectionIntoView), it should work in both cases.

Comment 24

[Marco Bonardo [:mak]]

(In reply to Renwa from comment #21)

Maybe using this list https://invisible-characters.com/ and anything from this with a space change it to %20 the list I think is from Microsoft Code https://github.com/microsoft/vscode/blob/bb5215fff67fd9f40e247a353cc0e5e84a28f49f/src/vs/base/common/strings.ts#L1175

we're covering all of them but: 7355 (1CBB), 7356 (1CBC), 8204 (200C), 8205 (200D), 78844 (133FC)
200C and 200D were explicitly excluded (see Comment 19).
The others could be added to our lists. The first 2 are apparently undefined/unsupported. The last one is apparently invisible on Windows?

Comment 25

[Jonathan Kew [:jfkthame]]

(In reply to Marco Bonardo [:mak] from comment #24)

The others could be added to our lists. The first 2 are apparently undefined/unsupported. The last one is apparently invisible on Windows?

That seems odd... it's just one character of many in the Egyptian Hieroglyphs block, nothing special about it. I just tried it on my Win10 machine and the glyph appears fine. Maybe it was broken in some older version of Segoe UI Historic?

Comment 26

[Marco Bonardo [:mak]]

It's invisible for me, in the Address Bar, in VS Code and also in a textarea on the web 𓏼 <= also here

Comment 27

[Jonathan Kew [:jfkthame]]

Interesting.... what font does the Inspector's Fonts panel say is being used if you examine that comment?

Comment 28

[Marco Bonardo [:mak]]

(In reply to Jonathan Kew [:jfkthame] from comment #27)

Interesting.... what font does the Inspector's Fonts panel say is being used if you examine that comment?

FiraGO, Segoe UI Historic

Comment 29

[Jonathan Kew [:jfkthame]]

That's exactly what I see. But the hieroglyph character appears fine here. Strange! What version is your Segoe UI Historic, if you check in Windows Settings / Fonts? Mine (on Windows 10) calls itself 1.03 (and is dated © 2014 Microsoft).

Comment 30

[Marco Bonardo [:mak]]

Attachment: Segoe UI Historic Versioning

This is from latest W11 italian. The font is marked as 1.07, the only date I see is 2022 on the copyright.

Comment 31

[Jonathan Kew [:jfkthame]]

OK, so it sounds like they've somehow broken/lost that glyph in the newer font! That's pretty sad.

Comment 32

[Jonathan Kew [:jfkthame]]

I don't have Win11 locally, but to double-check, I confirmed that U+133FC renders as a blank space if I load data:text/html,<h1>test:(&%23x133FC;) in Edge on Win11 using browserstack.com. :-(

Comment 33

[Jonathan Kew [:jfkthame]]

FTR, I've reported the broken font to the mozilla-microsoft-discuss list in the hope of getting some attention to it.

Comment 34

[Jonathan Kew [:jfkthame]]

(In reply to Marco Bonardo [:mak] from comment #30)

This is from latest W11 italian. The font is marked as 1.07, the only date I see is 2022 on the copyright.

Could you confirm the version number of Win11 you're on? Microsoft contact says he "can't repro on 24h2".

Comment 35

[Marco Bonardo [:mak]]

I'm on 23H2 22631.4391, so apparently not on the latest, though winupdate doesn't find any update (I didn't opt in to early updates, so that's likely why).

Comment 36

[Jonathan Kew [:jfkthame]]

According to Gov at Microsoft, the U+133FC issue is "confirmed fixed in Win11 24h2". So assuming most Win11 users will get that update fairly soon, the problem with that particular character should be short-lived and I don't think we need to hack around it in Firefox.

We may still want to deal with other "really-invisible" codepoints, of course. Looking at the list in comment 24:

7355 (1CBB), 7356 (1CBC), 8204 (200C), 8205 (200D), 78844 (133FC)

both U+1CBB and U+1CBC are unassigned Unicode codepoints, so I would expect them to display as missing-glyph hexboxes (and that's what I see on Win10, at least). Unless there's a font that actually supports those codepoints (despite them being unassigned) and then renders them with empty glyphs, they shouldn't be a problem.

U+200C and U+200D are the join controls, which are legitimately needed for writing some languages (e.g. Persian, various Indic langs, etc), but should probably be made visible if they occur in "unexpected" contexts (such as around spaces, or if multiple join controls appear in a run).

Comment 37

[Marco Bonardo [:mak]]

(In reply to Jonathan Kew [:jfkthame] from comment #36)

both U+1CBB and U+1CBC are unassigned Unicode codepoints, so I would expect them to display as missing-glyph hexboxes

Unfortunately, on win11 23h2 at least, these are also shown as spaces, like 133FC
In any case I don't see a use for these, so encoding them shouldn't be a problem.

Comment 38

[Jonathan Kew [:jfkthame]]

(In reply to Marco Bonardo [:mak] from comment #37)

(In reply to Jonathan Kew [:jfkthame] from comment #36)

both U+1CBB and U+1CBC are unassigned Unicode codepoints, so I would expect them to display as missing-glyph hexboxes

Unfortunately, on win11 23h2 at least, these are also shown as spaces, like 133FC

That's interesting. Are they zero-width, or visible spaces? What font(s) does Firefox report that it's using for something like data:text/html,[&%23x1CBB;]?

In any case I don't see a use for these, so encoding them shouldn't be a problem.

Yes, it should be harmless given that they don't have any reasonable use.

Comment 39

[Daniel Veditz [:dveditz]]

One advantage of a negation like your \u0020(?!\p{L}) suggestion in comment 20 is that unassigned character points will automatically be excluded. Might need to add numbers—\p{N}—or other things, too, for readability, but I'm not going to bikeshed on it too much.

• \u0020(?=\u0020) current -- too weak (this bug)
• \u0020(?=[\u0020\p{Cf}]) comment 19 -- on the cautious end, but might be OK
• \u0020(?!\p{L}) comment 20 -- on the aggressive end, but might be OK
• '\u0020(?![0-9A-Za-z])` comment 9-ish -- too aggressive

Comment 40

[Marco Bonardo [:mak]]

(In reply to Jonathan Kew [:jfkthame] from comment #38)

(In reply to Marco Bonardo [:mak] from comment #37)

(In reply to Jonathan Kew [:jfkthame] from comment #36)

both U+1CBB and U+1CBC are unassigned Unicode codepoints, so I would expect them to display as missing-glyph hexboxes

Unfortunately, on win11 23h2 at least, these are also shown as spaces, like 133FC

That's interesting. Are they zero-width, or visible spaces? What font(s) does Firefox report that it's using for something like data:text/html,[&%23x1CBB;]?

They are visible spaces in the address bar and in VS Code.
In content it uses Times New Roman and I see the missing glyph square.

Comment 41

[Marco Bonardo [:mak]]

Attachment: Bug 1925496. r=gijs,jfkthame

Comment 42

[Marco Bonardo [:mak]]

Attachment: Bug 1925496 - Test. r=adw

Comment 43

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/40518e514fc1
r=jfkthame,adw

Comment 44

[Sebastian Hengst [:aryx] (needinfo me if it's about an intermittent or backout)]

Backed out for causing bc failures @ browser_copying.js:
https://hg.mozilla.org/integration/autoland/rev/1b1c1bb19d281987299dd840ae29f0afc0677816

Push with failures
Failure log

[task 2024-11-08T11:38:43.430Z] 11:38:43     INFO - TEST-UNEXPECTED-FAIL | browser/components/urlbar/tests/browser/browser_copying.js | url bar value set to example.com/a%20%C2%A0test - Got "example.com/a%20%C2%A0test", expected "example.com/a %C2%A0test"

Comment 45

[Marco Bonardo [:mak]]

oops, changes to browser_copying were in the test revision, should have been in the fix revision.

Comment 46

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/7c9b373df399
r=jfkthame,adw

Comment 47

[Sebastian Hengst [:aryx] (needinfo me if it's about an intermittent or backout)]

https://hg.mozilla.org/mozilla-central/rev/7c9b373df399

Comment 48

[Marco Bonardo [:mak]]

Attachment: Bug 1925496. r=gijs,jfkthame

Original Revision: https://phabricator.services.mozilla.com/D228315

Comment 49

[Phabricator Automation]

beta Uplift Approval Request

• User impact if declined: security bug with url representation
• Code covered by automated testing: no
• Fix verified in Nightly: no
• Needs manual QE test: yes
• Steps to reproduce for manual QE testing: Check spaces are encoded in URLs as %20 when they are followed by a Separator or Other (according to Unicode standard. This includes Control and Formatting)
• Risk associated with taking this patch: low
• Explanation of risk level: We're modifying a regex to match more characters
• String changes made/needed: none
• Is Android affected?: no

Comment 50

[Marco Bonardo [:mak]]

Attachment: Bug 1925496. r=gijs,jfkthame

Original Revision: https://phabricator.services.mozilla.com/D228315

Comment 51

[Phabricator Automation]

esr128 Uplift Approval Request

• User impact if declined: security bug with url representation
• Code covered by automated testing: no
• Fix verified in Nightly: no
• Needs manual QE test: yes
• Steps to reproduce for manual QE testing: Check spaces are encoded in URLs as %20 when they are followed by a Separator or Other (according to Unicode standard. This includes Control and Formatting)
• Risk associated with taking this patch: low
• Explanation of risk level: We're modifying a regex to match more characters
• String changes made/needed: none
• Is Android affected?: no

Comment 52

[Pulsebot]

https://hg.mozilla.org/releases/mozilla-beta/rev/3c92b790c1df

Comment 53

[Pulsebot]

https://hg.mozilla.org/releases/mozilla-esr128/rev/e6099586845f

Comment 54

[Ardelean Oana, Desktop Test Engineering [:oardelean]]

Reproducible on a 2024-10-18 Nightly build on Windows 10.
Verified as fixed on Firefox Nightly 134.0a1, Firefox 133.0b8 and Firefox ESR 128.5.0( treeherder build), on Windows 10, Ubuntu 22, macOS 14.

Comment 55

[Renwa]

Hey there thanks for the bounty, I'm new to this how can I claim it and what payment methods are available? Thanks

Comment 56

[Tom Ritter [:tjr]]

Attachment: advisory.txt

Comment 57

[BugBot [:suhaib / :marco]]

2 months ago, tjr placed a reminder on the bug using the whiteboard tag [reminder-test 2025-02-04] .

mak, please refer to the original comment to better understand the reason for the reminder.

Comment 58

[Marco Bonardo [:mak]]

triggered landing for the test.

Comment 59

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/9f5291847282
Test. r=adw

Comment 60

[Donal Meehan [:dmeehan]]

https://hg.mozilla.org/mozilla-central/rev/9f5291847282