1919288 - Firefox decodes URIs in the omnibox upon backward or forward navigation, rendering pasted URIs innavigable-to.

Status: REOPENED

(Firefox :: Address Bar, defect, P3)

Description

[Mr. Beedell, Roke Julian Lockhart (RJLB) [:RokeJulianLockhart]]

Environment

User Agent: Mozilla/5.0 (X11; Linux x86_64; rv:130.0) Gecko/20100101 Firefox/130.0

Steps to reproduce

Like this old Reddit post explains, I attempted to duplicate https://forum.fairphone.com/search?q=lockscreen%20wallpaper%20unset from within the omnibox to my clipboard.

Actual results

https://forum.fairphone.com/search?q=lockscreen wallpaper unset was duplicated instead.

Expected results

1. Such behaviour is not compliant with the relevant RFCs, as is evident.

2. However, more importantly, it prevents the URI from being parsed by the consuming software, should it be pasted into freeform text where heuristics are necessary to isolate URIs (in my case, Discourse).

   This means that a user receiving the URI is liable to not access the entire URI, should they attempt to.

Comment 1

[Mr. Beedell, Roke Julian Lockhart (RJLB) [:RokeJulianLockhart]]

Attachment: Demonstration webpage (HTML).

Comment 2

[Mr. Beedell, Roke Julian Lockhart (RJLB) [:RokeJulianLockhart]]

Attachment: Demonstration screenshot (PNG).

Comment 3

[Marco Bonardo [:mak]]

How is browser.urlbar.decodeURLsOnCopy pref set in about:config?

Comment 4

[Mr. Beedell, Roke Julian Lockhart (RJLB) [:RokeJulianLockhart]]

Attachment: Query for `browser.urlbar.decodeURLsOnCopy` in `about:config`.

#c3

It's false in about:config, and isn't present in /home/RokeJulianLockhart/.mozilla/firefox/fimaebr1.default-release-1720105233406/prefs.js.

Comment 5

[Marco Bonardo [:mak]]

This is what I did:

1. visited https://forum.fairphone.com/search?q=lockscreen%20wallpaper%20unset
2. verified that the url is shown as decoded in the address bar
3. clicked on the address bar and pressed CTRL+C
4. opened a text editor and CTRL+V

The url was encoded (with %20) for me.

Though, if instead of relying of Select All I manually select a part of the url, then what I selected is copied as-is.
I suspect this is what is happening here, even if inadvertently (copying part of the url is not what you want clearly). I could indeed reproduce the bug intermittently after manipulating the URL a bit.
Let's make an experiment, try temporarily disabling browser.urlbar.trimURLs pref in about:config, is the bug reproducible then?
If so, could you please attache a screencast capture, so I can see the exact steps involved?

Comment 6

[Mr. Beedell, Roke Julian Lockhart (RJLB) [:RokeJulianLockhart]]

Attachment: Screencast of reproduction failure by OP.

This is what I did:

1. Visited https://forum.fairphone.com/search?q=lockscreen%20wallpaper%20unset
2. Verified that the url is shown as decoded in the address bar
3. Clicked on the address bar and pressed CTRL+C
4. Opened a text editor and CTRL+V

The url was encoded (with %20) for me.

[e-mail], likewise, now.

I could indeed reproduce the bug intermittently after manipulating the URL a bit.

That must be so, for I exactly reproduced what I did last time, yet it's now encoded.

Let's make an experiment, try temporarily disabling browser.urlbar.trimURLs pref in about:config, is the bug reproducible then?

Actually, I already have browser.urlbar.trimURLs set to false - I changed it a few days ago.

If so, could you please attache a screencast capture, so I can see the exact steps involved?

Certainly. I have a video attached. (Excuse the PIP - I can't be bothered to create a new OBS preset.)

Comment 7

[Chris Bellini (:cbellini)]

We'll mark this bug as resolved for now. But if in the future you encounter a case where this can reproduced, and the steps to do so, we'll investigate it.

Comment 8

[Mr. Beedell, Roke Julian Lockhart (RJLB) [:RokeJulianLockhart]]

#c7

This hasn't noticeably reproduced since.

Comment 9

[Mr. Beedell, Roke Julian Lockhart (RJLB) [:RokeJulianLockhart]]

Attachment: teams.live.com_config_v1_Skype_1415_1.0.0.0_Archive [25-05-01 18-56-30].har

But if in the future you encounter a case where this can reproduced, and the steps to do so, we'll investigate it.

I can consistently reproduce this when performing what github.com/orgs/community/discussions/158138 describes. Specifically, I presume that openId profile openid in the undermentioned should be percent-encoded:

https://login.microsoftonline.com/common/oauth2/v2.0/authorize?client_id=5e3ce6c0-2b1f-4285-8d4b-75ee78787346&scope=openId profile openid offline_access&redirect_uri=https%3A%2F%2Fteams.microsoft.com%2Fv2&client-request-id=01968cf3-d881-7bc3-9aa9-89157c32cbf3&response_mode=fragment&response_type=code&x-client-SKU=msal.js.browser&x-client-VER=3.28.1&client_info=1&code_challenge=4y82Lt11-hIZSiA1FLK1sI7mD0a6z-UZG-isZNCfSVk&code_challenge_method=S256&nonce=01968cf3-d881-7523-a1e3-739ef2741216&state=eyJpZCI6IjAxOTY4Y2YzLWQ4ODEtNzU1My1hNmRlLTM0OTE1YzNlYWM2ZSIsIm1ldGEiOnsiaW50ZXJhY3Rpb25UeXBlIjoicmVkaXJlY3QifX0%3D|https%3A%2F%2Fteams.microsoft.com%2Fv2%2F%23%2Fl%2Fapp%2Fca9e26b7-dce5-44a0-b2b7-a70a3d65ce25%3FdeeplinkId%3D3fceb267-cf07-4c83-819c-21d807c8ded6%26enablemcasfort21%3Dtrue

If that URI contains an authentication token, please mark as confidential.

Environment

firefox-137.0-2.fc42.x86_64

Comment 10

[Mr. Beedell, Roke Julian Lockhart (RJLB) [:RokeJulianLockhart]]

Attachment: A Screencast Demonstrating How To Consistently Reproduce This

Comment 11

[Marco Bonardo [:mak]]

I'm still confused, the string present in the address bar is "beautified" to make it more user readable. It's still a valid URI, but some characters may be decoded.
Though when you copy the whole url, we'll use what's loaded in the tab. Only a partial copy will retain what's visible in the urlbar.

In the screencast I don't see a copy paste operation of the whole url into a separate app.
If I select all the URL, copy and paste into vs code (for example) the URL is fully encoded as it should be.

Comment 12

[Mr. Beedell, Roke Julian Lockhart (RJLB) [:RokeJulianLockhart]]

Attachment: A Screencast Demonstrating That The URI Remains Unencoded Upon Paste

In the screencast I don't see a copy paste operation of the whole url into a separate app. If I select all the URL, copy and paste into vs code (for example) the URL is fully encoded as it should be.

It remains unencoded upon paste into code-1.99.3-1744761644.el8.x86_64:

Name            : code
Epoch           : 0
Version         : 1.99.3
Release         : 1744761644.el8
Architecture    : x86_64
Installed size  : 400.4 MiB
Source          : code-1.99.3-1744761644.el8.src.rpm
From repository : <unknown>
URL             : https://code.visualstudio.com/
Vendor          : Microsoft Corporation

Comment 13

[Mr. Beedell, Roke Julian Lockhart (RJLB) [:RokeJulianLockhart]]

Although I can now consistently reproduce this at the URI immediately aforementioned, I also just experienced this one-time (like I first did with the Fairphone URI) when creating webapps.stackexchange.com/revisions/181205/2: https://web.skype.com/?openPstnPage=true#rx-vlv-1:%7E:text=My%20Microsoft%20account became https://web.skype.com/?openPstnPage=true#rx-vlv-1:%7E:text=My Microsoft account upon initial paste into webapps.stackexchange.com/questions/61143/any-way-to-change-skype-profile-picture-on-line/181205#wmd-input.

Comment 14

[Marco Bonardo [:mak]]

(In reply to Mr. Beedell, Roke Julian Lockhart from comment #12)

Created attachment 9484983 [details]
A Screencast Demonstrating That The URI Remains Unencoded Upon Paste

At second 6 it's absolutely normal that the url is decoded, the address bar will try to make urls more readable.
Even later when the url is retrieved from history, it is normal that it's decoded today (more about this later...).
I can't tell from the video when copy and paste are happening (or maybe I missed a way to tell?).

If you copy a not loaded URL then it's possible it will copy the decoded version, since there's no loaded url.
For example you type part of a URL, select a result with UP/DOWN but don't load it, then copy the URL from the address bar.
Is this what is happening here?

We could evaluate decoding only when URL is actually loaded and not when it's put in by a result selection. That would make editing retrieved URLs a bit worse, but maybe it's not a big deal if this kind of copy operation is common.

Comment 15

[Mr. Beedell, Roke Julian Lockhart (RJLB) [:RokeJulianLockhart]]

Attachment: A Screencast That Demonstrates That The Decoded URI Comes Directly From The Omnibox

I can't tell from the video when copy and paste are happening (or maybe I missed a way to tell?).

I've uploaded a more explicit reproduction that should remediate this.

If you copy a not loaded URL then it's possible it will copy the decoded version, since there's no loaded url. For example you type part of a URL, select a result with UP/DOWN but don't load it, then copy the URL from the address bar. Is this what is happening here?

I believe so, now that you've mentioned it. It would explain why the behaviour appears so inconsistent – because it's decoded when navigated backward or forward to, but not decoded when it's the current/loaded URI.

Comment 16

[Mr. Beedell, Roke Julian Lockhart (RJLB) [:RokeJulianLockhart]]

Comment on attachment 9484840 [details]
teams.live.com_config_v1_Skype_1415_1.0.0.0_Archive [25-05-01 18-56-30].har

We could evaluate decoding only when URL is actually loaded and not when it's put in by a result selection. That would make editing retrieved URLs a bit worse, but maybe it's not a big deal if this kind of copy operation is common.

That's a tradeoff which I hadn't considered. However, I believe that always providing the encoded URI is better, because:

1. The behaviour is more consistent.

2. Decoding an already-encoded URI is significantly easier than encoding a decoded one.

3. Most URIs are pasted to be navigated to. They're less frequently manually modified.

Comment 17

[Moritz Beier [:mbeier]]

We could always encode URLs when copying everything from the urlbar (unless browser.urlbar.decodeURLsOnCopy is true). This would keep everything readable and makes sure the copied URLs always work.

Comment 18

[Moritz Beier [:mbeier]]

I think I misunderstood what you were doing. You were using CTRL+Z to restore the previously loaded URL, right? Like this:

STR:

1. Navigate to https://forum.fairphone.com/search?q=lockscreen%20wallpaper
2. Double-click the urlbar to untrim https://
3. Navigate to any other page
4. Focus the urlbar and press CTRL+Z twice to restore the URL from step 1.
5. Copy the whole URL

Expected: The percent-encoded version of the URL was copied.
Actual: The decoded version of the URL was copied.

I think this happens because URLs restored using CTRL+Z are treated as user typed values. I think it might be kind of complicated to treat previously loaded URLs restored by CTRL+Z differently and this is probably a very rare use-case. Marco, do you think this is worth fixing? And do you think the fact that step 2 is needed is a bug? (That the trimmed instead of the full URL is saved in the undo stack.)

Comment 19

[Mr. Beedell, Roke Julian Lockhart (RJLB) [:RokeJulianLockhart]]

Like this:

That's mostly accurate, except that I needn't modify the URI to have this recur – in the last screencast, I did not remove the schema as you state. I dont see where you would have explicitly seen that.

(Additionally, even before I utilised Control + Z to navigate backward, the omnibox renders the URI as having spaces, rather than percent-encoded values. Is that expected? I don't recall whether it renders percent-encoded values decoded until the omnibox is focussed, or not.)

I think this happens because URLs restored using CTRL+Z are treated as user typed values.

In that case, why is it decoding content entered into the omnibox? I did not do so, so it definitely shouldn't. I wouldn't expect it to modify what I've entered, except to encode it.

Comment 20

[Moritz Beier [:mbeier]]

STR are how to reproduce the bug, not how to reproduce your video.

In that case, why is it decoding content entered into the omnibox?

As Marco said, the string present in the address bar is "beautified" to make it more user readable. But the URL should still be encoded when copying. This works, but only for the currently loaded URL, not for ones restored using CTRL+Z.

Comment 21

[Mr. Beedell, Roke Julian Lockhart (RJLB) [:RokeJulianLockhart]]

STR are how to reproduce the bug, not how to reproduce your video.

Regardless, all that need be done is navigate backward to a URI containing beautified symbols. No modification to the content is necessary, much less specifically the schema.

As Marco said, the string present in the address bar is "beautified" to make it more user readable.

I wasn't aware that they were beautified until copy. Indeed, they appear to be, in retrospect. To bypass this problem, can beautification be disabled in about:config? (I find it makes them a lot more confusing.)

Comment 22

[Moritz Beier [:mbeier]]

No modification to the content is necessary, much less specifically the schema.

You don't need it because you disabled browser.urlbar.trimURLs. Otherwise, the URL without the schema will be in the undo stack.

Comment 23

[Marco Bonardo [:mak]]

(In reply to Moritz Beier [:mbeier] from comment #18)

I think this happens because URLs restored using CTRL+Z are treated as user typed values. I think it might be kind of complicated to treat previously loaded URLs restored by CTRL+Z differently and this is probably a very rare use-case. Marco, do you think this is worth fixing? And do you think the fact that step 2 is needed is a bug? (That the trimmed instead of the full URL is saved in the undo stack.)

I think it's still a bug worth fixing, though a low priority one.

In general if we detect the intent to copy a whole URL we should always copy encoded, unless the decodeURLsOnCopy pref is set.
We should probably not just rely on user typed value... IIRC there was an old bug where I suggested to make that tracking more precise, as Firefox setting a value is not the same as the user typing a value.

Comment 24

[Mr. Beedell, Roke Julian Lockhart (RJLB) [:RokeJulianLockhart]]

To me, there appears an inherent redundancy: we've already a separate search and address bar, yet there is special handling in the address bar to ensure that text duplicated into the address bar is not treated as a URI, despite that being its primary purpose. I am of the opinion that if anyone complains about their text that they've entered into the address bar being encoded, they can take the 5s necessary to put it through a decoder, then utilise the search bar for their non-URI-related purposes thereafter.