The default bug view has changed. See this FAQ.

Loading URL by pressing ENTER on already present URL in location bar doesn't maintain URL encoding

RESOLVED FIXED in Firefox 10

Status

()

Firefox
Location Bar
--
major
RESOLVED FIXED
9 years ago
5 years ago

People

(Reporter: Kai de Leeuw, Assigned: emk)

Tracking

Trunk
Firefox 10
x86
Windows XP
Points:
---
Dependency tree / graph
Bug Flags:
blocking-firefox3.5 -
wanted-firefox3.6 +
in-testsuite +

Firefox Tracking Flags

(Not tracked)

Details

(URL)

Attachments

(3 attachments, 5 obsolete attachments)

(Reporter)

Description

9 years ago
User-Agent:       Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.1b2pre) Gecko/20081022 Minefield/3.1b2pre
Build Identifier: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.1b2pre) Gecko/20081022 Minefield/3.1b2pre

When you load a page in Firefox the location bar makes some magic and "decodes" the URL, making %c3%b6 for instance appearing as "ö". All this is fine, but when I just go to the location bar and press ENTER the encoding gets screwed up.

What I think Firefox does wrong: 
It doesn't remember what encoding the page loaded had (UTF-8) and therefore just encodes it as ISO-8859-1 or something like this. It should "remember" the encoding.

Reproducible: Always

Steps to Reproduce:
1.Load the URL in the URL field, see that the textbox to the right contains "höjning"
2.Go to the location bar and press ENTER
3.See how the text box changes to h�jning
Actual Results:  
The URL encoding changed

Expected Results:  
The URL encoding stays the same
(Reporter)

Comment 1

9 years ago
According to my colleague who has a Mac this is a Windows only error.
Version: unspecified → Trunk

Comment 2

9 years ago
I see this too, steps to reproduce
1. load http://www.stockholm.se/-/Nyheter/?q=h%C3%B6jning&inkluderalokala=False
2. Now focus the url field and press ENTER

ACTUAL RESULTS
"höjning" in url changed to "h%F6jning" which led to no search results on the page

EXPECTED RESULTS
"höjning" should not change once you press enter in the url field

Updated

9 years ago
Status: UNCONFIRMED → NEW
Ever confirmed: true
Flags: blocking-firefox3.1?
Gavin, Mardak: any ideas?
Presumably this happens because network.standard-url.encode-query-utf8 is false, which seems to be bug 333859. I guess this is a duplicate of that bug. Some context is in bug 261929.

IE 7 has the same bug, but doesn't automatically decode visited URIs, so it's more difficult to hit (you need to paste in the UTF-8 string). I wonder whether we should avoid decoding the query part...
Status: NEW → RESOLVED
Last Resolved: 9 years ago
Flags: blocking-firefox3.1?
Resolution: --- → DUPLICATE
Duplicate of bug: 333859
Oops, misunderstood Gavin, who's proposing that we mitigate in Firefox.

The whole point of encoding the query strings is so that when someone types something like "höjning" into their location bar, it matches what they expect. If I'm understanding this right, though, when they then find the thing that looks as expected, hitting ENTER won't have predictable results.

To me that implies that not decoding the query string part will just mean that we won't get the right sort of smart location bar matches, which is a different problem.

I think the best solution is to have this fixed by bug 333859 ...
Status: RESOLVED → REOPENED
Flags: blocking-firefox3.1?
Resolution: DUPLICATE → ---
(In reply to comment #6)
> The whole point of encoding the query strings is so that when someone types
> something like "höjning" into their location bar, it matches what they expect.
> If I'm understanding this right, though, when they then find the thing that
> looks as expected, hitting ENTER won't have predictable results.

If the site expects it's parameters to be UTF-8 encoded, and the character is in the query string, yeah.

> To me that implies that not decoding the query string part will just mean that
> we won't get the right sort of smart location bar matches, which is a different
> problem.

I don't think the awesomebar is affected by this either way, since we explicitly decode URIs there.
(In reply to comment #7)
> I don't think the awesomebar is affected by this either way, since we
> explicitly decode URIs there.

I was wrong here... we store the correctly encoded URI, assuming you reached the site by actually submitting the search or clicking a link rather than just entering the URL in the URL bar, but when you recall that entry in the awesomebar we decode it again, triggering this bug.
This isn't a regression, so as much as it pains me, it doesn't block. Mardak, can you think of anything that could be causing this on w32-only?
Flags: wanted-firefox3.1+
Flags: blocking-firefox3.1?
Flags: blocking-firefox3.1-
Looks like it's windows-only because LoadURI() takes a wstring, and somehow before that gets passed to NS_NewURI that is assumed to be in the platform native character set (which is UTF-8 on Linux and Mac)?

Updated

8 years ago
Target Milestone: --- → Firefox 3.1
(Assignee)

Comment 11

8 years ago
Bug 393246 will fix this. It will have fewer side-effects than bug 333859.
Bug 393246 was backouted because of bug 432836. But I don't think bug 432836 is a valid claim because even Firefox 2 does not "work" if the system codepage is not Windows-1252.
Duplicate of this bug: 484244
Target Milestone: Firefox 3.1 → Firefox 3
Target Milestone: Firefox 3 → Firefox 3.5

Updated

8 years ago
Depends on: 393246
Target Milestone: Firefox 3.5 → ---
Flags: wanted-firefox3.5+ → wanted-firefox3.6+
Duplicate of this bug: 551955
Blocks: 673500

Comment 14

6 years ago
Instead of waiting for a fix for bug 393246 or bug 333859, I see the simple solution to just only decode URLs when encoding them again would lead to the same URL.

For example when we encode using the system charset, only decode URLs that are in the system charset, too. If we encode only in UTF-8, only decode when the source URL is in UTF-8, too.
(Assignee)

Comment 15

6 years ago
Created attachment 569979 [details] [diff] [review]
Part 1: Add UTF-8 flag to nsDocShell::LoadURI
Assignee: nobody → VYV03354
Status: REOPENED → ASSIGNED
Attachment #569979 - Flags: review?(Olli.Pettay)
(Assignee)

Comment 16

6 years ago
Created attachment 569980 [details] [diff] [review]
Part 2: Use UTF-8 flag in the browser

We can be sure that the URI is encoded in UTF-8 if the value of location bar is not typed.
Attachment #569980 - Flags: review?(gavin.sharp)
Comment on attachment 569979 [details] [diff] [review]
Part 1: Add UTF-8 flag to nsDocShell::LoadURI

I'd prefer if Boris could review this since he reviewed Bug 691690.
Attachment #569979 - Flags: review?(Olli.Pettay) → review?(bzbarsky)
Comment on attachment 569979 [details] [diff] [review]
Part 1: Add UTF-8 flag to nsDocShell::LoadURI

Should that be an else if, or just an if?
(Assignee)

Comment 19

6 years ago
Created attachment 570453 [details] [diff] [review]
Part 1: Add UTF-8 flag to nsDocShell::LoadURI, v2

Oh, I don't recall why I wrote "else if" here.
Attachment #569979 - Attachment is obsolete: true
Attachment #570453 - Flags: review?(bzbarsky)
Attachment #569979 - Flags: review?(bzbarsky)
Comment on attachment 570453 [details] [diff] [review]
Part 1: Add UTF-8 flag to nsDocShell::LoadURI, v2

r=me
Attachment #570453 - Flags: review?(bzbarsky) → review+
Comment on attachment 569980 [details] [diff] [review]
Part 2: Use UTF-8 flag in the browser

>diff --git a/browser/base/content/tabbrowser.xml b/browser/base/content/tabbrowser.xml

>       <method name="loadOneTab">
>+        <parameter name="aIsUTF8"/>

>       <method name="addTab">
>+        <parameter name="aIsUTF8"/>

Please don't add to the actual parameter lists - just make any consumers that need to pass this parameter use the argument-object form (looks like they already all are).

>diff --git a/browser/base/content/urlbarBindings.xml b/browser/base/content/urlbarBindings.xml

>           function loadCurrent() {

>+            if (!this.valueIsTyped)
>+              flags |= Ci.nsIWebNavigation.LOAD_FLAGS_URI_IS_UTF8;

Can you add a comment explaining the reasoning here? "If the value wasn't typed, we know that we decoded the value as UTF-8 (see losslessDecodeURI)" or somesuch.

>diff --git a/browser/base/content/utilityOverlay.js b/browser/base/content/utilityOverlay.js

>@@ -194,16 +194,17 @@ function openLinkIn(url, where, params) 

>+  var aIsUTF8               = params.isUTF8;

Looks like you only implemented support for this for where=="tab" or "current" - this won't work if where=="window" (and probably "save"). That's probably not a big deal but it's worth a comment at least.

Looks good otherwise, will r+ with those addressed.
Attachment #569980 - Flags: review?(gavin.sharp) → review-
(Assignee)

Comment 22

6 years ago
Created attachment 571337 [details] [diff] [review]
Part 2: Use UTF-8 flag in the browser, v2

Resolved review comments.
> Looks like you only implemented support for this for where=="tab" or "current" - this won't work if where=="window" (and probably "save"). That's probably not a big deal but it's worth a comment at least.
Yeah, I couldn't find a way to open the non-typed url to a new window or save it from the location bar (not from the dropdown which works already). Added a comment.
Attachment #569980 - Attachment is obsolete: true
Attachment #571337 - Flags: review?(gavin.sharp)
Comment on attachment 571337 [details] [diff] [review]
Part 2: Use UTF-8 flag in the browser, v2

Looks good. Can you write a test for this?
Attachment #571337 - Flags: review?(gavin.sharp) → review+
(Assignee)

Comment 24

6 years ago
Created attachment 571557 [details] [diff] [review]
An automated test
Attachment #571557 - Flags: review?(gavin.sharp)
Comment on attachment 571557 [details] [diff] [review]
An automated test

>diff --git a/browser/base/content/test/Makefile.in b/browser/base/content/test/Makefile.in

>+                 browser_bug461304.js \

Please name the test "browser_urlbarEnter.js" instead (we'd like to discourage naming test files browser_bugXXXXXX).

>diff --git a/browser/base/content/test/browser_bug461304.js b/browser/base/content/test/browser_bug461304.js

>+let gFocusManager = Cc["@mozilla.org/focus-manager;1"].
>+                    getService(Ci.nsIFocusManager);

I think you could remove this from this test (and the associated checks), since that's already covered in other tests.

>+function locationBarEnter(aEvent, aClosure) {
>+  setTimeout(function() {

You can use executeSoon for this instead of setTimeout. But I wonder whether you need to do this here - can you do it only for the cleanup and runNextTest call that is in runNextTest? E.g.:

function runNextTest() {
...
  addPageShowListener(function() {
    locationBarEnter(test.event, function() {
      test.check(tab);

      executeSoon(function () {
        // Clean up
        while (gBrowser.tabs.length > 1)
        gBrowser.removeTab(gBrowser.selectedTab)

        runNextTest();
      });

r=me with those addressed, thanks for the test!
Attachment #571557 - Flags: review?(gavin.sharp) → review+
(Assignee)

Comment 26

6 years ago
Created attachment 571660 [details] [diff] [review]
Test, for checkin

> But I wonder whether you need to do this here - can you do it only for the
> cleanup and runNextTest call that is in runNextTest? E.g.:
Without using executeSoon here, "current tab" case unexpectedly pass the test. I don't know the reason why.
Other points are addressed.
Attachment #571557 - Attachment is obsolete: true
Attachment #571660 - Flags: review+
(Assignee)

Comment 27

6 years ago
Created attachment 571661 [details] [diff] [review]
Part 1: Add UTF-8 flag to nsDocShell::LoadURI, for checkin
Attachment #570453 - Attachment is obsolete: true
Attachment #571661 - Flags: review+
(Assignee)

Comment 28

6 years ago
Created attachment 571662 [details] [diff] [review]
Part 2: Use UTF-8 flag in the browser, for checkin
Attachment #571337 - Attachment is obsolete: true
Attachment #571662 - Flags: review+
(Assignee)

Comment 29

6 years ago
(In reply to Masatoshi Kimura [:emk] from comment #26)
> > But I wonder whether you need to do this here - can you do it only for the
> > cleanup and runNextTest call that is in runNextTest? E.g.:
> Without using executeSoon here, "current tab" case unexpectedly pass the
> test. I don't know the reason why.
"current tab" case unexpectedly pass the test even if unpatched.
Keywords: checkin-needed
https://tbpl.mozilla.org/?tree=Try&rev=41d5c15a4430
https://hg.mozilla.org/integration/mozilla-inbound/rev/2c4733dd653e
https://hg.mozilla.org/integration/mozilla-inbound/rev/5fdf8f720c7c
https://hg.mozilla.org/integration/mozilla-inbound/rev/604530cabdf6
Flags: in-testsuite+
Keywords: checkin-needed
Target Milestone: --- → Firefox 10
https://hg.mozilla.org/mozilla-central/rev/2c4733dd653e
https://hg.mozilla.org/mozilla-central/rev/5fdf8f720c7c
https://hg.mozilla.org/mozilla-central/rev/604530cabdf6
Status: ASSIGNED → RESOLVED
Last Resolved: 9 years ago6 years ago
Resolution: --- → FIXED
(Assignee)

Updated

6 years ago
Duplicate of this bug: 532872

Updated

6 years ago
Blocks: 472669

Updated

5 years ago
Blocks: 801287
You need to log in before you can comment on or make changes to this bug.