Closed Bug 461304 Opened 16 years ago Closed 13 years ago

Loading URL by pressing ENTER on already present URL in location bar doesn't maintain URL encoding

Categories

(Firefox :: Address Bar, defect)

x86
Windows XP
defect
Not set
major

Tracking

()

RESOLVED FIXED
Firefox 10

People

(Reporter: deleeuw+bugzilla, Assigned: emk)

References

()

Details

Attachments

(3 files, 5 obsolete files)

User-Agent:       Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.1b2pre) Gecko/20081022 Minefield/3.1b2pre
Build Identifier: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.1b2pre) Gecko/20081022 Minefield/3.1b2pre

When you load a page in Firefox the location bar makes some magic and "decodes" the URL, making %c3%b6 for instance appearing as "ö". All this is fine, but when I just go to the location bar and press ENTER the encoding gets screwed up.

What I think Firefox does wrong: 
It doesn't remember what encoding the page loaded had (UTF-8) and therefore just encodes it as ISO-8859-1 or something like this. It should "remember" the encoding.

Reproducible: Always

Steps to Reproduce:
1.Load the URL in the URL field, see that the textbox to the right contains "höjning"
2.Go to the location bar and press ENTER
3.See how the text box changes to h�jning
Actual Results:  
The URL encoding changed

Expected Results:  
The URL encoding stays the same
According to my colleague who has a Mac this is a Windows only error.
Version: unspecified → Trunk
I see this too, steps to reproduce
1. load http://www.stockholm.se/-/Nyheter/?q=h%C3%B6jning&inkluderalokala=False
2. Now focus the url field and press ENTER

ACTUAL RESULTS
"höjning" in url changed to "h%F6jning" which led to no search results on the page

EXPECTED RESULTS
"höjning" should not change once you press enter in the url field
Status: UNCONFIRMED → NEW
Ever confirmed: true
Flags: blocking-firefox3.1?
Gavin, Mardak: any ideas?
Presumably this happens because network.standard-url.encode-query-utf8 is false, which seems to be bug 333859. I guess this is a duplicate of that bug. Some context is in bug 261929.

IE 7 has the same bug, but doesn't automatically decode visited URIs, so it's more difficult to hit (you need to paste in the UTF-8 string). I wonder whether we should avoid decoding the query part...
Status: NEW → RESOLVED
Closed: 16 years ago
Flags: blocking-firefox3.1?
Resolution: --- → DUPLICATE
Oops, misunderstood Gavin, who's proposing that we mitigate in Firefox.

The whole point of encoding the query strings is so that when someone types something like "höjning" into their location bar, it matches what they expect. If I'm understanding this right, though, when they then find the thing that looks as expected, hitting ENTER won't have predictable results.

To me that implies that not decoding the query string part will just mean that we won't get the right sort of smart location bar matches, which is a different problem.

I think the best solution is to have this fixed by bug 333859 ...
Status: RESOLVED → REOPENED
Flags: blocking-firefox3.1?
Resolution: DUPLICATE → ---
(In reply to comment #6)
> The whole point of encoding the query strings is so that when someone types
> something like "höjning" into their location bar, it matches what they expect.
> If I'm understanding this right, though, when they then find the thing that
> looks as expected, hitting ENTER won't have predictable results.

If the site expects it's parameters to be UTF-8 encoded, and the character is in the query string, yeah.

> To me that implies that not decoding the query string part will just mean that
> we won't get the right sort of smart location bar matches, which is a different
> problem.

I don't think the awesomebar is affected by this either way, since we explicitly decode URIs there.
(In reply to comment #7)
> I don't think the awesomebar is affected by this either way, since we
> explicitly decode URIs there.

I was wrong here... we store the correctly encoded URI, assuming you reached the site by actually submitting the search or clicking a link rather than just entering the URL in the URL bar, but when you recall that entry in the awesomebar we decode it again, triggering this bug.
This isn't a regression, so as much as it pains me, it doesn't block. Mardak, can you think of anything that could be causing this on w32-only?
Flags: wanted-firefox3.1+
Flags: blocking-firefox3.1?
Flags: blocking-firefox3.1-
Looks like it's windows-only because LoadURI() takes a wstring, and somehow before that gets passed to NS_NewURI that is assumed to be in the platform native character set (which is UTF-8 on Linux and Mac)?
Target Milestone: --- → Firefox 3.1
Bug 393246 will fix this. It will have fewer side-effects than bug 333859.
Bug 393246 was backouted because of bug 432836. But I don't think bug 432836 is a valid claim because even Firefox 2 does not "work" if the system codepage is not Windows-1252.
Target Milestone: Firefox 3.1 → Firefox 3
Target Milestone: Firefox 3 → Firefox 3.5
Depends on: 393246
Target Milestone: Firefox 3.5 → ---
Flags: wanted-firefox3.5+ → wanted-firefox3.6+
Blocks: 673500
Instead of waiting for a fix for bug 393246 or bug 333859, I see the simple solution to just only decode URLs when encoding them again would lead to the same URL.

For example when we encode using the system charset, only decode URLs that are in the system charset, too. If we encode only in UTF-8, only decode when the source URL is in UTF-8, too.
Assignee: nobody → VYV03354
Status: REOPENED → ASSIGNED
Attachment #569979 - Flags: review?(Olli.Pettay)
We can be sure that the URI is encoded in UTF-8 if the value of location bar is not typed.
Attachment #569980 - Flags: review?(gavin.sharp)
Comment on attachment 569979 [details] [diff] [review]
Part 1: Add UTF-8 flag to nsDocShell::LoadURI

I'd prefer if Boris could review this since he reviewed Bug 691690.
Attachment #569979 - Flags: review?(Olli.Pettay) → review?(bzbarsky)
Comment on attachment 569979 [details] [diff] [review]
Part 1: Add UTF-8 flag to nsDocShell::LoadURI

Should that be an else if, or just an if?
Oh, I don't recall why I wrote "else if" here.
Attachment #569979 - Attachment is obsolete: true
Attachment #570453 - Flags: review?(bzbarsky)
Attachment #569979 - Flags: review?(bzbarsky)
Comment on attachment 570453 [details] [diff] [review]
Part 1: Add UTF-8 flag to nsDocShell::LoadURI, v2

r=me
Attachment #570453 - Flags: review?(bzbarsky) → review+
Comment on attachment 569980 [details] [diff] [review]
Part 2: Use UTF-8 flag in the browser

>diff --git a/browser/base/content/tabbrowser.xml b/browser/base/content/tabbrowser.xml

>       <method name="loadOneTab">
>+        <parameter name="aIsUTF8"/>

>       <method name="addTab">
>+        <parameter name="aIsUTF8"/>

Please don't add to the actual parameter lists - just make any consumers that need to pass this parameter use the argument-object form (looks like they already all are).

>diff --git a/browser/base/content/urlbarBindings.xml b/browser/base/content/urlbarBindings.xml

>           function loadCurrent() {

>+            if (!this.valueIsTyped)
>+              flags |= Ci.nsIWebNavigation.LOAD_FLAGS_URI_IS_UTF8;

Can you add a comment explaining the reasoning here? "If the value wasn't typed, we know that we decoded the value as UTF-8 (see losslessDecodeURI)" or somesuch.

>diff --git a/browser/base/content/utilityOverlay.js b/browser/base/content/utilityOverlay.js

>@@ -194,16 +194,17 @@ function openLinkIn(url, where, params) 

>+  var aIsUTF8               = params.isUTF8;

Looks like you only implemented support for this for where=="tab" or "current" - this won't work if where=="window" (and probably "save"). That's probably not a big deal but it's worth a comment at least.

Looks good otherwise, will r+ with those addressed.
Attachment #569980 - Flags: review?(gavin.sharp) → review-
Resolved review comments.
> Looks like you only implemented support for this for where=="tab" or "current" - this won't work if where=="window" (and probably "save"). That's probably not a big deal but it's worth a comment at least.
Yeah, I couldn't find a way to open the non-typed url to a new window or save it from the location bar (not from the dropdown which works already). Added a comment.
Attachment #569980 - Attachment is obsolete: true
Attachment #571337 - Flags: review?(gavin.sharp)
Comment on attachment 571337 [details] [diff] [review]
Part 2: Use UTF-8 flag in the browser, v2

Looks good. Can you write a test for this?
Attachment #571337 - Flags: review?(gavin.sharp) → review+
Attached patch An automated test (obsolete) — Splinter Review
Attachment #571557 - Flags: review?(gavin.sharp)
Comment on attachment 571557 [details] [diff] [review]
An automated test

>diff --git a/browser/base/content/test/Makefile.in b/browser/base/content/test/Makefile.in

>+                 browser_bug461304.js \

Please name the test "browser_urlbarEnter.js" instead (we'd like to discourage naming test files browser_bugXXXXXX).

>diff --git a/browser/base/content/test/browser_bug461304.js b/browser/base/content/test/browser_bug461304.js

>+let gFocusManager = Cc["@mozilla.org/focus-manager;1"].
>+                    getService(Ci.nsIFocusManager);

I think you could remove this from this test (and the associated checks), since that's already covered in other tests.

>+function locationBarEnter(aEvent, aClosure) {
>+  setTimeout(function() {

You can use executeSoon for this instead of setTimeout. But I wonder whether you need to do this here - can you do it only for the cleanup and runNextTest call that is in runNextTest? E.g.:

function runNextTest() {
...
  addPageShowListener(function() {
    locationBarEnter(test.event, function() {
      test.check(tab);

      executeSoon(function () {
        // Clean up
        while (gBrowser.tabs.length > 1)
        gBrowser.removeTab(gBrowser.selectedTab)

        runNextTest();
      });

r=me with those addressed, thanks for the test!
Attachment #571557 - Flags: review?(gavin.sharp) → review+
> But I wonder whether you need to do this here - can you do it only for the
> cleanup and runNextTest call that is in runNextTest? E.g.:
Without using executeSoon here, "current tab" case unexpectedly pass the test. I don't know the reason why.
Other points are addressed.
Attachment #571557 - Attachment is obsolete: true
Attachment #571660 - Flags: review+
Attachment #570453 - Attachment is obsolete: true
Attachment #571661 - Flags: review+
Attachment #571337 - Attachment is obsolete: true
Attachment #571662 - Flags: review+
(In reply to Masatoshi Kimura [:emk] from comment #26)
> > But I wonder whether you need to do this here - can you do it only for the
> > cleanup and runNextTest call that is in runNextTest? E.g.:
> Without using executeSoon here, "current tab" case unexpectedly pass the
> test. I don't know the reason why.
"current tab" case unexpectedly pass the test even if unpatched.
Keywords: checkin-needed
Blocks: 472669
Blocks: 801287
You need to log in before you can comment on or make changes to this bug.