131193 - URL that returns proxy error page remembered in urlbar autocomplete list

Status: NEW

(Core :: DOM: Navigation, defect)

Description

[Georg Wild]

From Bugzilla Helper:
User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:0.9.9+) Gecko/20020313
BuildID:    2002031321

When you use a proxy (squid) and type a wrong URL, this is recorded in
autocomplete list, because the proxy sends a "real page" that the DNS could not
be resolved and so on.

Reproducible: Didn't try
Steps to Reproduce:
1.Use proxy
2.type wrong URL
3.Proxy error message page displayed


Actual Results:  URL in autocomplete

Comment 1

[Paul Wyskoczka]

--> networking

Comment 2

[Boris Zbarsky [:bzbarsky]]

How do we tell that the URL did not lead to a correct pageload?  Shouldn't the
proxy be propagating the http error code back to us?

Comment 3

[Peter Trudelle]

->component

Comment 4

[Bradley Baetz (:bbaetz)]

The error-code stuff nsILinkChecker does should be a boolean attribute on the
channel, I think. Not only would that simplify the link checker, but it would
fix this bug, too.

The simple fix of a QI to nsIHttpChannel and checking for certain response codes
isn't really flexable.

darin, akk?

Comment 5

[Boris Zbarsky [:bzbarsky]]

Whoa!  What does this have to do with the link checker?  Resummarizing to make
it clear what the issue is.

The problem as I see it is that we add a url to urlbar autocomplete iff the url
load succeeds.  Now this proxy apparently returns an error page with a success
status or something like that...  More details on the exact behavior of the
proxy would be nice.  Georg, could you please set the environment variable
"NSPR_LOG_MODULES" to "nsHttp:5", set the environment variable "NSPR_LOG_FILE"
to /tmp/httplog, start up mozilla, load up only a single URL (one that exhibits
the undesired behavior), close mozilla, and attach the log file to this bug?

Comment 6

[Georg Wild]

Attachment: Log described in comment

Comment 7

[Georg Wild]

Attachment: I hope now the right

Attachment 1 [details] [diff] is rubbish

Comment 8

[Georg Wild]

Sorry, the first attachment I uploaded is rubbish - but I can't delete it! Maybe
someone with more power can do it!

A remark to the problem: The "non existing" site goes also to history, but I
can't use "Save as" to save!

Comment 9

[Boris Zbarsky [:bzbarsky]]

1026[80e91f0]: http response [
1026[80e91f0]:   HTTP/1.0 503 Service Unavailable
1026[80e91f0]:   Mime-Version: 1.0
1026[80e91f0]:   Date: Sat, 16 Mar 2002 16:37:50 GMT
1026[80e91f0]:   Content-Type: text/html
1026[80e91f0]:   Content-Length: 856
1026[80e91f0]:   Expires: Sat, 16 Mar 2002 16:37:50 GMT
1026[80e91f0]:   X-Squid-Error: ERR_DNS_FAIL 0
1026[80e91f0]:   X-Cache: MISS from Iserver.WORKGROUP
1026[80e91f0]:   Proxy-Connection: keep-alive
1026[80e91f0]: ]

We should not be adding to history given that response....

Comment 10

[Bradley Baetz (:bbaetz)]

Err, no, thats my point. The status codein OnStartRequest is a sucess code,
since there is data. The underlying http response code is the issue - the code
has to QI to nsIHTppChannel, get the repsonse code, and then work out what it
means, exactly like the link checker and serveral other places.

Or we could make it a generic attribute on nsIChannel before that freezes.

Comment 11

[Darin Fisher]

bbaetz: while i agree it would be nice to better abstract the determination of
errors, bz's point i think is that nsURIChecker should be treating 503 as an
error, and should work as is.  so, what explains this bug?  perhaps we're
barking up the wrong tree.

Comment 12

[benc]

I think Ari Luotonen said to me once that 5xx class errors are not mean to be
treated as content, except that they have a response w/ html for the purposes of
displaying an error.

If they were, a proxy would return a 4xx error with the same stuff, if was
intended to be both a displayed error and treated as other content.

Comment 13

[Jesse Ruderman]

This bug depends on bug 66049, "erroneous urls still try to autocomplete".  It
might also depend on bug 2875, "Proxy: map HTTP 500 errors to necko errors (so
Internet Keywords and Domain Guessing would work)".

Comment 14

[benc]

-> URL bar.

If URL bar can see the 5xx error code, then it should make some smart choices.

From the discussion in bug 66049, this doesn't even work for direct connections.

I think you should have this URL in autocomplete, because it could be a
transient network error that the proxy had to send back.

Comment 15

[Andrew Schultz]

Erm, this bug can't be fixed in browser because the browser never gets a failed status.  Try (enable dump() first and) loading http://www.mozilla.org/sdfasdfg.  You get a 404 page and in the console, you'll see:

Document http://www.mozilla.org/sdfasdfg loaded successfully
That's with a 404 page.  I don't have a 5xx URL to test with, but from the discussion in this bug it's no different.

Comment 16

[Christian :Biesinger (don't email me, ping me on IRC)]

? it can be fixed - necko has nsIHttpChannel.requestSucceeded for a reason.

Comment 17

[Boris Zbarsky [:bzbarsky]]

So where does the autocomplete list actually stored?  Where do we add entries to it?

Comment 18

[Brett Wilson]

(In reply to comment #17)
> So where does the autocomplete list actually stored?  Where do we add entries
> to it?

The autocomplete list just comes from global history. This includes all the pages you would see in your history sidebar. Things get added when the docshell calls AddURI. I don't think it's terribly picky about what to add.

Comment 19

[Boris Zbarsky [:bzbarsky]]

Ah.  And not adding error pages to global history is not an option, since that would break visited link coloring...

Sounds like we want to be storing a bit more information in global history here, perhaps?

Comment 20

[Andrew Schultz]

The autocomplete list doesn't just come from history (sort of).  URLs also get added to global history via MarkPageAsTyped
http://lxr.mozilla.org/seamonkey/search?string=markpageastyped

If the URL the user typed isn't already in history, it's added but with a visited count of 0 (so it doesn't show up if you just look at the history).  Autocomplete normally only shows typed URLs (SeaMonkey has a pref).  For bug 9203/bug 66049 I was attempting to delete typed bogus URLs from history (the fix takes advantage of the fact that bogus URLs are always "unvisited").

AddURI could include the status and history could remember whether each URI was successfully loaded the last time or not, but nsDocLoader's onStateChange also needs a failed status so that chrome can attempt to remove the bogus typed URL (which isn't always the eventual URI due to URI fixup).

Comment 21

[Boris Zbarsky [:bzbarsky]]

OnStateChange gets passed the channel, no?  So chrome can check whether it's an HTTP channel and if so whether it succeeded...

Comment 22

[Brett Wilson]

(In reply to comment #20)
> The autocomplete list doesn't just come from history (sort of).  URLs also get
> added to global history via MarkPageAsTyped

This is not really correct. When you call MarkPageAsTyped on a new URL, it will get added to history as typed, but also marked hidden. When you visit a toplevel page that is marked hidden, it will get unhidden. Hidden pages will not be returned by AutoComplete or displayed in history:
http://lxr.mozilla.org/seamonkey/source/toolkit/components/history/src/nsGlobalHistory.cpp#4381
Also, as far as I can tell, browser.urlbar.matchOnlyTyped in Firefox is defaulted to false (I don't think this affects the problem at all).

This will probably be replaced by nsNavHistory.cpp in browser/components/places in v2.0, so a patch to nsGlobalHistory may not be very useful. What is really needed is a flag to pass to history to keep it hidden weven when you call AddURI. I'm working on a some expanded history API functions, we could pass some enum indicating success or different types of failure.

Comment 23

[Andrew Schultz]

> OnStateChange gets passed the channel, no?  So chrome can check whether it's 
> an HTTP channel and if so whether it succeeded...

sure, but for 4xx URLs, the channel claims to have succeeded.

SeaMonkey shows the "hidden" URIs in the autocomplete dropdown.  Otherwise, javascript (and maybe another kind) URIs would not be shown.

> toplevel page that is marked hidden, it will get unhidden. Hidden pages will
> not be returned by AutoComplete or displayed in history:
http://lxr.mozilla.org/seamonkey/source/toolkit/components/history/src/nsGlobalHistory.cpp#4381

if the URI was typed, how would it get to line 4381?

Comment 24

[Boris Zbarsky [:bzbarsky]]

> sure, but for 4xx URLs, the channel claims to have succeeded.

I'm not talking about the nsresult status that OnStateChange gets, nor the nsIRequest.status property on the channel.  I'm talking about nsIHttpChannel.requestSucceeded (see comment 16).  You need to QI the channel to nsIHttpChannel, of course.

Comment 25

[AndrewM]

Is this bug still a problem after the switch to SQLite and Places?