85025 - DNS lookup stalls b/c failed requests are cached

Status: VERIFIED FIXED

(Core :: Networking: HTTP, defect, P3)

Description

[pg]

From Bugzilla Helper:
User-Agent: Mozilla/5.0 (X11; U; Linux 2.4.4 i686; en-US; rv:0.9.1+) Gecko/20010609
BuildID:    2001060906

Somtimes but often, I can not access the web site www.bbc.co.uk & 
news.bbc.co.uk BUT the ip address aways works (http://212.58.226.40/)
and Konqueror (2.1.1) does not seem to have this problem

Reproducible: Sometimes
Steps to Reproduce:
1.select http://news.bbc.co.uk/
2.You may get the error  news.bbc.co.uk could not be found 
3.then try the ip address http://212.58.226.40/ is should works ok
4.also if you try Konqueror it does not seem to have this problem


Actual Results:  error message news.bbc.co.uk could not be found 

Expected Results:  to access the site

Comment 1

[Emlyn Shannon]

I can't reproduce this problem, and I've never experienced anything even vaugely
like it. Might help if you gave was more information about your ISP or Proxy.

Comment 2

[James Green]

Reporter: the message, <site> cannot be found, is usually a result of a
temporary network or server hiccup, and almost never a result of browser
software failure. If it was something in our code, lots of people would be
seeing it.

I use news.bbc.co.uk several times a day and don't see problems. Marking
WORKSFORME. If you are able to reproduce this problem reliably, please reopen
this bug and report exactly how you are able to reproduce such that an engineer
can reliably repeat it. Many thanks.

Comment 3

[R.K.Aa.]

i see this quite frequently. There's some hickup in netwerk that makes a first
request get stuck. More often the beheaviour is that i have to click a link
twice. Then statusbar will read "connecting to" OR "Resolving host". And if i
tried to open the link in a new window, i only see throbber spin and
"about:blank" in statusbar.  Nothing more happens till i click again or hit
enter in url-bar. Seems it's all part of the same problem, and probably also the
reason a reload sometimes behave weird. (bug 82079)

I commented about this all in bug 82181, which is marked fixed. Timewise, i
noticed this behaviour around the fix for bug 80975, but that may be coincidental.

Some bugs that may be the real bug are bug 84019, bug 84817

Comment 4

[pg]

Yes, I think maybe there is a small nework error, on my side, 
but it seems that once mozilla caches the first attempt as bad,
it either doesn't try another DNS lookup and just relys on its DNS cache.

These error message migh give you a clue:

I am inside the initialize
 Hey : You are in QFA Startup 
(QFA)Talkback loaded Ok.
Registering plugin 0 for: "*","All types",".*"
Document http://www.mozilla.org/ loaded successfully
Document http://www.bbc.co.uk/ loaded successfully
Error loading URL http://www.bbc.co.uk/news/ : 2152398878
Error loading URL http://www.bbc.co.uk/news/ : 2152398878
Error loading URL http://www.bbc.co.uk/news/ : 2152398878
Error loading URL http://www.bbc.co.uk/news/ : 2152398878
Error loading URL http://www.bbc.co.uk/news/ : 2152398878
Error loading URL http://www.bbc.co.uk/news/ : 2152398878
Error loading URL http://www.bbc.co.uk/news/ : 2152398878
Hey : You are in QFA Shutdown 
[root@cable-213-132-137-20 /etc]# 



As a test I have now entered news.bbc.co.uk in my /etc/hosts, so maybe this,
will solve my problem.

[root@cable-213-132-137-20 /etc]# /usr/local/mozilla/mozilla
/usr/local/mozilla/run-mozilla.sh /usr/local/mozilla/mozilla-bin
MOZILLA_FIVE_HOME=/usr/local/mozilla
  LD_LIBRARY_PATH=/usr/local/mozilla:/usr/local/mozilla/plugins
     LIBRARY_PATH=/usr/local/mozilla:/usr/local/mozilla/components
       SHLIB_PATH=/usr/local/mozilla
          LIBPATH=/usr/local/mozilla
       ADDON_PATH=/usr/local/mozilla
      MOZ_PROGRAM=/usr/local/mozilla/mozilla-bin
      MOZ_TOOLKIT=
        moz_debug=0
     moz_debugger=
 I am inside the initialize
 Hey : You are in QFA Startup 
(QFA)Talkback loaded Ok.
Registering plugin 0 for: "*","All types",".*"
Document http://www.mozilla.org/ loaded successfully
Document http://news.bbc.co.uk/ loaded successfully
Document http://www.mozilla.org/ loaded successfully


 

Comment 5

[James Green]

I have seen the initial lookup stalls problem, which may be what pg is
describing, but I don't recal having had it time out with an error (possibly mye
cancelling the request and restarting the page load).

My sister has also encountered the problem several times whilst visiting an
A-Level resision site, something like www.e-cool.com (don't recall the name
exactly).

OK Reopening based on the observations that sometimes a request fails.
Unfortunately I cannot be asked if this is reproducable since I have noticed
that my ISP's caching DNS servers can sometimes be slow to respond; anyone else
having a problem with dns only in mozilla?

Comment 6

[pg]

Assuming my DNS is slightly troublesome, which may also be the root cause of
this, problem:


User-Agent: Mozilla/5.0 (X11; U; Linux 2.4.4 i686; en-US; rv:0.9.1+) Gecko/20010609
BuildID:    2001060906

Somtimes but very often, I have to press enter or go twice, (reload does not
work) to load the site www.itn.co.uk. 

Reproducible: Sometimes
Steps to Reproduce:
1.Start with a up a new browser (it appears to make this problem worse)
2 you should see your orignal web site
3.type http://www.itn.co.uk
4.You will get a slight transfer, but nothing will be displayed
5.Check out the view source - you will see the source has changed an now has 
the HTML title "ITN Home page"!!, but you will still see your orginal page.
6. hit enter or "go" again and the page will be now be fully downloaded and
displayed.

Actual Results: www.itn.co.uk not display on first try

Expected Results: www.itn.co.uk should be displayed on the first try


Probably connect to my DNS problem and similar to the click a link twice probelm
mentioned above  Why the DNS problem? - because this site trys to load ads from
another URL.

The commadn line output says two sucessfull loads, but in reality only one
get displayed.
Document http://www.itn.co.uk/ loaded successfully
Document http://www.itn.co.uk/ loaded successfully

Comment 7

[R.K.Aa.]

*** Bug 85037 has been marked as a duplicate of this bug. ***

Comment 8

[R.K.Aa.]

Importing comments from the dup:
linux 2001060905

Often i have to click a link twice to make moz load a page.
It's somewhat intermittent, but a pretty reliable approach is this:

fill in counter.li.org in url-bar - hit enter
you get to http://counter.li.org
Let page load, then click link on top named "Statistics"

More often than not, Mozilla will now just spin the throbber, and status-bar
reads "Resolving host" or "Connecting to". It can seemingly spin forever - it
won't load anything till i click the link once more.


Another sample is when submitting queries in bugzilla. The first time i hit the
"Submit Query" button, nothing happens. "Resolving host" or "Connecting to.." is
all i see. When button is clicked once more, the query is submitted.

It may be be cache related. I've been seeing this on and off in all builds - CVS
and official - since bug 81764. In bug 82181, now marked fixed,
[e-mail] comments he still sees this too.

Both those bugs are now considered fixed, but something remains wrong.

A fixes that went in around the time this bug appeared was for bug 72507
(checked in on the 18th). I believe I first noticed this bug on builds from May
19th.

Comment 9

[R.K.Aa.]

Also, when loading a page within same site i've just been to, often in new
window, status bar reads "Connected to".. but nothing loads.

Comment 10

[R.K.Aa.]

It seems sockets aren't being reused, dns caching fails, and lookups stall.
Almost like these features are knocking heads and then decide to do nothing.
I also noticed that while loading a page in one window and opening a new blank
window, the url i ask the second window to go in some cases won't even start
loading before i click stop in the first window.

This happens on occations when moz has trouble loading/finding one of multiple
small images to load in the first window.

All in all networking feels pretty flaky these days.

Comment 11

[R.K.Aa.]

*** Bug 85085 has been marked as a duplicate of this bug. ***

Comment 12

[Bradley Baetz (:bbaetz)]

If you get to a stage where mozilla won't load anything, from any site, without
restarting, then that is bug 77072 (which should probably be retitled). The
"lots of images" bit does point to that bug. (The further bug is that the number
of connections per server is currently taken to be number of http connections in
total. darin has a fix for that bug)

reporter: DNS caching was added last week - do you see this problem in 0.9.1?
(which branched before that landed)

Note that the status bar saying that we're currently resolving the host name
doesn't mean that thats whats actually happening. That text will stay there
until something else updates it, which may not be for a while.

Comment 13

[R.K.Aa.]

This is not bug 77072.

Comment 14

[neeti]

cc'ing darin and gordon

Comment 15

[neeti]

-->darin

Comment 16

[benc]

I don't think the bugs marked as dupes are the same problem.

Lets go back to talking about the original steps that caused the problem.

If someone starts the browser, and types a URL that is not in cache and they get
a DNS error when they shouldn't, then you should start with the assumption that
caching, connection problems are not related, unless you do not trust the DNS
error returned.

I tried to access the bbc site via nslookup on a PacBell DNS server, and it
timed out the first time, but worked the second time. This is probably because
my request to the DNS server timed out, but the DNS server's attempt to scan the
DNS tree and get the data was successful.

It's been a long time since I read "DNS and BIND", so I'm not sure if there is
client behavior we could add for correctness, but that seems to me what the real
problem is here.

If you don't get a DNS error, I think that is a connection/cache problem, and it
should go elsewhere.

Comment 17

[doctor__j]

I got some reproducible steps (for Dial-up networking fellas like myself):

1. Disconnected from the Internet. (I mean hang-up the sloooow analog modem.)
2. Start Mozilla.
3. Launch an URL.
4. You got the error message "<URL> could not be found. Please check the name
and try again."
5. Dial-up to your ISP and get connected to the Internet.
6. Launch the same URL again.
7. You *still* got the same error message, although you are connected to the
Net.  Launching other URLs *will* work.

This kind of behaviour has been addressed in Linux platform in bug 63564.  And
that bug 63564 was marked as WONTFIX.

Comment 18

[Darin Fisher]

-> gordon, since this sounds like a problem with the DNS cache.

Comment 19

[gordon]

It looks like we're caching DNS errors as well as good responses.  I'll submit a 
patch to change that.

Comment 20

[R.K.Aa.]

*** Bug 85473 has been marked as a duplicate of this bug. ***

Comment 21

[benc]

Just so I understand, we are going to not cache failures + not cache negative
DNS entries right?

Comment 22

[dylang]

To whomever marked my bug report a dupe of a DNS problem, you should understand
that I use a proxy.  Mazilla shouldn't be doing any DNS locally since the proxy
is supposed to handle everything!  Mozilla stalling on image transfers with an
HTTP proxy is something else entirely.

Comment 23

[pg]

Reproducible: Always

An interesting test is as follows:

1) edit /etc/nsswitch.conf and set "host file" (no DNS)
2) edit /etc/host and put an invalid ip address for www.cnn.com
212.58.240.256 www.cnn.com
3) Start mozilla and access the url www.cnn.com (node not found as expected)
4) whilst mozilla is still running edit and set the www.cnn.com to its correct
address
207.25.71.22 www.cnn.com
5) mozilla will not see this change and keep reporting www.cnn.com as not found.

I think this something similar is happening in my case (a very short temporary
DNS problem) and in the case of the dial up access, disconecting and reconecting.

Once mozilla decides that a node is non-accessable, there nothing
the user can do get it to do another DNS lookup, either using the
reload button or re-entering the URL.

The error message are similar to the ones above
Error loading URL http://www.cnn.com/ : 2152398878
Error loading URL http://www.cnn.com/ : 2152398878
Error loading URL http://www.cnn.com/ : 2152398878
Error loading URL http://www.cnn.com/ : 2152398878


(Also it is quite interesting to see what happen as mozilla try to load display
from the local disk cache, when the node is inaccessable is appear to screw up
and give a blank page, but if you view the source you can see code from www.cnn.com)

Comment 24

[gordon]

Attachment: patch to evict dns entries that returned "host not found".

Comment 25

[gordon]

Attachment: patch to evict "host not found" dns entries (with fix for Mac)

Comment 26

[benc]

Okay, we were drifting for a while, but I think this is back on track, so I've
changed the summary.

I also reopened the bug 85037 b/c it wasn't a dupe.

Comment 27

[Darin Fisher]

gordon: looks like this patch has the patch for bug 84817 in it.

otherwise, sr=darin

Comment 28

[Asa Dotzler [:asa]]

a= [e-mail] for checkin to the trunk.
(on behalf of drivers)

Comment 29

[gordon]

Patch checked in. Marking FIXED.

Comment 30

[pg]

From Bugzilla Helper:
User-Agent: Mozilla/5.0 (X11; U; Linux 2.4.4 i686; en-US; rv:0.9.1+) Gecko/20010614
BuildID: 2001061414

This build just hangs (I have to contrl-c) and crashes (i have used the crash
feedback) for me on www.bbc.co.uk and for news.bbc.co.uk

Comment 31

[[e-mail]]

Probably fixed in bug 85802.

If you are still crashing w/ newer builds from this week, please file NEW bug.

Comment 32

[Sivakiran Tummala]

verified

Comment 33

[dylang]

I just want to mention bug 92542 .. it is much eviler form of this in that the
cache seems to use the successful lookup of another domain as the IP to query,
rather than the new domain.  This makes all recent builds (past May) useless for
me.  Could this fix have broken it?

Comment 34

[gordon]

Is this domain that is getting its IP changed yours?  How often are you
switching the IP?

Comment 35

[dylang]

No, it's not about my domain.  It's that the networking code is incorrectly caching.

Example: load Slashdot.org.  Then hit ctrl+n.  Then type in
"bugzilla.mozilla.org."  Then you get "Ho-hum" because Mozilla used the IP of
images.slashdot.org, instead of doing a DNS lookup on bugzilla.mozilla.org.

So you wait a minute, then hit reload.  And you get Bugzilla.  Then you go back
to Slasdot and load one of its MLP.  And you get a 404 error from the Apache
server on port 80 of mothra.mozilla.org.

So, no, not some dynDNS issue.  A legit Networking bug which makes all recent
builds useless because I generate new windows faster than the new cache bug
takes te expire.

Comment 36

[gordon]

This behavior sound very odd.  I can't reproduce it.  Perhaps you should open
another bug for it, because I don't think it is related to this bug (which is
closed).

Comment 37

[dylang]

Grrr!  My original posting here was about the damned bug which I filed a report
for!  It is very easy to reproduce on my setup (Linux behind a proxy), so
perhaps it is an issue with the proxy code.  In any case, as I said 24 hours ago
in a post to this bug:

"I just want to mention bug 92542 .."

I have filed it as bug 92542 -- bug 92542 !  I hate the bug, it causes me to not
be able to use one of the more recent (and faster) builds.